Primary and secondary school body forward flexion distance measurement method and device, storage medium and program product

Abstract

The invention provides a primary and secondary school body forward flexion distance measurement method and device, a storage medium and a program product, and relates to the technical field of computer vision, and the method comprises the steps: training an initial detection model through first image data, and obtaining a part detection model; training the initial hand recognition model through the first and second image data to obtain a target hand recognition model; when a plurality of target objects need body forward flexion motion, obtaining body forward flexion posture detection results of the plurality of target objects through the detection model; when it is determined that at least one second object is in a second body forward bending posture according to the detection result, obtaining a sub-part detection result of the at least one second object through the detection model; and according to the detection result and the identification model, carrying out body forward bending distance measurement on at least one second object. The technical problem of low accuracy of body forward flexion distance measurement in a multi-human body forward flexion scene in the prior art is solved, and the technical effect of improving the accuracy of body forward flexion distance measurement in the multi-human body forward flexion scene is achieved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, device, storage medium and program product for measuring distance between forward flexion of primary and secondary schools. Background Art

[0002] With the growing demand for refined assessment of students' physical health in modern education and the gradual popularization of digital and intelligent educational methods, the importance of accurate and efficient forward flexion ranging technology in physical education and health monitoring in primary and secondary schools has become increasingly prominent.

[0003] In the related art, a simple mechanical distance measuring device is usually used. The mechanical distance measuring device consists of a fixed measuring track and a slidable measuring pointer. When the student performs the forward bending movement, the finger pushes the measuring pointer to slide on the measuring track, and the measuring track is marked with a scale in centimeters. The scale display method is relatively simple, and only the digital mark on the measuring track is used to indicate the distance. There is no design for auxiliary reading. In the case of dim light, it may be difficult for the teacher to read the scale value. In a humid environment, the measuring track and the measuring pointer are prone to rust. Rust will increase the friction between the measuring pointer and the measuring track, which not only affects the smoothness of the sliding of the measuring pointer, but also may cause the measuring pointer to jam during the sliding process, thereby affecting the accuracy of the measurement result. In terms of posture detection, basic pressure sensor array technology is used. Multiple pressure sensors are installed on the ground of the test area according to a specific layout, and the pressure forms a sensor array. When the student stands in the specified position, the feet generate pressure on the ground pressure sensor. The sensor determines whether the position of the student's feet is within the standard test range based on the pressure distribution. During the forward bending process, the change in the center of gravity of the body causes the pressure value of the pressure sensor to change. By analyzing the dynamic change of the pressure value, it is determined whether the student's body remains stable. However, the detection principle of the pressure sensor array is mainly based on pressure perception, and posture changes such as knee bending and back straightening may not directly and significantly change the pressure distribution detected by the pressure sensor. Knee bending is mainly a change in joint angle, and whether the back is straight or not is more reflected in changes in body contour and angle. Such posture changes may not be accurately captured by ground pressure sensors.

[0004] However, when using the above method, during large-scale multi-person forward flexion tests, the simple mechanical ranging device is easily disturbed by environmental factors, and problems such as rust and dim light occur frequently, resulting in the continuous accumulation of measurement errors. At the same time, the pressure sensor array cannot accurately detect key posture details, which increases posture misjudgments, and thus leads to a low accuracy rate of forward flexion ranging in multi-person forward flexion scenarios in related technologies. Summary of the invention

[0005] The present application provides a method, device, storage medium and program product for measuring forward flexion distance in primary and secondary schools, which are used to improve the accuracy of forward flexion distance measurement in a multi-person forward flexion scenario.

[0006] In a first aspect, the present application provides a forward flexion ranging method for primary and secondary schools, which is applied to the above-mentioned electronic device, and the method includes: when first image data is acquired, an initial detection model is trained according to the first image data to obtain a part detection model, wherein the first image data includes human body position image data, hand position image data, knee position image data, and foot position image data when multiple first objects are in a first forward flexion posture, and the first forward flexion posture is a forward flexion posture displayed by multiple first objects included in the first image data; when second image data is acquired, an initial hand recognition model is trained according to the first image data and the second image data to obtain a target hand recognition model, wherein the second image data includes hand image data when multiple first objects are in the first forward flexion posture; In the case where it is determined that multiple target objects need to perform forward flexion movement, forward flexion posture detection is performed on the multiple target objects using a part-by-part detection model to obtain a forward flexion posture detection result, wherein the multiple first objects include the multiple target objects; in the case where it is determined that at least one second object among the multiple target objects is in a second forward flexion posture according to the forward flexion posture detection result, part-by-part detection is performed on the at least one second object using the part-by-part detection model to obtain a part-by-part detection result, wherein the multiple target objects include at least one second object, and the second forward flexion posture is a forward flexion posture displayed by at least one second object determined after forward flexion posture detection is performed on the multiple target objects using the part-by-part detection model; a forward flexion ranging operation is performed on the at least one second object according to the part-by-part detection result and the target hand recognition model.

[0007] By adopting the above technical solution, first image data containing image data of multiple parts of multiple first objects in forward bending postures is obtained, and the initial detection model is trained to obtain a part detection model, laying the foundation for posture detection. The initial hand recognition model is then trained in combination with the second image data to obtain a target hand recognition model. Later, when it is determined that multiple target objects need to perform forward bending movements, the part detection model is used to perform posture detection on them. Such a process, from data collection, model training to actual detection, gradually builds a complete forward bending ranging system, providing a systematic solution for accurate detection of forward bending postures and ranging, thereby improving the accuracy and reliability of forward bending ranging. This solves the technical problem of low accuracy of forward bending ranging in multi-person forward bending scenarios in related technologies, and achieves the technical effect of improving the accuracy of forward bending ranging in multi-person forward bending scenarios.

[0008] Optionally, when it is determined that multiple target objects need to perform forward flexion movement, forward flexion posture detection is performed on the multiple target objects using a part-by-part detection model to obtain forward flexion posture detection results, specifically including: obtaining a motion image sequence of multiple target objects acquired in real time by a video acquisition device; preprocessing each frame of the motion image sequence to obtain a preprocessed image sequence; inputting the preprocessed image sequence into the part-by-part detection model, so that the part-by-part detection model detects the first human body detection frame, the first hand detection frame, the first knee detection frame and the first foot detection frame of each target object in the multiple target objects respectively; judging whether each target object is in the second forward flexion posture according to the first vertical distance between the first hand detection frame and the first foot detection frame; if the first vertical distance is continuously greater than a preset vertical threshold, and the first horizontal offset of the knee detection frame is less than the preset horizontal offset threshold, it is judged to be the second forward flexion posture; if at least one second object among the multiple target objects is judged to be in the second forward flexion posture, then the forward flexion posture detection result is output as valid, otherwise it is output as invalid.

[0009] By adopting the above technical solution, in the forward bending posture detection link, the motion image sequence of the target object is first obtained, and then each frame of the image is preprocessed to remove interference information such as noise and improve the image quality. The preprocessed image sequence is input into the part detection model, and the part detection model can detect the detection frame of each part. According to the first vertical distance between the first hand detection frame and the first foot detection frame and the first horizontal offset of the knee detection frame, it is judged whether it is the second forward bending posture. Through the detection and judgment of these specific features, it is possible to accurately identify whether multiple target objects are in a forward bending posture that meets the requirements, provide accurate posture judgment results for subsequent part detection and ranging, and ensure the accuracy of the starting link of the entire ranging process.

[0010] Optionally, when it is determined that at least one second object among multiple target objects is in a second forward flexion posture according to the forward flexion posture detection result, a part-by-part detection is performed on the at least one second object using a part-by-part detection model to obtain a part-by-part detection result, specifically including: establishing a target object list according to the at least one second object, and recording a unique identifier of the at least one second object; determining the first position of the first current frame image of the at least one second object in the motion image sequence; obtaining a first image sequence including the first current frame image and N first adjacent frame images before and after the first current frame image according to the first position, wherein N is a positive integer greater than or equal to 1; inputting the first image sequence and the target object list into the part-by-part detection model, so that the part-by-part detection model performs a multi-scale feature extraction operation on the first image sequence and the target object list; when it is determined that the part-by-part detection model has completed the multi-scale feature extraction operation, obtaining the part-by-part detection result output by the part-by-part detection model.

[0011] By adopting the above technical solution, when it is determined that at least one second object is in the second forward bending posture, a target object list is first established and a unique identifier is recorded to facilitate tracking of specific objects. After determining the first position of the first current frame image of at least one second object, an image sequence consisting of N frames before and after the first current frame image is obtained, and the image sequence is input into the sub-part detection model to perform a multi-scale feature extraction operation. Multi-scale feature extraction can extract object features from different resolutions, comprehensively and meticulously describe the object, and enable the sub-part detection model to have a deeper understanding of at least one second object. The sub-part detection results are obtained, and the sub-part detection results include detailed part feature information of the object, providing key data support for the subsequent accurate execution of the forward bending ranging operation.

[0012] Optionally, a forward flexion ranging operation is performed on at least one second object according to the part-by-part detection results and the target hand recognition model, specifically including: when it is determined according to the part-by-part detection results that the hand of at least one second object is not detected, determining a second human body detection frame and a first hand motion trajectory of at least one second object in a second current frame image from the part-by-part detection results, wherein the preprocessed image sequence includes the second current frame image, and the first human body detection frame includes a second human body detection frame; determining a first position prediction value of the hand of at least one second object according to the second human body detection frame and the first hand motion trajectory; determining a second position of the hand of at least one second object according to the first position prediction value, and adding a second hand detection frame at the second position; inputting the at least one second object to which the second hand detection frame has been added into the target hand recognition model; obtaining a hand detection confidence output by the target hand recognition model after performing hand recognition detection on the at least one second object to which the second hand detection frame has been added; and performing a forward flexion ranging operation on the at least one second object according to the hand detection confidence.

[0013] By adopting the above technical solution, in the forward flexion ranging operation, if the part detection result shows that the hand is not detected, first determine the second human body detection frame and the first hand motion trajectory to predict the hand position. After adding the second hand detection frame, input the target hand recognition model to obtain the hand detection confidence. The hand detection confidence can reflect the reliability of the target hand recognition model for the hand detection results. Through subsequent operations based on the hand detection confidence, when there is a problem with the hand detection, the hand position can be accurately determined by using model prediction and confidence judgment, which provides the possibility for forward flexion ranging, thereby improving the feasibility and accuracy of forward flexion ranging in complex detection situations.

[0014] Optionally, a forward flexion ranging operation is performed on at least one second object according to the hand detection confidence, specifically including: performing a confidence comparison on the hand detection confidence and a preset confidence threshold to obtain a confidence comparison result; when it is determined according to the confidence comparison result that the hand detection confidence is greater than or equal to the preset confidence threshold, marking the first hand detection frame as a valid hand detection frame; determining the first spatial coordinates of the second knee detection frame of at least one second object, and determining the second spatial coordinates of the second foot detection frame of at least one second object, wherein the first knee detection frame includes the second knee detection frame, and the first foot detection frame includes the second foot detection frame; determining the forward flexion ranging value of at least one second object according to the first position prediction value, the first spatial coordinates, and the second spatial coordinates.

[0015] By adopting the above technical solution, the hand detection confidence is compared with the preset confidence threshold to determine whether the detection result is reliable. If the hand detection confidence meets the standard, the valid hand detection frame is marked, and the first spatial coordinates of the second knee detection frame and the second spatial coordinates of the second foot detection frame are determined. The forward flexion distance measurement value is determined using these spatial coordinates and the hand position prediction value. Through this method based on confidence screening and coordinate calculation, a more accurate forward flexion distance measurement value can be calculated using accurate coordinate information while ensuring the reliability of hand detection, thereby improving the accuracy and credibility of the forward flexion distance measurement results.

[0016] Optionally, after performing a confidence comparison between the hand detection confidence and a preset confidence threshold to obtain a confidence comparison result, the method also includes: when it is determined according to the confidence comparison result that the hand detection confidence is less than the preset confidence threshold, determining a third human body detection frame and a second hand motion trajectory of at least one second object in the second adjacent frame image from the part detection result, wherein the second adjacent frame image is an image included in the preprocessed image sequence and adjacent to the second current frame image, and the first human body detection frame includes a third human body detection frame; determining a second position prediction value of the hand of at least one second object according to the third human body detection frame and the second hand motion trajectory; determining a third position of the hand of at least one second object according to the second position prediction value; adjusting the first position prediction value to a third position prediction value according to the third position, and determining a forward flexion ranging value of at least one second object according to the third position prediction value, the first spatial coordinate and the second spatial coordinate.

[0017] By adopting the above technical solution, when the hand detection confidence is lower than the preset confidence threshold, the adjacent frame image information is obtained from the part detection result, and the hand position prediction value (corresponding to the second position prediction value) is re-determined. The original prediction value (corresponding to the first position prediction value) is adjusted according to the new position (corresponding to the third position), and the forward flexion distance measurement value is determined in combination with the first spatial coordinate and the second spatial coordinate. When the initial hand detection confidence is insufficient, this process can make up for the deficiency of the initial detection by using the adjacent frame image information for secondary prediction and adjustment, and provide more reasonable hand position information for forward flexion distance measurement, thereby improving the accuracy and stability of the forward flexion distance measurement results to a certain extent.

[0018] Optionally, the first position prediction value is adjusted to a third position prediction value according to the third position, and the forward flexion ranging value of at least one second object is determined according to the third position prediction value, the first spatial coordinate and the second spatial coordinate, specifically including: determining a position adjustment parameter according to the spatial offset between the third position and the first position prediction value, wherein the position adjustment parameter includes a second horizontal offset and a vertical offset; adjusting the first position prediction value according to the position adjustment parameter to generate a third position prediction value; updating the second position of the hand of at least one second object to a fourth position according to the third position prediction value, and adding a third hand detection frame at the fourth position; determining a second vertical distance between the third hand detection frame and the second foot detection frame according to the third position prediction value, the first spatial coordinate and the second spatial coordinate; comparing the second vertical distance with a preset vertical threshold to obtain a distance comparison result; when it is determined according to the distance comparison result that the second vertical distance is less than the preset vertical threshold, adjusting the third position prediction value according to the historical change trend of the first position prediction value.

[0019] By adopting the above technical solution, the position adjustment parameter is determined according to the spatial offset between the third position and the first position prediction value, the first position prediction value is adjusted using the position adjustment parameter, the second position of the hand is updated to update the fourth position and add the third hand detection frame. The second vertical distance between the third hand detection frame and the second foot detection frame is calculated, and the second vertical distance is compared with the preset vertical threshold. If the second vertical distance does not meet the requirements, it is adjusted again according to the historical change trend of the first position prediction value. Through such multi-step adjustment and comparison, the hand position prediction can be continuously optimized, so that the hand position prediction is more consistent with the actual hand position in the forward bending posture, thereby improving the accuracy of the forward bending distance measurement value, and then ensuring that the distance measurement result can more truly reflect the actual situation of the students' forward bending.

[0020] In a second aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions. When the instructions are executed on an electronic device, the electronic device executes the method described in the first aspect and any possible implementation manner of the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The forward flexion ranging method for primary and secondary schools provided in the present application obtains first image data containing image data of multiple parts of multiple first objects in forward flexion postures, and uses this to train the initial detection model to obtain a part-by-part detection model, laying the foundation for posture detection. The initial hand recognition model is then trained in combination with the second image data to obtain a target hand recognition model. Later, when it is determined that multiple target objects need to perform forward flexion movements, the part-by-part detection model is used to perform posture detection on them. Such a process, from data collection, model training to actual detection, gradually builds up a complete forward flexion ranging system, providing a systematic solution for accurate detection of forward flexion posture and ranging, thereby improving the accuracy and reliability of forward flexion ranging.

[0024] 2. The forward flexion ranging method for primary and secondary schools provided by the present application, in the forward flexion posture detection link, first obtains the motion image sequence of the target object, and then preprocesses each frame of the image, which can remove interference information such as noise and improve image quality. The preprocessed image sequence is input into the part detection model, and the part detection model can detect the detection frame of each part. According to the first vertical distance between the first hand detection frame and the first foot detection frame and the first horizontal offset of the knee detection frame, it is judged whether it is the second forward flexion posture. Through the detection and judgment of these specific features, it is possible to accurately identify whether multiple target objects are in a forward flexion posture that meets the requirements, provide accurate posture judgment results for subsequent part detection and ranging, and ensure the accuracy of the starting link of the entire ranging process.

[0025] 3. The forward flexion ranging method for primary and secondary schools provided in the present application, when it is determined that at least one second object is in the second forward flexion posture, first establishes a target object list and records a unique identifier to facilitate tracking of specific objects. After determining the first position of the first current frame image of at least one second object, obtain an image sequence composed of N frames of images before and after the first current frame image, and input the image into the part detection model to perform a multi-scale feature extraction operation. Multi-scale feature extraction can extract object features from different resolutions, comprehensively and meticulously describe the object, and enable the part detection model to have a deeper understanding of at least one second object. Obtain the part detection results, which include detailed part feature information of the object, providing key data support for the subsequent accurate execution of the forward flexion ranging operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of a method for measuring distance by forward flexion of primary and secondary schools in an embodiment of the present application; Figure 2 It is a schematic diagram of a physical device structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0029] This application provides a method for measuring distance of forward flexion in primary and secondary schools, see Figure 1 , Figure 1 This is a flow chart of a method for measuring distance by forward flexion of primary and secondary schools in an embodiment of the present application, comprising the following steps: Step S101, when first image data is acquired, an initial detection model is trained according to the first image data to obtain a part-by-part detection model, wherein the first image data includes body position image data, hand position image data, knee position image data, and foot position image data of a plurality of first objects in a first forward flexion posture, and the first forward flexion posture is a forward flexion posture displayed by the plurality of first objects included in the first image data; Step S102, when the second image data is acquired, training the initial hand recognition model according to the first image data and the second image data to obtain a target hand recognition model, wherein the second image data includes hand image data of a plurality of first objects in a first forward bending posture; Step S103, when it is determined that the multiple target objects need to perform forward flexion exercise, using the part-by-part detection model to perform forward flexion posture detection on the multiple target objects to obtain forward flexion posture detection results, wherein the multiple first objects include the multiple target objects; Step S104, when it is determined according to the forward flexion posture detection result that at least one second object among the multiple target objects is in the second forward flexion posture, performing a part-by-part detection on the at least one second object using the part-by-part detection model to obtain a part-by-part detection result, wherein the multiple target objects include the at least one second object, and the second forward flexion posture is a forward flexion posture displayed by the at least one second object determined after the forward flexion posture detection is performed on the multiple target objects using the part-by-part detection model; Step S105: performing a forward flexion ranging operation on at least one second object according to the part-by-part detection result.

[0030] In the above embodiment, the first image data represents a set of image data including the human body position, hand position, knee position and foot position of multiple first objects when they are in a forward flexion posture, and these data are used to train the initial detection model. The initial detection model refers to a model that has not been fully trained and is used for subsequent training with the first image data to improve its detection ability. The part detection model is a model that is obtained after training with the first image data and can detect various parts of the human body. The second image data is the hand image data of multiple first objects when they are in a first forward flexion posture, and is used to train the initial hand recognition model together with the first image data. The initial hand recognition model is a model that is not fully trained and is intended to improve the hand recognition ability by combining training with the first image data, and the target hand recognition model is obtained after training. Multiple target objects refer to a set of objects that need to perform forward flexion movement and be detected, and the first object includes multiple target objects. The second forward flexion posture is a forward flexion posture displayed by at least one second object determined after the forward flexion posture detection of multiple target objects using the part detection model.

[0031] In the above embodiment, the timing for executing the above embodiment is when it is necessary to build a forward flexion ranging system for primary and secondary schools. Specifically, the first image data is obtained, and these data cover the image information of multiple parts of multiple objects in forward flexion postures, and the initial detection model is trained with them, so that the initial detection model learns the characteristics of each part in the forward flexion posture, thereby obtaining a part-by-part detection model. Then the second image data is obtained, and the initial hand recognition model is trained in combination with the first image data, so that the initial hand recognition model learns the hand characteristics in the forward flexion posture, and obtains the target hand recognition model. When it is determined that multiple target objects are to perform forward flexion movements, posture detection is performed on them using the part-by-part detection model to determine whether there are objects in the second forward flexion posture, in preparation for subsequent ranging. In some embodiments, the first image data can be obtained in a variety of ways: Optionally, a high-definition camera at a fixed position in the campus is used to organize students to perform forward bending posture demonstrations at a specific time, and the camera captures and records image data of the students as the first image data; Optionally, a portable image acquisition device is used, carried by a physical education teacher, to take images of students' forward bending postures one by one during physical education class, and the first image data is aggregated. It is understandable that other methods can also be used to obtain the first image data, for example, by allowing students to take photos of forward bending postures that meet the requirements and upload them through an online teaching platform, which is not limited here.

[0032] In the above embodiment, the training process of the part detection model is as follows: a large number of representative samples are selected from the first image data. These samples cover the human body position image data, hand position image data, knee position image data and foot position image data of students of different body shapes, heights and genders in the first forward bending posture. To ensure the diversity of data, samples should be taken from different shooting scenes, such as indoor gymnasiums, outdoor playgrounds, etc., and include various situations such as normal postures, postures close to the edge of qualified and unqualified. The selected image data are annotated, and the positions of the human body, hands, knees and feet in each image are accurately marked to generate corresponding annotation files. The annotated content includes information such as the bounding box coordinates and key point coordinates of each part, providing accurate supervision signals for subsequent model training. A basic neural network architecture suitable for the detection task is selected, such as Faster R-CNN (Faster Region-based Convolutional Neural Networks). The selected model is initialized, and the weight parameters of the model are randomly initialized. The purpose of initialization is to provide a starting state for the model during the training process, so that the model can adjust the weights by continuously learning the features in the data and gradually improve the detection performance. The labeled image data and the corresponding annotation files are divided into training set, validation set and test set.

[0033] In the above embodiment, the training set is used to update the parameters of the model, the validation set is used to monitor the performance of the model during the training process to prevent overfitting, and the test set is used to evaluate the final performance of the model after the training is completed. The training set is input into the initialized model. The model extracts features from the image through operations such as convolutional layers and pooling layers, and gradually learns the characteristic patterns of different parts in the forward bending posture. In this process, the model calculates the loss function between the predicted result and the true annotation based on the annotation information. Common loss functions include cross entropy loss, mean square error loss, etc. Use optimization algorithms, such as stochastic gradient descent, Adagrad (Adaptive Gradient Algorithm), Adadelta (Adadelta algorithm), etc., to update the weight parameters of the model according to the gradient of the loss function. Through continuous iterative training, the prediction results of the model gradually approach the true annotation, and the loss function value continues to decrease. During the training process, the performance indicators of the model are regularly evaluated on the validation set, such as accuracy, recall, etc. When the performance indicators on the validation set no longer improve or show signs of overfitting, the training parameters, such as learning rate, regularization coefficient, etc., are adjusted, or some methods to prevent overfitting are adopted. After the training is completed, the model is comprehensively evaluated using the test set. The various performance indicators of the model on the test set are calculated, such as mean average precision (mAP), accuracy, recall, etc., to accurately measure the accuracy and reliability of the model for different parts detection. According to the evaluation results, the types of errors that occur in the model during the detection process are analyzed, such as missed detection, false detection, etc. For these problems, the model is further optimized. You can try to adjust the architecture of the model, increase or decrease the number of network layers, adjust the size of the convolution kernel, etc. You can also readjust the training parameters, or collect more data for retraining to improve the performance and generalization ability of the model, and finally obtain a good performance of the part detection model.

[0034] In the above embodiment, the training process of the target hand recognition model is as follows: extracting hand image data of a plurality of first objects in a first forward bending posture from the first image data and the second image data. These data should include hand images under different lighting conditions and angles to enhance the robustness of the model. The extracted hand image data are sorted and annotated. The annotation content includes the category label of the hand (for example, a normally extended hand, a clenched hand, etc., if there are multiple hand states that need to be recognized), and the position information of the hand in the image (for example, bounding box coordinates, etc.). For some special hand postures or gestures, if the model needs to recognize them, they should also be accurately annotated. Select a neural network architecture suitable for image recognition tasks, such as LeNet model, AlexNet model, VGG model, etc. in convolutional neural networks, or a lightweight network designed for image classification, such as MobileNet (Mobile Convolutional Neural Networks), ShuffleNet, etc. Build a target hand recognition model according to the selected architecture. Determine the number of network layers, the type of each layer (for example, convolutional layer, pooling layer, fully connected layer, etc.), the number of neurons and other parameters. After the model is built, initialize the model and assign initial values ​​to the model's weight parameters. Random initialization is generally used. The sorted hand image data is divided into a training set, a validation set, and a test set. The training set is used to learn the model's parameters, the validation set is used to monitor the model's performance during the training process, and the test set is used to evaluate the performance of the final trained model. Input the training set into the built model.

[0035] In the above embodiment, the model extracts the features of the hand image through the convolution operation, the pooling operation reduces the dimension of the features, and the fully connected layer maps the extracted features to different categories. During the training process, the model calculates the loss function based on the difference between the predicted result and the true annotation. Common loss functions include cross entropy loss, etc. Use optimization algorithms, such as Adam (Adaptive Moment Estimation) optimizer, etc., to update the weight parameters of the model according to the gradient of the loss function. Through multiple iterative training, the weights are continuously adjusted so that the model can accurately classify the hand images in the training set. During the training process, the performance indicators such as the accuracy and recall rate of the model are regularly evaluated on the validation set. When the performance indicators on the validation set are no longer improved or overfitting occurs, take corresponding measures, such as. Adjust the learning rate, increase the regularization term, etc. to prevent overfitting and improve the generalization ability of the model. After the training is completed, the model is evaluated using the test set. Calculate the accuracy, recall rate and other indicators of the model on the test set to comprehensively evaluate the accuracy and reliability of the model for hand recognition. According to the evaluation results, analyze the error cases that occur in the recognition process of the model. If you find that the model does not work well for certain types of hand image recognition, for example, the recognition accuracy of hand images under low light conditions or certain special gestures is low, you can collect more relevant data and fine-tune the model. When fine-tuning, you can fix the weights of the first few layers of the model and only update the parameters of the later layers, or readjust the model architecture and training parameters and train again until the model performance meets the requirements and an accurate and reliable target hand recognition model is obtained.

[0036] Through the above steps, the first image data containing multiple image data of multiple parts of the first object in the forward bending posture is obtained, and the initial detection model is trained to obtain the part detection model, which lays the foundation for posture detection. The initial hand recognition model is then trained in combination with the second image data to obtain the target hand recognition model. Later, when it is determined that multiple target objects need to perform forward bending movements, the part detection model is used to perform posture detection on them. Such a process, from data collection, model training to actual detection, gradually builds a complete forward bending ranging system, providing a systematic solution for accurate detection of forward bending posture and ranging, thereby improving the accuracy and reliability of forward bending ranging. In addition, the technical problem of low accuracy of forward bending ranging in multi-person forward bending scenes in the related technology is solved, and the technical effect of improving the accuracy of forward bending ranging in multi-person forward bending scenes is achieved.

[0037] Among them, the executor of the above steps can be a control system with the ability to measure distance during forward flexion, or a control device with the ability to measure distance during forward flexion, or a controller or processor in a device or system, or a separate controller or processor, or other processing devices or processing units with similar processing functions, etc., but not limited to these.

[0038] In an optional embodiment, when it is determined that multiple target objects need to perform forward flexion movement, forward flexion posture detection is performed on the multiple target objects using a part detection model to obtain forward flexion posture detection results, specifically including: obtaining a motion image sequence of multiple target objects acquired in real time by a video acquisition device; preprocessing each frame of the motion image sequence to obtain a preprocessed image sequence; inputting the preprocessed image sequence into the part detection model, so that the part detection model detects the first human body detection frame, the first hand detection frame, the first knee detection frame and the first foot detection frame of each target object in the multiple target objects respectively; judging whether each target object is in a second forward flexion posture according to a first vertical distance between the first hand detection frame and the first foot detection frame; if the first vertical distance is continuously greater than a preset vertical threshold, and the first horizontal offset of the knee detection frame is less than the preset horizontal offset threshold, it is judged to be the second forward flexion posture; if at least one second object among the multiple target objects is judged to be in the second forward flexion posture, then the forward flexion posture detection result is output as valid, otherwise it is output as invalid.

[0039] In the above embodiment, the video acquisition device is used to acquire a motion image sequence of multiple target objects in real time, and the motion image sequence is a series of image sets that continuously reflect the motion process of the target object. Preprocessing is to perform operations such as denoising and contrast enhancement on the motion image sequence to improve the image quality, and the obtained preprocessed image sequence is more suitable for model processing. The first human body detection frame, the first hand detection frame, the first knee detection frame and the first foot detection frame are respectively used to represent the position range of the human body, hand, knee and foot of the target object detected by the model in the image. The first vertical distance is the distance between the first hand detection frame and the first foot detection frame in the vertical direction. The preset vertical threshold and the preset horizontal offset threshold are pre-set standard values ​​for judging whether the posture is qualified.

[0040] In the above embodiment, the timing of executing the above embodiment is after determining that multiple target objects are to perform forward flexion and a part detection model has been obtained. Specifically, first use a video acquisition device, for example, a surveillance camera installed in a gymnasium, to collect a sequence of images of multiple target objects performing forward flexion in real time. Then preprocess each frame of the sequence, for example, use Gaussian filtering to remove noise, histogram equalization to enhance contrast, etc. The preprocessed image sequence is input into the part detection model, and the model detects the part detection frames of each target object according to the learned features. By calculating the vertical distance between the hand and foot detection frames, and judging the horizontal offset of the knee detection frame, and comparing it with the preset threshold, it is determined whether the target object is in the second forward flexion posture, and finally the detection result is output. In some embodiments, the motion image sequence can be obtained in a variety of ways: Optionally, multiple cameras are installed at different angles in the test site to simultaneously photograph the target object, and the image sequences photographed by the multiple cameras are fused and processed later; Optionally, a portable camera with an anti-shake function is used to follow the movement of the target object to ensure that the complete forward flexion movement process is captured. It is understandable that other methods can also be used to obtain motion image sequences, for example, using a drone equipped with a high-definition camera to shoot from above, which is not limited here. Image preprocessing can be implemented using image processing libraries such as OpenCV (Open Source Computer Vision Library).

[0041] In an optional embodiment, when it is determined that at least one second object among multiple target objects is in a second forward flexion posture according to the forward flexion posture detection result, a part-by-part detection is performed on the at least one second object using a part-by-part detection model to obtain a part-by-part detection result, specifically including: establishing a target object list according to the at least one second object, and recording a unique identifier of the at least one second object; determining the first position of the first current frame image of the at least one second object in the motion image sequence; obtaining a first image sequence including the first current frame image and N first adjacent frame images before and after the first current frame image according to the first position, wherein N is a positive integer greater than or equal to 1; inputting the first image sequence and the target object list into the part-by-part detection model, so that the part-by-part detection model performs a multi-scale feature extraction operation on the first image sequence and the target object list; when it is determined that the part-by-part detection model has completed the multi-scale feature extraction operation, obtaining the part-by-part detection result output by the part-by-part detection model.

[0042] In the above embodiment, the target object list is established based on at least one second object in the second body forward bending posture, and is used to record the list of relevant information of these objects. The unique identifier is an identification code unique to each second object, which is convenient for distinguishing and tracking it. The first current frame image refers to a specific frame image of at least one second object in the motion image sequence. The first position represents the position number of the frame image in the motion image sequence. The first image sequence is an image set consisting of the first current frame image and the N adjacent frame images before and after it. The multi-scale feature extraction operation refers to feature extraction of images and target object lists at different resolutions to comprehensively obtain object features.

[0043] In the above embodiment, the timing of execution of the above embodiment is after the forward flexion posture detection result determines that at least one second object is in the second forward flexion posture. Specifically, a target object list is established based on at least one second object in the second forward flexion posture, and a unique identifier is recorded for each object. Determine the first position of the first current frame image of one of the second objects in the motion image sequence. According to the first position, a first image sequence is selected including the first current frame image and the N frames of images before and after. The first image sequence and the target object list are input into the part detection model, and the part detection model performs multi-scale feature extraction on them, mines the object features from different scales, and finally obtains the part detection results output by the model. In some embodiments, the target object list can be established in a variety of ways: Optionally, create an Excel table, and fill in the unique identifier of each second object, the class to which it belongs, and other information in sequence into the table to form a target object list; Optionally, a database is used to create a table, in which fields contain information such as a unique identifier, and relevant data is entered into the database table to form a target object list. It is understandable that other methods can also be used to create a target object list, for example, using a dictionary data structure in the Python programming language to store relevant information, which is not limited here.

[0044] In the above embodiment, the part detection model performs a multi-scale feature extraction operation on the first image sequence and the target object list, which is mainly achieved through the following process: the part detection model constructs an image pyramid for each image in the first image sequence. Taking a certain frame of image as an example, starting from the image of the original resolution, a series of images with gradually reduced resolutions are generated according to a certain downsampling ratio (for example, 2 times downsampling, etc.). The original image is the bottom layer of the pyramid, and the downsampled images are stacked up in sequence to form a pyramid structure. In this way, the target object can be observed at different resolutions, because the detailed features of the target object can be captured in the high-resolution image, while the low-resolution image can reflect the overall contour and large-scale features of the target object. For each layer of the image in the image pyramid, the model inputs it into its own feature extraction network. Assuming that the model is based on a convolutional neural network architecture, the convolution layer in the network performs convolution operations on the image through convolution kernels of different sizes (for example, 3×3, 5×5, etc.). For the bottom high-resolution image, a smaller convolution kernel can extract fine local features such as hand joints and knee contours; as the pyramid level increases, the image resolution decreases, and a larger convolution kernel is used to extract large-scale features such as the overall shape of the target object and human posture. In this way, images of different resolutions can obtain feature maps reflecting their corresponding scale features after passing through the feature extraction network. The part detection model encodes the information in the target object list. For example, the unique identification of each object in the list, the category it belongs to (here, the student object participating in the forward bending exercise), and other information are converted into vector form through the embedding layer (for example, the Embedding layer commonly used in deep learning), so that these non-image information can be fused with image features. These encoded vectors carry key information such as the identity and category of the target object, which helps the model to extract more targeted features in combination with the specific situation of the target object when extracting image features.

[0045] In the above embodiment, for the feature map obtained by the feature extraction network of each layer of the image pyramid, the model fuses it with the encoded target object information. Taking a certain layer of feature map as an example, the part detection model can splice the feature map with the target object encoding vector so that the feature map not only contains the visual features of the image itself at the scale, but also fuses the relevant information of the target object. In this way, in the subsequent processing, the model can better identify the features related to the specific target object. For example, in the forward bending posture detection, the forward bending posture features of different students can be more accurately distinguished without confusion caused by the interference of multiple objects in the image. After the above steps, the model obtains a feature map that fuses the target object information at each layer of the image pyramid. In order to obtain comprehensive and representative multi-scale features, the model will aggregate these feature maps of different scales. Common aggregation methods include maximum pooling or average pooling. Through maximum pooling, the model selects the maximum value of the corresponding position in the feature map of different scales as the feature value after aggregation, which can highlight the most significant features at different scales. Average pooling calculates the average value of the feature value of the corresponding position to integrate the feature information of different scales. Through this multi-scale feature aggregation, the part detection model finally obtains a multi-scale feature representation that integrates image features of different resolutions and target object information. These features will be used for subsequent part detection of at least one second object to accurately determine information such as the status and position of each part of the object.

[0046] In an optional embodiment, a forward flexion ranging operation is performed on at least one second object according to the part-by-part detection results and the target hand recognition model, specifically comprising: when it is determined according to the part-by-part detection results that the hand of at least one second object is not detected, determining a second human body detection frame and a first hand motion trajectory of at least one second object in a second current frame image from the part-by-part detection results, wherein the preprocessed image sequence includes the second current frame image, and the first human body detection frame includes a second human body detection frame; determining a first position prediction value of the hand of at least one second object according to the second human body detection frame and the first hand motion trajectory; determining a second position of the hand of at least one second object according to the first position prediction value, and adding a second hand detection frame at the second position; inputting the at least one second object to which the second hand detection frame has been added into the target hand recognition model; obtaining a hand detection confidence output by the target hand recognition model after performing hand recognition detection on the at least one second object to which the second hand detection frame has been added; and performing a forward flexion ranging operation on the at least one second object according to the hand detection confidence.

[0047] In the above embodiment, the second current frame image is a frame image in the preprocessed image sequence, which is used to determine the relevant information of at least one second object. The second human body detection frame is the human body position range of at least one second object detected by the part detection model, and belongs to the category of the first human body detection frame. The first hand motion trajectory refers to the moving path record of at least one second object's hand during the movement process. The first position prediction value is the hand position value estimated based on the second human body detection frame and the first hand motion trajectory. The second position is a more accurate hand position determined based on the first position prediction value, and a second hand detection frame is added at this position. The hand detection confidence is the reliability score of the hand detection result given by the target hand recognition model.

[0048] In the above embodiment, the timing for executing the above embodiment is after performing part detection on at least one second object to obtain the part detection result and when a forward flexion ranging operation is required. Specifically, when the part detection result shows that the hand of at least one second object is not detected, the second human body detection frame and the first hand motion trajectory of the object are determined from the second current frame image corresponding to the part detection result. Based on this information, the first position prediction value of the hand is predicted, and then a more accurate second position is determined and a detection frame is added. The object after adding the detection frame is input into the target hand recognition model, and the hand detection confidence output by the target hand recognition model is obtained to provide a basis for subsequent ranging operations. In some embodiments, the first position prediction value can be determined in a variety of ways: Optionally, a linear regression algorithm is used to predict the hand position based on the historical data of the hand motion trajectory and the current position of the human body detection frame; Optionally, based on the speed and direction of the hand movement, combined with the range of the human body detection frame, the first position prediction value is calculated through a physical model. It is understandable that other methods can also be used to determine the first position prediction value, for example, using a recurrent neural network in deep learning to learn and predict the hand movement trajectory, which is not limited here.

[0049] In the above embodiment, the target hand recognition model performs hand recognition detection on at least one second object to which a second hand detection frame has been added and outputs the hand detection confidence, which mainly includes the following process: the target hand recognition model receives the image data of at least one second object to which a second hand detection frame has been added. After receiving the image, a series of preprocessing operations are performed on the image, such as normalization, which uniformly scales the pixel values ​​of the image to a specific range, such as [0, 1] or [-1, 1], etc., which helps to speed up the training and convergence speed of the model while improving the stability and accuracy of the model. It is also possible to standardize the image, that is, subtract the mean of the image and divide it by the standard deviation to eliminate the differences in brightness and contrast between different images. The image data enters the convolution layer of the target hand recognition model. The convolution layer consists of multiple convolution kernels, which slide on the image and extract local features in the image through convolution operations. Different convolution kernels can capture different types of features, such as edges, textures, etc. As the convolutional layers are continuously stacked, the model can gradually extract more complex and abstract high-level features from simple low-level features. For example, the shallow convolution kernel may detect the contour edge of the hand, while the deep convolution kernel can recognize the shape of the hand, joint position and other features. The feature map extracted by the convolution layer enters the pooling layer. The main function of the pooling layer is to downsample the feature map, reduce the dimension of the feature map, reduce the amount of calculation, and prevent overfitting to a certain extent. Common pooling methods include maximum pooling and average pooling. Taking maximum pooling as an example, the maximum value is selected as the output within a fixed-size pooling window, which can retain the most significant features in the image.

[0050] In the above embodiment, the features processed by the pooling layer will be expanded into a one-dimensional vector and input to the fully connected layer. Each neuron in the fully connected layer is connected to all neurons in the previous layer, and can comprehensively process the extracted features and map the features to the final classification space. In the fully connected layer, the features are linearly transformed through a series of weight matrix multiplications and bias addition operations, and then pass through an activation function, such as the ReLU (Rectified Linear Unit, linear rectifier function) function, to introduce nonlinear factors so that the target hand recognition model can learn more complex patterns. Finally, the output of the fully connected layer will pass through a Softmax layer. The Softmax layer converts the output of the fully connected layer into a probability distribution, with each category corresponding to a probability value, and the sum of these probability values ​​is 1. The hand detection confidence is the probability value corresponding to the hand category. For example, if the model determines that the probability of a hand in an image is 0.8, then the hand detection confidence is 0.8, which means that the model is 80% sure that the object in the image is a hand. In this way, the target hand recognition model completes the hand recognition detection of at least one second object to which the second hand detection frame has been added, and outputs the hand detection confidence.

[0051] In an optional embodiment, a forward flexion ranging operation is performed on at least one second object based on the hand detection confidence, specifically including: performing a confidence comparison on the hand detection confidence and a preset confidence threshold to obtain a confidence comparison result; when it is determined according to the confidence comparison result that the hand detection confidence is greater than or equal to the preset confidence threshold, marking the first hand detection frame as a valid hand detection frame; determining the first spatial coordinates of a second knee detection frame of at least one second object, and determining the second spatial coordinates of a second foot detection frame of at least one second object, wherein the first knee detection frame includes the second knee detection frame, and the first foot detection frame includes the second foot detection frame; determining the forward flexion ranging value of at least one second object based on the first position prediction value, the first spatial coordinates, and the second spatial coordinates.

[0052] In the above embodiment, the preset confidence threshold is a standard value preset for judging whether the hand detection confidence is reliable. The confidence comparison result is the judgment result obtained by comparing the hand detection confidence with the preset confidence threshold. A valid hand detection frame refers to a hand detection frame whose hand detection confidence reaches or exceeds the preset confidence threshold. The second knee detection frame is the knee position range corresponding to at least one second object in the first knee detection frame, and the same is true for the second foot detection frame. The first spatial coordinate and the second spatial coordinate respectively represent the position coordinates of the second knee detection frame and the second foot detection frame in space. The body flexion distance measurement value is the distance value of the body flexion of at least one second object calculated based on the relevant position information.

[0053] In the above embodiment, the timing for executing the above embodiment is after obtaining the hand detection confidence output by the target hand recognition model and when the forward flexion ranging value is to be calculated. Specifically, the hand detection confidence is first compared with the preset confidence threshold to obtain a confidence comparison result. If the confidence is greater than or equal to the threshold, the corresponding first hand detection frame is marked as valid. Then, the spatial coordinates of the second knee detection frame and the second foot detection frame of at least one second object are determined, and the forward flexion ranging value is determined using the spatial coordinate calculation method in combination with the first position prediction value obtained previously. In some embodiments, the spatial coordinates can be determined in a variety of ways: Optionally, the spatial coordinates are calculated by triangulation using the intrinsic and extrinsic parameters of the camera and the pixel positions of the detection box in the image; Optionally, multiple reference points are arranged in the test site, and the spatial coordinates are determined based on the relative position relationship between the detection frame and the reference points in combination with a geometric algorithm. It is understandable that other methods can also be used to determine the spatial coordinates, for example, using laser radar and other equipment to assist in determining the spatial coordinates, which is not limited here.

[0054] In the above embodiment, to determine the forward flexion ranging value of at least one second object according to the first position prediction value, the first spatial coordinate (the first spatial coordinate of the second knee detection frame) and the second spatial coordinate (the second spatial coordinate of the second foot detection frame), the following calculation process can be used: Assume that the hand position coordinate corresponding to the first position prediction value is , the first spatial coordinate (i.e. the spatial coordinate of the second knee detection box) is , the second spatial coordinate (i.e. the spatial coordinate of the second foot detection frame) is Generally speaking, the distance measurement of forward flexion mainly focuses on the distance between the hand and the foot in the vertical direction (usually set to the y-axis direction, which can be adjusted according to the actual coordinate system). At the same time, it may be combined with information such as the knee position for comprehensive judgment (for example, judging whether the posture is standard, etc., where the distance measurement calculation is mainly considered). Calculate the vertical distance between the hand position and the foot position , consider the effect of knee position on distance measurement (in some more accurate distance measurement methods, corrections may be made based on knee position, assuming there is a correction factor , the correction factor can be determined according to actual conditions and experience, for example, by analyzing a large amount of standard posture data) , determine the final forward flexion distance measurement value For example, if the hand position coordinates are (10, 80, 20), the knee position coordinates are (12, 50, 22), and the foot position coordinates are (10, 30, 20), assuming the correction coefficient is .calculate ,calculate , then the final forward flexion distance measurement value (The units of the coordinate values ​​and calculation results here are determined according to the actual situation, for example, millimeters, centimeters, etc.) It should also be noted that the above calculation process is a possible implementation method. In actual applications, the calculation process and parameters can be adjusted according to the specific coordinate system settings, measurement requirements and accuracy requirements.

[0055] In an optional embodiment, after performing a confidence comparison between the hand detection confidence and a preset confidence threshold to obtain a confidence comparison result, the method further includes: when it is determined according to the confidence comparison result that the hand detection confidence is less than the preset confidence threshold, determining a third human body detection frame and a second hand motion trajectory of at least one second object in a second adjacent frame image from the part detection result, wherein the second adjacent frame image is an image included in the preprocessed image sequence and adjacent to the second current frame image, and the first human body detection frame includes a third human body detection frame; determining a second position prediction value of a hand of at least one second object according to the third human body detection frame and the second hand motion trajectory; determining a third position of a hand of at least one second object according to the second position prediction value; adjusting the first position prediction value to a third position prediction value according to the third position, and determining a forward flexion ranging value of at least one second object according to the third position prediction value, the first spatial coordinate, and the second spatial coordinate.

[0056] In the above embodiment, the second adjacent frame image is an image adjacent to the second current frame image in the preprocessed image sequence, and is used to obtain more information to adjust the hand position prediction value. The third human body detection frame is the human body position range corresponding to at least one second object in the second adjacent frame image in the part detection result, and belongs to the category of the first human body detection frame. The second hand motion trajectory is the hand motion path of at least one second object in the second adjacent frame image. The second position prediction value is the hand position value re-estimated based on the third human body detection frame and the second hand motion trajectory. The third position is a new hand position determined based on the second position prediction value. The third position prediction value is a new prediction value obtained by adjusting the first position prediction value according to the third position.

[0057] In the above embodiment, the execution time of the above embodiment is when the hand detection confidence is compared with the preset confidence threshold and the confidence is less than the threshold and the forward flexion ranging value needs to be recalculated. Specifically, when the hand detection confidence is insufficient, the second adjacent frame image is found from the part detection result to determine the third human body detection frame and the second hand motion trajectory of at least one second object. Based on this information, the second position prediction value of the hand is re-predicted, and then the third position is determined. Then, according to the third position, the first position prediction value is adjusted to the third position prediction value, and finally, the forward flexion ranging value is calculated in combination with the previously determined knee and foot detection frame spatial coordinates. In some embodiments, the second position prediction value can be determined in a variety of ways: Optionally, in the second adjacent frame image, the hand position is predicted using a Kalman filter algorithm according to a change trend of the hand motion trajectory and a moving direction of the human body detection frame; Optionally, by comparing the relative position relationship between the human body and the hand in the previous and next frame images, combined with the acceleration information of the hand movement, the second position prediction value is calculated through the dynamic model. It is understandable that other methods can also be used to determine the second position prediction value, for example, using a support vector machine in machine learning to learn and predict the relationship between the hand movement trajectory and the human body position, which is not limited here.

[0058] In an optional embodiment, the first position prediction value is adjusted to the third position prediction value according to the third position, and the forward flexion ranging value of at least one second object is determined according to the third position prediction value, the first spatial coordinate and the second spatial coordinate, specifically including: determining a position adjustment parameter according to the spatial offset between the third position and the first position prediction value, wherein the position adjustment parameter includes a second horizontal offset and a vertical offset; adjusting the first position prediction value according to the position adjustment parameter to generate a third position prediction value; updating the second position of the hand of at least one second object to a fourth position according to the third position prediction value, and adding a third hand detection frame at the fourth position; determining a second vertical distance between the third hand detection frame and the second foot detection frame according to the third position prediction value, the first spatial coordinate and the second spatial coordinate; comparing the second vertical distance with a preset vertical threshold to obtain a distance comparison result; when it is determined according to the distance comparison result that the second vertical distance is less than the preset vertical threshold, adjusting the third position prediction value according to the historical change trend of the first position prediction value.

[0059] In the above embodiment, the spatial offset is the difference in space between the third position and the first position prediction value, thereby determining the position adjustment parameters, including the second horizontal offset and the vertical offset. The position adjustment parameters are used to adjust the first position prediction value to generate a third position prediction value. The fourth position is the position of at least one second object hand after updating according to the third position prediction value, and a third hand detection frame is added at this position. The second vertical distance is the distance between the third hand detection frame and the second foot detection frame in the vertical direction. The distance comparison result is the judgment result obtained by comparing the second vertical distance with the preset vertical threshold. The historical change trend of the first position prediction value refers to the change trend of the first position prediction value predicted multiple times before, which is used to adjust the third position prediction value when the distance does not meet the requirements.

[0060] In the above embodiment, the timing for executing the above embodiment is when the first position prediction value has been adjusted to the third position prediction value according to the third position and the forward flexion distance measurement value needs to be further refined. Specifically, the spatial offset between the third position and the first position prediction value is first calculated to determine the position adjustment parameter. The first position prediction value is adjusted according to the parameter to generate a third position prediction value, and the hand position is updated to the fourth position and a detection frame is added accordingly. The second vertical distance between the third hand detection frame and the second foot detection frame is calculated and compared with a preset vertical threshold. If the distance is less than the threshold, the third position prediction value is further adjusted according to the historical change trend of the first position prediction value, for example, an upward or downward trend, so as to more accurately calculate the forward flexion distance measurement value.

[0061] In some embodiments, the spatial offset may be determined in a number of ways: Optionally, a three-dimensional coordinate calculation method is used to calculate the coordinate difference between the third position and the predicted value of the first position in the three-dimensional space to obtain a spatial offset; Optionally, the spatial offset is calculated by analyzing the number of pixels of the hand position change in the image and the scale of the image. It is understandable that other methods can also be used to determine the spatial offset, such as using a depth camera to obtain depth information to calculate the spatial offset, which is not limited here.

[0062] In the above embodiment, when it is determined that the second vertical distance is less than the preset vertical threshold, the third position prediction value is adjusted according to the historical change trend of the first position prediction value, which can be achieved by the following process: collect and organize a series of first position prediction values ​​calculated previously, and these historical position prediction value data can reflect the change of the hand position prediction value in the time dimension. For example, in the process of multiple forward bending movements, the first position prediction values ​​corresponding to multiple time points have been obtained. Analyze the change trend of these historical position prediction value data. Time series analysis methods can be used, such as moving average method. Calculate the moving average of the first position prediction value within a certain period of time, and judge the overall trend by observing the trend of the moving average. Assume that the first position prediction value of the last 5 times is a calculation window, and calculate the moving average of each time. If the moving average shows a trend of gradually increasing, it means that the hand position is moving away from the foot as a whole in the previous prediction. On the contrary, if the moving average gradually decreases, it means that the hand position has a trend of approaching the foot. Adjust the third position prediction value according to the trend obtained by analysis. If the historical change trend shows that the hand position has been moving away from the feet, and the current second vertical distance is less than the preset threshold, it means that the currently predicted hand position may be too close to the feet. In this case, the third position prediction value needs to be adjusted in the direction away from the feet to increase the distance value in the vertical direction. Conversely, if the historical trend is that the hand position is approaching the feet, then the vertical distance between the third position prediction value and the feet needs to be appropriately reduced to make the adjusted position more in line with the historical change trend and the reasonable range of the hand position in the forward flexion posture, thereby optimizing the calculation basis of the forward flexion distance measurement value.

[0063] Through the embodiment of the present application, first image data containing image data of multiple parts of multiple first objects in forward flexion postures is obtained, and the initial detection model is trained to obtain a part-by-part detection model, laying the foundation for posture detection. The initial hand recognition model is then trained in combination with the second image data to obtain a target hand recognition model. Later, when it is determined that multiple target objects need to perform forward flexion movements, the part-by-part detection model is used to perform posture detection on them. Such a process, from data collection, model training to actual detection, gradually builds up a complete forward flexion ranging system, providing a systematic solution for accurate detection of forward flexion posture and ranging, thereby improving the accuracy and reliability of forward flexion ranging.

[0064] The electronic device in the embodiment of the present invention is described below from the perspective of hardware processing. Figure 2 , Figure 2 It is a schematic diagram of a physical device structure of an electronic device in an embodiment of the present application.

[0065] It should be noted that Figure 2The structure of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0066] like Figure 2 As shown, the electronic device includes a central processing unit (CPU) 201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 202 or the program loaded from the storage part 208 to the random access memory (RAM) 203, such as executing the method described in the above embodiment. In the RAM 203, There are various programs and data required for system operation. The CPU 201 , the ROM 202 , and the RAM 203 are connected to each other via a bus 204 . An input / output (I / O) interface 205 is also connected to the bus 204 .

[0067] The following components are connected to the I / O interface 205: an input section 206 including an audio input device, a button switch, etc.; an output section 207 including a liquid crystal display (LCD) and an audio output device, an indicator light, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 209 performs communication processing via a network such as the Internet. A drive 210 is also connected to the I / O interface 205 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as needed so that a computer program read therefrom is installed into the storage section 208 as needed.

[0068] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 209, and / or installed from the removable medium 211. When the computer program is executed by the central processing unit (CPU) 201, various functions defined in the present invention are performed.

[0069] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0070] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings.

[0071] Specifically, the electronic device of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, the method for measuring distance by forward flexion of primary and secondary school students provided in the above embodiment is implemented.

[0072] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of the electronic device, the electronic device implements the method for measuring distance of forward flexion in primary and secondary schools provided in the above embodiment.

[0073] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0074] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A method for measuring distance by forward bending in primary and secondary schools, characterized in that: include: In the case of acquiring the first image data, training the initial detection model according to the first image data to obtain the part-by-part detection model, wherein the first image data includes body position image data, hand position image data, knee position image data, and foot position image data of a plurality of first objects in a first forward flexion posture, and the first forward flexion posture is a forward flexion posture displayed by the plurality of first objects included in the first image data; When the second image data is acquired, the initial hand recognition model is trained according to the first image data and the second image data to obtain a target hand recognition model, wherein the second image data includes hand image data of the plurality of first objects when they are in the first forward bending posture; In the case where it is determined that a plurality of target objects need to perform forward flexion movement, using the part-by-part detection model to perform forward flexion posture detection on the plurality of target objects to obtain forward flexion posture detection results, wherein the plurality of first objects include the plurality of target objects; In a case where it is determined according to the forward flexion posture detection result that at least one second object among the multiple target objects is in the second forward flexion posture, performing a part-by-part detection on the at least one second object using the part-by-part detection model to obtain a part-by-part detection result, wherein the multiple target objects include the at least one second object, and the second forward flexion posture is a forward flexion posture displayed by the at least one second object determined after performing the forward flexion posture detection on the multiple target objects using the part-by-part detection model; A forward flexion ranging operation is performed on the at least one second object according to the part-by-part detection result and the target hand recognition model.

2. The method according to claim 1, characterized in that In the case where it is determined that a plurality of target objects need to perform forward bending movements, using the part-by-part detection model to perform forward bending posture detection on the plurality of target objects to obtain forward bending posture detection results specifically includes: Acquire a motion image sequence of the plurality of target objects captured in real time by a video capture device; Preprocessing each frame of the motion image sequence to obtain a preprocessed image sequence; Inputting the preprocessed image sequence into the part detection model, so that the part detection model detects a first human body detection frame, a first hand detection frame, a first knee detection frame and a first foot detection frame of each target object in the multiple target objects respectively; determining whether each target object is in the second forward bending posture according to a first vertical distance between the first hand detection frame and the first foot detection frame; If the first vertical distance is continuously greater than a preset vertical threshold, and the first horizontal offset of the knee detection frame is less than a preset horizontal offset threshold, it is determined to be the second body forward bending posture; If the at least one second object among the multiple target objects is determined to be in the second forward bending posture, the forward bending posture detection result is output as valid, otherwise it is output as invalid.

3. The method according to claim 2, characterized in that In the case where it is determined according to the forward flexion posture detection result that at least one second object among the multiple target objects is in the second forward flexion posture, using the part-by-part detection model to perform part-by-part detection on the at least one second object to obtain the part-by-part detection result, specifically includes: Establishing a target object list according to the at least one second object, and recording a unique identifier of the at least one second object; Determine a first current frame image of the at least one second object at a first position of the motion image sequence; Acquire, according to the first position, a first image sequence including the first current frame image and N first adjacent frame images before and after the first current frame image, wherein N is a positive integer greater than or equal to 1; Inputting the first image sequence and the target object list into the part detection model so that the part detection model performs a multi-scale feature extraction operation on the first image sequence and the target object list; When it is determined that the sub-part detection model has completed the multi-scale feature extraction operation, the sub-part detection result output by the sub-part detection model is obtained.

4. The method according to claim 2, characterized in that: The performing the forward bending ranging operation on the at least one second object according to the part detection result and the target hand recognition model specifically includes: In a case where it is determined according to the part-by-part detection result that the hand of the at least one second object is not detected, determining a second human body detection frame and a first hand motion trajectory of the at least one second object in a second current frame image from the part-by-part detection result, wherein the preprocessed image sequence includes the second current frame image, and the first human body detection frame includes the second human body detection frame; Determine a first position prediction value of a hand of the at least one second object according to the second human body detection frame and the first hand motion trajectory; Determine a second position of a hand of the at least one second object according to the first position prediction value, and add a second hand detection frame at the second position; inputting the at least one second object to which the second hand detection frame has been added into the target hand recognition model; Obtaining a hand detection confidence level output by the target hand recognition model after performing hand recognition detection on the at least one second object to which the second hand detection frame has been added; The forward bending ranging operation is performed on the at least one second object according to the hand detection confidence.

5. The method according to claim 4, characterized in that The performing the forward bending ranging operation on the at least one second object according to the hand detection confidence level specifically includes: Performing a confidence comparison between the hand detection confidence and a preset confidence threshold to obtain a confidence comparison result; In a case where it is determined according to the confidence comparison result that the hand detection confidence is greater than or equal to a preset confidence threshold, marking the first hand detection frame as a valid hand detection frame; Determine first spatial coordinates of a second knee detection frame of the at least one second object, and determine second spatial coordinates of a second foot detection frame of the at least one second object, wherein the first knee detection frame includes the second knee detection frame, and the first foot detection frame includes the second foot detection frame; A forward flexion ranging value of the at least one second object is determined according to the first position prediction value, the first spatial coordinate, and the second spatial coordinate.

6. The method according to claim 5, characterized in that After performing confidence comparison between the hand detection confidence and a preset confidence threshold to obtain a confidence comparison result, the method further includes: In the case where it is determined according to the confidence comparison result that the hand detection confidence is less than a preset confidence threshold, a third human body detection frame and a second hand motion trajectory of the at least one second object in a second adjacent frame image are determined from the part detection result, wherein the second adjacent frame image is an image included in the pre-processed image sequence and adjacent to the second current frame image, and the first human body detection frame includes the third human body detection frame; Determine a second position prediction value of a hand of the at least one second object according to the third human body detection frame and the second hand motion trajectory; determining a third position of a hand of the at least one second object according to the second position prediction value; The first position prediction value is adjusted to a third position prediction value according to the third position, and a forward flexion ranging value of the at least one second object is determined according to the third position prediction value, the first spatial coordinate, and the second spatial coordinate.

7. The method according to any one of claims 1 to 6, characterized in that: The method of adjusting the first position prediction value to a third position prediction value according to the third position, and determining the forward flexion ranging value of the at least one second object according to the third position prediction value, the first spatial coordinate and the second spatial coordinate specifically includes: Determining a position adjustment parameter according to a spatial offset between the third position and the first position prediction value, wherein the position adjustment parameter includes a second horizontal offset and a vertical offset; adjusting the first position prediction value according to the position adjustment parameter to generate a third position prediction value; updating the second position of the hand of at least one second object to a fourth position according to the third position prediction value, and adding a third hand detection frame at the fourth position; Determine a second vertical distance between the third hand detection frame and the second foot detection frame according to the third position prediction value, the first spatial coordinates, and the second spatial coordinates; Comparing the second vertical distance with a preset vertical threshold to obtain a distance comparison result; When it is determined according to the distance comparison result that the second vertical distance is less than the preset vertical threshold, the third position prediction value is adjusted according to the historical change trend of the first position prediction value.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions so that the electronic device executes the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 7.

Images