Sit-up counting method and system based on motion posture estimation
Through the sit-up counting method based on motion posture estimation, the posture tracking algorithm and adaptive adjustment factor are used to solve the inaccuracy and environmental interference of the traditional counting method, and high-precision and real-time exercise data in modern fitness environments are achieved feedback.
Patent Information
- Application Number
- CN202411197476.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-11-26
AI Technical Summary
The traditional sit-up counting method relies on manual recording, is cumbersome and error-prone, and cannot provide real-time and accurate feedback on exercise information, and cannot meet the needs of modern fitness environments for intelligent sports support.
Using sit-up counting method based on motion posture estimation, the human body skeleton data was extracted from the video using the posture tracking algorithm Detection2, and posture correction was performed through adaptive adjustment factors, combining the angle of the legs and hips and the distance between the eyes and feet to determine the supine lying Standards of sitting up and counting.
It improves the accuracy of the judgment of action standards and the robustness of the action recognition algorithm, and provides stable and reliable counting and feedback in different environments.
Smart Images

Figure CN119028021A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of image data processing, and in particular to a sit-up counting method based on motion posture estimation and a system thereof. Background Art
[0002] Fitness has gradually evolved into a lifestyle in people's work and life, and sit-ups are one of them. In the traditional sit-up counting method, it usually relies on manual recording, which is not only cumbersome and error-prone, but also unable to provide real-time and accurate exercise information feedback. As fitness enthusiasts have higher requirements for exercise effects and data accuracy, the demand for intelligent exercise support is also increasing. As a result, users may not be able to obtain reliable exercise data when doing exercises such as sit-ups, which in turn affects their evaluation and adjustment of exercise effects.
[0003] With the advancement of technology, users' expectations for motion counting systems are also increasing. They hope to get accurate data feedback in real time during exercise so that they can better adjust the training intensity and posture to improve the exercise effect. In order to meet this demand, the traditional counting method is obviously unable to adapt to the challenges of the modern fitness environment. Therefore, the development of a smarter and more convenient counting system has become an urgent problem to be solved. Such a system not only needs to have high-precision motion detection capabilities, but also be able to effectively cope with the interference of various environmental factors, so as to provide users with stable and reliable counting and feedback. Summary of the invention
[0004] In order to overcome the above-mentioned technical defects, the embodiments of the present invention provide a sit-up counting method and system based on motion posture estimation, which can effectively improve the accuracy of motion standard judgment during the sit-up movement process, while enhancing the robustness of the motion recognition algorithm in different environments.
[0005] In a first aspect, an embodiment of the present invention provides a sit-up counting method based on motion posture estimation, comprising:
[0006] During the sit-up counting process, the posture tracking algorithm Detection2 is used to extract human skeleton data representing the position characteristics of the torso of the person who performs the action from each frame of the video output. The human skeleton data includes the position data of the eyes, legs, hips, feet and knees.
[0007] Using the adaptive adjustment factor calculated from the human skeleton data, the human skeleton data is subjected to posture correction, and the adaptive adjustment factor acts independently on each bone point in the human skeleton data;
[0008] The standardization of sit-ups is determined and counted based on the angle between the legs and hips and the distance between the eyes and feet of the person performing the sit-ups, which are calculated from the posture-corrected human skeleton data.
[0009] In a second aspect, another aspect of an embodiment of the present invention provides a sit-up counting system based on motion posture estimation, comprising:
[0010] Feature extraction module and sit-up counting module;
[0011] The feature extraction module is used to: during the sit-up counting process, use the posture tracking algorithm Detection2 to extract the human skeleton data representing the position characteristics of the torso of the person who performs the action from each frame of the video output, and the human skeleton data includes the position data of the eyes, legs, hips, feet and knees; use the adaptive adjustment factor calculated from the human skeleton data to perform posture correction on the human skeleton data, and the adaptive adjustment factor acts independently on each bone point in the human skeleton data;
[0012] The sit-up counting module is used to determine the standardization and count of sit-ups based on the angle between the legs and hips and the distance between the eyes and feet of the person who performs the action calculated from the human skeleton data after posture correction.
[0013] In a third aspect, an embodiment of the present invention provides a computer device, including:
[0014] at least one processor; and
[0015] a memory in communication with the at least one processor and an image acquisition device;
[0016] Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.
[0017] The embodiments of the present invention adopt the above technical solution to achieve the following technical effects:
[0018] In the sit-up counting method based on facial recognition and motion posture estimation of the present invention, on the one hand, the posture tracking algorithm Detection2 is used to perform human skeleton data such as position data of eyes, legs, hips, feet and knees. The algorithm not only has high precision in complex scenes, but also has the advantage of low latency, ensuring high efficiency and accuracy in real-time processing; on the other hand, the posture correction algorithm is used to correct the skeleton point data, and at the same time, the standardization of sit-ups is determined and counted based on the leg angle, hip angle and distance from eyes to feet of the action sender calculated from the corrected skeleton data. It can accurately and automatically determine whether the action is standard and count it. Combining the improvements in the above two aspects, the accuracy of the action standard judgment can be effectively improved during the sit-up movement, and the robustness of the action recognition algorithm in different environments can be enhanced. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.
[0020] Figure 1 A schematic flow chart of a sit-up counting method based on motion posture estimation in an embodiment of the present invention;
[0021] Figure 2 Schematic diagram of the position structure of human skeleton data in an embodiment of the present invention;
[0022] Figure 3 A schematic diagram of a network structure of a posture correction module in an embodiment of the present invention;
[0023] Figure 4 It is a comparison diagram of the experimental effects before and after correction in the embodiment of the present invention;
[0024] Figure 5 Schematic diagram of the effect of counting sit-ups in an embodiment of the present invention;
[0025] Figure 6 A schematic diagram of the system structure of a sit-up counting system based on motion posture estimation in an embodiment of the present invention;
[0026] Figure 7 The figure is a schematic diagram of the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION
[0027] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0028] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present invention are only for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.
[0029] In the description of the present invention, it should be understood that the numerical labels before the steps do not identify the order of executing the steps, but are only used to facilitate the description of the present invention and distinguish each step, and therefore cannot be understood as a limitation of the present invention.
[0030] In order to facilitate understanding of the sit-up counting method in the embodiment of the present invention, the technical solution in the embodiment of the present invention is described in detail below in conjunction with specific embodiments.
[0031] Figure 1 FIG. 2 is a flow chart of a method for counting sit-ups based on motion posture estimation in an embodiment of the present invention. Figure 1 As shown, the sit-up counting method includes:
[0032] Step S101: Use a lightweight face recognition network model to identify the user identity of the person who initiates the action.
[0033] The lightweight face recognition network model is optimized based on the network parameter optimization method of greedy first-order algorithm search, and the network parameter pruning algorithm is used to iteratively fine-tune and prune the optimized network model to obtain a lightweight network model; when the user identity changes, the sit-up count is cleared; when the user identity does not change, the sit-up count continues.
[0034] It should be noted that step S101 is an optional step, and in some embodiments, this step may not be performed, and only subsequent steps S102, S103 and S104 are performed.
[0035] Step S102: During the sit-up counting process, the posture tracking algorithm Detection2 is used to extract human skeleton data representing the position characteristics of the torso of the person who performs the action from each frame of the video output. The human skeleton data includes the position data of the eyes, legs, hips, feet and knees.
[0036] Specifically, Figure 2 As shown, the human skeleton data includes the position data of 17 key human bone points, and the key human bone points u include: mouth, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left hand, right hand, left hip, right hip, left knee, right knee, left foot and right foot.
[0037] Step S103: Use the adaptive adjustment factor calculated from the human skeleton data to perform posture correction on the human skeleton data, and the adaptive adjustment factor acts independently on each bone point in the human skeleton data.
[0038] Specifically, the above-mentioned use of the adaptive adjustment factor calculated from the human skeleton data to perform posture correction on the human skeleton data includes:
[0039] First, a two-layer CNN is used to learn the posture features of the skeleton graph corresponding to the human skeleton data;
[0040] Secondly, two fully connected layers are used to perform learning regression calculation on the upsampled posture features to obtain the adaptive adjustment factor;
[0041] Finally, the posture adjustment factor is used to perform matrix multiplication and inverse transformation on the human skeleton data to restore the posture-corrected human skeleton data.
[0042] It is easy to know that the above posture correction method can adaptively adjust the posture, effectively improving the model's ability to represent the posture.
[0043] Step S104: determine the standardization of sit-ups and count them based on the angle between the legs and hips and the distance between the eyes and feet of the person who performs the action calculated from the human skeleton data after posture correction.
[0044] Specifically, the angle between the legs and the buttocks includes a first angle and a second angle, the first angle is the angle between the left leg-left knee-left buttock, and the second angle is the angle between the right leg-right knee-right buttock; the distance between the eyes and the feet includes a first distance and a second distance, the first distance is the distance between the left eye-left foot, and the second distance is the distance between the right eye-right foot; determining the standardization of sit-ups and counting them, including:
[0045] If the first angle and the second angle are both smaller than the preset angle value, and the first distance and the second distance are both smaller than the preset distance value, it is determined that the sit-up action is standard and the count is increased by 1.
[0046] It is easy to know that during sit-ups, the distance from the head to the feet is smaller when the head is raised, and this distance becomes larger when the head is lying down. In the present invention, the distance from the left eye to the left foot and the distance from the right eye to the right foot are used to determine whether the action is up to standard. The angle of the legs and hips and the distance from the eyes to the feet can be accurately counted and the start and end of the action can be automatically determined.
[0047] In the sit-up counting method based on facial recognition and motion posture estimation of the present invention, on the one hand, the posture tracking algorithm Detection2 is used to perform human skeleton data such as position data of eyes, legs, hips, feet and knees. The algorithm not only has high precision in complex scenes, but also has the advantage of low latency, ensuring high efficiency and accuracy in real-time processing; on the other hand, the posture correction algorithm is used to correct the skeleton point data, and at the same time, the standardization of sit-ups is determined and counted based on the leg angle, hip angle and distance from eyes to feet of the action sender calculated from the corrected skeleton data. It can accurately and automatically determine whether the action is standard and count it. Combining the improvements in the above two aspects, the accuracy of the action standard judgment can be effectively improved during the sit-up movement, and the robustness of the action recognition algorithm in different environments can be enhanced.
[0048] Next, the specific operation steps of extracting features based on the posture tracking algorithm Detection2 and correcting the posture of the human skeleton data in the embodiment of the present invention are described as follows:
[0049] First, based on the posture tracking algorithm of Detection2, the camera is called through the OpenCV library or the video is input locally. Detection2 can output the information of the 17 skeleton points of the human body detected in each frame frame by frame according to the input video. The distribution of the 17 skeleton points in each frame is as follows: Figure 2 As shown. And it can track the changes of each skeleton point in real time. The data format of the output skeleton point is (X, Y, P), where X and Y represent the horizontal coordinate and the vertical coordinate with the upper left corner of the video frame as the origin, and P represents the confidence of the skeleton point, that is, the model calculates the probability that the position (X, Y) is a skeleton key point. Therefore, the present invention uses 17 skeleton points to represent the trunk position characteristics of the action sender in each frame, and the subsequent evaluation of the action standard can be realized through these 17 skeleton points.
[0050] Figure 2There are 17 key bone points in total, 0 represents the mouth, 2 and 1 represent the left eye and the right eye respectively; 4 and 3 represent the left ear and the right ear respectively; 6 and 5 represent the left shoulder and the right shoulder respectively; 8 and 7 represent the left elbow and the right elbow respectively; 10 and 9 represent the left hand and the right hand respectively; 12 and 11 represent the left hip and the right hip respectively; 14 and 13 represent the left knee and the right knee respectively; 16 and 15 represent the left foot and the right foot respectively.
[0051] Finally, in practical applications, the human body posture in the image is often affected by the environment and camera angle, resulting in tilt or deformation. In order to solve this problem, the present invention proposes a posture correction algorithm, which can correct the skeleton graph, thereby effectively enhancing the robustness of the action recognition algorithm in different environments. The posture correction module is inspired by the spatial transformer network and the rotation matrix. The spatial transformer is a method for RGB images. After the image is translated and rotated, the image pixels are filled according to the coordinate transformation, but it is not suitable for joint point data. The posture correction module in the present invention is an unsupervised learning module for human skeleton data. The principle is as follows Figure 3 As shown: The correction method of the posture correction module is different from the simple rotation of the entire skeleton by the rotation matrix. Instead, it calculates the adaptive adjustment factor to act relatively independently on each joint point of the skeleton graph. First, the skeleton data sent to the posture correction module needs to be adjusted as follows:
[0052] V N,D,J,T →V N×T,1,J,D
[0053] Among them: N represents the batch number of the model; D represents the number of joint point dimensions; J and T represent the total number of joint points and the total number of frames of the skeleton image. Then, two layers of CNN are used to learn the posture features of the skeleton image, and two layers of fully connected layers are used to learn the regression adaptive adjustment factor of the upsampled posture features. The adaptive posture adjustment factor of the 3D skeleton image calculated by the posture correction module is shown as follows:
[0054]
[0055] Among them: α, β, γ correspond to the adjustment factors of the X, Y, and Z axis coordinates of the 3D skeleton image. The corrected skeleton image V is obtained by matrix multiplication of the posture adjustment factor and the input skeleton data as shown in the following formula:
[0056] V' N×T,1,J,D =V N×T,1,J,D (α, β, γ)
[0057] Then do V again for V N,D,J,T →V N×T,1,J,D The inverse transformation can restore the skeleton data as shown below:
[0058] V' N×T,1,J,D →V' N,D,J,T
[0059] In order to analyze the effectiveness of the posture correction module, Figure 4 The comparison of experimental results before and after correction is shown. Figure 4 Part (a) represents the input skeleton graph. Figure 4 Part (b) represents the output of the posture correction module. Figure 4 It can be seen that the posture correction module can adaptively adjust the posture, effectively improving the model's ability to represent the posture.
[0060] Next, the specific operation steps of determining the standardization and counting of sit-ups based on the angle between the legs and the buttocks and the distance between the eyes and the feet in the embodiment of the present invention are described as follows:
[0061] After the video is input into Detection2, the skeleton key point information of each frame in the video will be output. Each skeleton key point contains the horizontal and vertical coordinates of the skeleton point in the video frame. The skeleton key points extracted by Detection2 are used to calculate whether the angles between the skeleton points meet the sit-up standard. If they meet the standard, the count is increased by 1, otherwise it is not counted.
[0062] First, the angle calculation based on the bone point:
[0063] In the present invention, the angle is calculated by using the cosine theorem using the coordinate values. Figure 2 As shown, for example, given a bone point To calculate the degree of bending of the left leg during a sit-up, we can calculate the degree of the angle formed by point 14. First, calculate the vector formed by points 12 and 14. Then calculate the vector formed by point 14 and point 16 Next we calculate the dot product of the vectors:
[0064]
[0065] The magnitude of a vector is:
[0066]
[0067] So the final angle is:
[0068]
[0069] The above formula gives an example of calculating the angle of the left leg during a sit-up. The same calculation is done for the right leg, left hip and right hip.
[0070] Secondly, the distance calculation based on the bone points:
[0071] During sit-ups, the distance from the head to the feet is smaller when the person is looking up, and becomes larger when the person is lying down. In the present invention, the distance from the left eye to the left foot and the distance from the right eye to the right foot are used to determine whether the action is up to standard. The angles of the legs and hips and the distance from the eyes to the feet can be used to accurately count and automatically determine the start and end of the action. Figure 2 As shown, the distances from point 4 to point 16 and from point 3 to point 15 are calculated by the following formulas:
[0072]
[0073] When lying down, the action is valid if dis(eye,foot) is greater than the threshold d2, otherwise it is invalid and will not be counted.
[0074] Finally, the number of sit-ups is counted in the background according to the identity ID:
[0075] The present invention can automatically record background data according to identity information. If the initiator of sit-ups is changed, the face recognition module will immediately capture the change of identity information. At this time, the count and identity ID of the previous moment will be automatically filled into the background database. After filling, the count will be cleared and counted again according to the angle and distance standards. The system repeats this process all the time, and the exercise data and the information of the initiator of the exercise can be recorded in pairs.
[0076] The final effect diagram of the present invention is as follows Figure 5 As shown in the figure: When a person is detected in the lens, Detection2 will automatically detect the various bone points of the human body. Since it uses bone point data, it can have good real-time performance. Considering that in actual applications, the human body posture in the image will be tilted or deformed due to the influence of the environment and camera angle, a posture correction module is added to correct the deformed bone data points, which can make the calculation of the torso angle more accurate. Figure 1 As shown in the left picture, after the character lies down, the angles of the knees and waist are calculated, and it is judged that the posture at this time is lying down, so the output of lying down is 1 (indicating lying down, 0 indicates not lying down), and the output of looking up is 0. The posture of the next stage is shown in the right picture. When the character looks up, the output is 1, and the output of lying down is 0. At this time, the transition from lying down to looking up is completed, so the number is increased by 1.
[0077] Next, the specific operation steps of using the lightweight face recognition network model to identify the user identity of the action sender in the embodiment of the present invention are described in detail as follows:
[0078] First, the network parameter optimization method based on greedy first-order algorithm search:
[0079] The method calculates the importance score of a single neuron by iteratively estimating its impact on the network loss. Then, the neuron with the lowest score is pruned to obtain a lightweight network. Given the parameters of the neural network W = {ω0,ω1,....,ω M}, (the neural network parameters include some convolution kernel parameters and some linear layer parameters, which are automatically updated by back propagation during network training) and the paired input (x i ) and output (y i ) composed of a data set D = {(x0,y0),(x1,y2),...,(x M ,y M )}, the training task is to minimize the error by the following formula:
[0080]
[0081] First start with a complete parameter set W, and then gradually reduce some parameters. Specifically, each convolution kernel is responsible for extracting the features of the image. If some convolution kernels contribute little to the final output, these convolution kernels can be removed. This not only reduces the number of parameters, but also reduces the amount of computational effort of the convolution operation. In this process, it is assumed that these parameters are independent of each other, so it is possible to decide which parameters to remove by evaluating the importance of each parameter. This simplified step-by-step search process is called a greedy first-order algorithm search. In the present invention, the importance of a parameter can be quantified by the error caused by removing a parameter. Under the assumption of independent and identical distribution, the error generated in this way can be determined by the presence or absence of a parameter (ω m ) is quantified by the squared difference of the prediction error when m is from 0 to M:
[0082] I m =(E(D,W)-E(D,W|ω m =0)) 2
[0083] The importance of all parameters is calculated by the above formula I m . M parameter importances can be obtained, so it needs to evaluate M versions of the network, each version corresponding to a removed parameter.
[0084] Secondly, the importance of I m Perform a first-order Taylor expansion to simplify the calculation of parameter importance:
[0085] I can be approximated by using its first-order Taylor expansion around W: m , thus avoiding the evaluation of M different networks. Expanded by the following formula:
[0086]
[0087] in is the element of the gradient g. The importance of the parameters can be easily calculated by the above formula, because the gradient g has been obtained through back propagation. In order to approximate a set of parameters W S The importance of a parameter (such as a convolution kernel) is approximated by summing the importance of each parameter in the present invention:
[0088]
[0089] Finally, use the network parameter pruning algorithm for fine-tuning and pruning:
[0090] The trained network is used as input and pruned through an iterative fine-tuning process with a small learning rate. The learning rate refers to the step size of the model's weight adjustment at each parameter update. A small learning rate means that the model's weight adjustment is small during the iteration process. During the pruning process, after some weights are pruned, the structure of the model changes and the performance may be reduced. In order to restore or maintain the performance of the model, the pruned model needs to be fine-tuned. Using a small learning rate can make the model's parameters adjust at a slower speed during the fine-tuning process, avoiding the drastic fluctuation of the model's performance due to large weight updates, thereby gradually restoring the model's performance. Repeat the following steps in each cycle. In the pruning algorithm, a cycle usually refers to a complete execution cycle of the two steps of pruning and fine-tuning. A cycle includes pruning the model and then fine-tuning it with a small learning rate. In each cycle, the model will undergo a pruning and fine-tuning. This process may require multiple iterations and repeated cycles to gradually reduce the number of parameters of the model. Each cycle will further prune and fine-tune until the expected parameter reduction or performance standard is achieved. For each input to the network, the parameter gradient is calculated and the network weights are updated by gradient descent. At the same time, the importance of each neuron is calculated by step 2. After that, the importance score of each neuron (convolution kernel) is averaged and the N neurons with the smallest importance score are removed. Fine-tuning and pruning continue until the target number of neurons is pruned or the maximum tolerable loss can no longer be achieved.
[0091] The present invention adopts the SqueezeNet implementation previously proposed for FR (face recognition) and makes it lightweight through a network pruning method, called LightSqueezeFaceNet. Specifically, the network uses an input size of 113×113 instead of the original 227×227 of SqueezeNet. This is achieved by changing the stride of the first convolutional layer from 2 to 1, while keeping the rest of the network unchanged, so that the parameters of ImageNet can be reused as the starting model. This transfer learning strategy from ImageNet has been shown to provide the same or better performance as initialization from scratch, while converging faster. In the present invention, batch normalization is also added between the convolutional layer and the ReLU layer. This is missing in the original SqueezeNet, but batch normalization is often used before nonlinear activation to help training and reasoning of deep networks.
[0092] The sit-up counting system based on motion posture estimation in an embodiment of the present invention is described in detail below. Figure 6 As shown, the sit-up counting system 60 includes: a feature extraction module 601 and a sit-up counting module 602;
[0093] The feature extraction module 601 is used to: during the sit-up counting process, extract human skeleton data representing the position characteristics of the torso of the person who performs the action from each frame of the video output using the posture tracking algorithm Detection2, the human skeleton data including the position data of the eyes, legs, hips, feet and knees; perform posture correction on the human skeleton data using the adaptive adjustment factor calculated from the human skeleton data, the adaptive adjustment factor acting independently on each bone point in the human skeleton data;
[0094] The sit-up counting module 602 is used to determine the standardization of sit-ups and count them according to the angle between the legs and hips and the distance between the eyes and feet of the person who performs the action calculated from the human skeleton data after posture correction.
[0095] Optional, such as Figure 6 As shown in the dotted line, in the embodiment of the present invention, the system 60 may also include: a lightweight face recognition module 603, which is used to use a lightweight face recognition network model to identify the user identity of the person who issues the action, wherein the lightweight face recognition network model is optimized based on a network parameter optimization method based on a greedy first-order algorithm search, and a network parameter pruning algorithm is used to iteratively fine-tune and prune the optimized network model to obtain a lightweight network model; when the user identity changes, the sit-up counting module clears the sit-up count; when the user identity does not change, the sit-up counting module continues to execute the sit-up count.
[0096] It should be noted that the feature extraction module 601, the sit-up counting module 602 and the lightweight face recognition module 603 can also perform the operations of the relevant parts in the above method embodiment, and will not be described in detail here.
[0097] Next, in the embodiment of the present invention, a computer device is also provided, such as Figure 7 As shown, the computer device 70 includes: a memory 701 and a processor 702, wherein the recognition system is deployed in the memory 701, and the processor 702 executes the method for recognizing the pull-up posture in the above embodiment by calling the recognition system deployed in the memory 701. The computer device 70 may be a smart camera with an AI chip, etc. Optionally, an image acquisition device may also be set in the computer device 70.
[0098] In the computer device 70, the memory 701 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 701 can be an internal storage module of the computer device 70, such as a hard disk or memory of the computer device 70. In other embodiments, the memory 701 can also be an external storage device of the computer device 70, such as a plug-in hard disk equipped on the computer device 70, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 701 can also include both the internal storage module of the computer device 70 and its external storage device. In this embodiment, the memory 701 is generally used to store the operating system and various application software installed on the computer device 70, such as the program code of the bullet screen display method, etc. In addition, the memory 701 can also be used to temporarily store various types of data that have been output or are to be output.
[0099] In some embodiments, the processor 702 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 702 is generally used to control the overall operation of the computer device 70, such as performing control and processing related to data interaction or communication with the computer device 70. In this embodiment, the processor 702 is used to run the program code stored in the memory 701 or process data.
[0100] In addition, in some special computer devices, a network interface is also provided on the computer device, and the network interface may include a wireless network interface or a wired network interface, and the network interface is generally used to establish a communication link between the computer device 70 and other computer devices. For example, the network interface is used to connect the computer device 70 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 70 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 7G network, 7G network, Bluetooth, Wi-Fi, etc.
[0101] Obviously, those skilled in the art should understand that each module or each step of the above-mentioned embodiment of the present application can be implemented with a general-purpose computer device, which can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices. Optionally, they can be implemented with a program code executable by a computer device, so that they can be stored in a storage device and executed by a computer device, and in some cases, the steps shown or described can be performed in an order different from that herein, or they can be made into each integrated circuit module respectively, or multiple modules or steps therein can be made into a single integrated circuit module to implement. In this way, the embodiment of the present application is not limited to any specific hardware and software combination.
[0102] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A sit-up counting method based on motion posture estimation, characterized in that: include: During the sit-up counting process, the posture tracking algorithm Detection2 is used to extract human skeleton data representing the position characteristics of the torso of the person who performs the action from each frame of the video output. The human skeleton data includes the position data of the eyes, legs, hips, feet and knees. Using the adaptive adjustment factor calculated from the human skeleton data, performing posture correction on the human skeleton data, wherein the adaptive adjustment factor acts independently on each bone point in the human skeleton data; The standardization of sit-ups is determined and counted based on the angle between the legs and hips and the distance between the eyes and feet of the person performing the sit-ups, which are calculated from the posture-corrected human skeleton data.
2. The method according to claim 1, characterized in that: The method of using the adaptive adjustment factor calculated from the human skeleton data to perform posture correction on the human skeleton data comprises: First, a two-layer CNN is used to learn the posture features of the skeleton graph corresponding to the human skeleton data; Secondly, two fully connected layers are used to perform learning regression calculation on the upsampled posture features to obtain the adaptive adjustment factor; Finally, the posture adjustment factor is used to perform matrix multiplication and inverse transformation on the human skeleton data to restore the posture-corrected human skeleton data.
3. The method according to claim 1 or 2, characterized in that The angle between the legs and the buttocks includes a first angle and a second angle, the first angle being the angle between the left leg-left knee-left buttock, and the second angle being the angle between the right leg-right knee-right buttock; The distance between the eyes and the feet includes a first distance and a second distance, the first distance is the distance between the left eye and the left foot, and the second distance is the distance between the right eye and the right foot; Determining the standardization and counting of sit-ups includes: If the first angle and the second angle are both smaller than the preset angle value, and the first distance and the second distance are both smaller than the preset distance value, it is determined that the sit-up action is standard and the count is increased by 1.
4. The method according to claim 1, characterized in that: The method further comprises: Use a lightweight face recognition network model to identify the user identity of the action sender; The lightweight face recognition network model is optimized by a network parameter optimization method based on a greedy first-order algorithm search, and a network parameter pruning algorithm is used to iteratively fine-tune and prune the optimized network model to obtain a lightweight network model; When the user identity changes, the sit-up count is reset to zero; When the user identity does not change, continue to perform the sit-up count.
5. The method according to claim 1, characterized in that The human skeleton data includes position data of 17 key human skeleton points, and the key human skeleton points include: mouth, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left hand, right hand, left hip, right hip, left knee, right knee, left foot and right foot.
6. A sit-up counting system based on motion posture estimation, characterized in that: include: Feature extraction module and sit-up counting module; The feature extraction module is used to: during the sit-up counting process, extract human skeleton data representing the position characteristics of the torso of the person who performs the action from each frame of the video output using the posture tracking algorithm Detection2, wherein the human skeleton data includes the position data of the eyes, legs, hips, feet and knees; perform posture correction on the human skeleton data using the adaptive adjustment factor calculated from the human skeleton data, wherein the adaptive adjustment factor acts independently on each bone point in the human skeleton data; The sit-up counting module is used to determine the standardization of sit-ups and count them according to the angle between the legs and hips and the distance between the eyes and feet of the person who performs the action calculated from the human skeleton data after posture correction.
7. The method according to claim 6, characterized in that The system further comprises: A lightweight face recognition module, used to identify the user identity of the action sender using a lightweight face recognition network model, wherein the lightweight face recognition network model is optimized based on a network parameter optimization method searched by a greedy first-order algorithm, and a lightweight network model is obtained by iteratively fine-tuning and pruning the optimized network model using a network parameter pruning algorithm; When the user identity changes, the sit-up counting module clears the sit-up count; When the user identity does not change, the sit-up counting module continues to perform sit-up counting.
8. A computer device, characterized in that: include: at least one processor; and a memory and an image acquisition device in communication with the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Method and device for determining target person in sit-up assessment, equipment and medium
CN115937969A
Sit-up counting and anti-cheating system based on human body detection and skeleton key point detection
CN116052277A
Sit-up evaluation system and method
CN117877105A
