Intelligent rope skipping counting system and method based on attitude estimation
By obtaining rope skipping videos and using deep learning technology to extract keyframes and human key points characteristics, the problem of low accuracy of traditional rope skipping counting is solved, automatic counting and action recognition is realized, counting efficiency and accuracy is improved, and monitoring support is provided for fitness. .
Patent Information
- Application Number
- CN202411099294.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional rope skipping counting relies on manual methods, which leads to visual and mental fatigue of counting people when skipping rope for a long time, and is prone to missing numbers or majority, affecting the accuracy of counting.
By obtaining the rope skipping video of human objects, extracting the rope skipping keyframes and using deep learning-based keyframe image analysis and processing technology, the characteristics and timing analysis of the whole body keypoints of the human body are extracted, and the difference between the whole body keypoints of each two adjacent time points is calculated. Intelligently calculate the number of rope skipping times.
It realizes automatic calculation of the number of rope skipping times, reduces the need for manual counting, improves counting efficiency, and accurately identify rope skipping actions by analyzing keyframes and human body key points during rope skipping, reduces counting errors, and provides monitoring for users' health and fitness.
Smart Images

Figure CN118942013A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent counting, and more specifically, to a rope skipping intelligent counting system and method based on posture estimation. Background Art
[0002] Rope skipping is a popular aerobic exercise that involves using a long rope with handles at both ends, which the user holds to swing the rope and jumps over it as it passes around the body. Rope skipping can effectively improve cardiorespiratory endurance, enhance the health of the cardiovascular system, and as a high-intensity exercise, it can help burn a large number of calories in a shorter period of time, which helps to lose weight and control weight. Rope skipping counting allows rope jumpers to track their exercise progress and understand whether their performance has improved, which helps to improve the exercise experience and effect.
[0003] However, since traditional rope skipping counting usually relies on manual methods, manual counting may cause visual and mental fatigue of the counter during long rope skipping, resulting in momentary distraction or obstruction of vision, resulting in missed or excessive counts, thus affecting the accuracy of the count.
[0004] Therefore, an optimized intelligent counting system for skipping rope is expected. Summary of the invention
[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a skipping rope intelligent counting system and method based on posture estimation, which obtains a skipping rope video of a human object, extracts multiple skipping rope key frames from it to obtain a time series of skipping rope key frames, and uses key frame image analysis and processing technology based on deep learning to perform human body key point feature extraction and timing analysis on the time series of the skipping rope key frames, so as to intelligently obtain the number of skipping ropes according to the difference between the timing features of the whole body key points at every two adjacent time points. In this way, the system can automatically calculate the number of skipping ropes, reduce the need for manual counting, and improve the efficiency of counting. At the same time, by analyzing the key frames and human key points during the skipping rope process, the system can more accurately identify the skipping rope action, reduce counting errors, and provide monitoring for the health and fitness of users.
[0006] According to one aspect of the present application, there is provided a rope skipping intelligent counting system based on posture estimation, which comprises: A human subject rope skipping video acquisition module, used to acquire a rope skipping video of a human subject; A rope skipping process key frame extraction module, used for extracting key frames from the rope skipping video of the human object to obtain a time sequence of rope skipping key frames; A human body posture key point coordinate extraction module, used for inputting the time series of the rope skipping key frames into a human body posture key point feature extraction module to obtain a time series of multiple human body key point coordinates; A human body key point coordinate arrangement timing capture module is used to arrange the time series of the multiple human body key point coordinates into a time series of human body key point coordinates and then input the time series capture model of human body key point coordinates to obtain a time series of human body key point coordinate timing feature vectors; A human body key point coordinate difference calculation module is used to calculate the position difference between two adjacent human body key point coordinate time series feature vectors in the time series of the human body key point coordinate time series feature vectors to obtain a time series of human body key point coordinate time series change feature vectors; A key point coordinate time series fusion module is used to fuse the time series of the time series change feature vectors of the key point coordinates of the whole body of the human body to obtain the full time series change feature vectors of the key point coordinates of the whole body of the human body; The rope skipping times generating module is used to obtain the rope skipping times based on the full time series change feature vector of the coordinates of the key points of the human body.
[0007] According to another aspect of the present application, a method for intelligently counting rope skipping based on posture estimation is provided, which comprises: Get a video of a human subject skipping rope; Extract key frames from the rope skipping video of the human subject to obtain a time sequence of rope skipping key frames; Inputting the time sequence of the rope skipping key frames into a human body posture key point feature extraction module to obtain a time sequence of multiple human body key point coordinates; Arranging the time series of the plurality of human body key point coordinates into a time series of human body key point coordinates and then inputting the time series capture model of human body key point coordinates to obtain a time series of human body key point coordinate time series feature vectors; Calculating the position difference between two adjacent time series feature vectors of the human body key point coordinates in the time series of the human body key point coordinates to obtain a time series of time series change feature vectors of the human body key point coordinates; The time series of the time series change feature vectors of the key points coordinates of the whole body of the human body are integrated to obtain the full time series change feature vectors of the key points coordinates of the whole body of the human body; The number of rope skipping is obtained based on the full time-series change feature vector of the coordinates of the key points of the human body.
[0008] Compared with the prior art, the present application provides a rope skipping intelligent counting system and method based on posture estimation, which obtains a rope skipping video of a human object, extracts multiple rope skipping key frames from it to obtain a time series of rope skipping key frames, and uses key frame image analysis and processing technology based on deep learning to perform human body key point feature extraction and timing analysis on the time series of the rope skipping key frames, so as to intelligently obtain the number of rope skipping according to the difference between the timing features of the whole body key points at every two adjacent time points. In this way, the system can automatically calculate the number of rope skipping, reduce the need for manual counting, and improve the efficiency of counting. At the same time, by analyzing the key frames and human key points during rope skipping, the system can more accurately identify rope skipping actions, reduce counting errors, and provide monitoring for the health and fitness of users. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0010] Figure 1 It is a block diagram of an intelligent rope skipping counting system based on posture estimation according to an embodiment of the present application.
[0011] Figure 2 Schematic diagram of the architecture of an intelligent rope skipping counting system based on posture estimation according to an embodiment of the present application.
[0012] Figure 3 It is a block diagram of a human body key point coordinate arrangement timing capture module in a rope skipping intelligent counting system based on posture estimation according to an embodiment of the present application.
[0013] Figure 4 The present invention is a flowchart of a method for intelligently counting rope skipping based on posture estimation according to an embodiment of the present application. DETAILED DESCRIPTION
[0014] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, but rather these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0015] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0016] It should be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that the specific order or sequence of "first\second\third" can be interchanged where permitted. It should be understood that the objects distinguished by "first\second\third" can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.
[0017] As shown in this disclosure, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.
[0018] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.
[0019] Rope skipping is a popular aerobic exercise that involves using a long rope with handles at both ends, which the user holds to swing the rope and jumps over it as it passes around the body. Rope skipping can effectively improve cardiorespiratory endurance, enhance the health of the cardiovascular system, and as a high-intensity exercise, it can help burn a large number of calories in a shorter period of time, which helps to lose weight and control weight. Rope skipping counting allows rope jumpers to track their exercise progress and understand whether their performance has improved, which helps to improve the exercise experience and effect.
[0020] However, since traditional rope skipping counting usually relies on manual methods, manual counting may cause visual and mental fatigue of the counter during long rope skipping, resulting in momentary distraction or obstruction of vision, resulting in missed or excessive counts, thus affecting the accuracy of the count.
[0021] Therefore, a skipping rope intelligent counting system based on posture estimation is desired, which obtains a skipping rope video of a human object, extracts multiple skipping rope key frames from it to obtain a time series of skipping rope key frames, and uses key frame image analysis and processing technology based on deep learning to perform human body key point feature extraction and timing analysis on the time series of the skipping rope key frames, so as to intelligently obtain the number of skipping ropes according to the difference between the timing features of the whole body key points at every two adjacent time points. In this way, the system can automatically calculate the number of skipping ropes, reduce the need for manual counting, and improve the efficiency of counting. At the same time, by analyzing the key frames and human key points during the skipping rope process, the system can more accurately identify the skipping rope action, reduce counting errors, and provide monitoring for the user's health and fitness.
[0022] Figure 1 It is a block diagram of an intelligent rope skipping counting system based on posture estimation according to an embodiment of the present application. Figure 2 FIG. 1 is a schematic diagram of the architecture of an intelligent rope skipping counting system based on posture estimation according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the rope skipping intelligent counting system 100 based on posture estimation includes: a human body object rope skipping video acquisition module 110, which is used to acquire the rope skipping video of the human body object; a rope skipping process key frame extraction module 120, which is used to extract key frames from the rope skipping video of the human body object to obtain a time series of rope skipping key frames; a human body posture key point coordinate extraction module 130, which is used to input the time series of the rope skipping key frames into a human body posture key point feature extraction module to obtain a time series of multiple human body key point coordinates; a human body key point coordinate arrangement timing capture module 140, which is used to arrange the time series of the multiple human body key point coordinates into a time series of human body whole body key point coordinates and then input the human body key point coordinates into a time series. A coordinate timing capture model is used to obtain a time series of human body key point coordinate timing feature vectors; a human body key point coordinate differential calculation module 150 is used to calculate the positional difference between two adjacent human body key point coordinate timing feature vectors in the time series of human body key point coordinate timing feature vectors to obtain a time series of human body key point coordinate timing change feature vectors; a key point coordinate timing fusion module 160 is used to fuse the time series of human body key point coordinate timing change feature vectors to obtain a human body key point coordinate full timing change feature vector; a skipping number generation module 170 is used to obtain the number of skipping times based on the human body key point coordinate full timing change feature vector.
[0023] In an embodiment of the present application, the human object rope skipping video acquisition module 110 is used to obtain a rope skipping video of a human object. It should be understood that the rope skipping video of a human object refers to a video image that records a person performing rope skipping. This video usually captures the movements of the rope jumper from the perspective of a bystander and can be used for a variety of purposes, including motion analysis, health monitoring, skill improvement, etc. Based on this, in the technical solution of the present application, in order to be able to count rope skipping more accurately, a rope skipping video of a human object is obtained, and key frame sampling and feature analysis are performed on it, which can be used to automatically count the number of rope skipping, reduce the need for manual counting, and thus improve the counting efficiency of rope skipping. In particular, in a specific embodiment of the present application, a device such as a smart phone, a digital camera, or a professional video camera can be used to directly record the rope skipping process.
[0024] In the embodiment of the present application, the rope skipping process key frame extraction module 120 is used to extract key frames from the rope skipping video of the human object to obtain the time sequence of the rope skipping key frames. Specifically, in the embodiment of the present application, the rope skipping process key frame extraction module is used to extract key frames from the rope skipping video of the human object at fixed time intervals to obtain the time sequence of the rope skipping key frames. Accordingly, considering that the rope skipping video of the human object usually contains a large amount of data information, which contains key action information image frames about the human object rope skipping. Based on this, in the technical solution of the present application, key frames are extracted from the rope skipping video of the human object at fixed time intervals to obtain the time sequence of the rope skipping key frames. That is to say, extracting key frames from the rope skipping video of the human object can reduce the amount of data processed and improve the analysis efficiency, and the key frames usually contain important information of the rope skipping action or event, which can identify and analyze the specific stage of the rope skipping action. The appropriate time interval can ensure that all stages of the rope skipping action are captured, making the action analysis and counting more systematic and efficient, which is crucial for the rope skipping intelligent counting system or other motion analysis applications.
[0025] In an embodiment of the present application, the human body posture key point coordinate extraction module 130 is used to input the time series of the rope skipping key frame into the human body posture key point feature extraction module to obtain a time series of multiple human body key point coordinates. Specifically, in an embodiment of the present application, the human body posture key point coordinate extraction module is used to: input the time series of the rope skipping key frame into the human body posture key point feature extraction module based on OpenPose to obtain the time series of the multiple human body key point coordinates. It should be understood that, considering that the time series of the rope skipping key frame contains a large number of changes in the key points of the human body when skipping, the time series changes of these key points are crucial for rope skipping counting. Therefore, in order to better understand the information of the key points of the human body during rope skipping, in the technical solution of the present application, the time series of the rope skipping key frame is input into the human body posture key point feature extraction module based on OpenPose to obtain the time series of the multiple human body key point coordinates. It is worth mentioning that the OpenPose is a popular human body posture estimation framework based on deep learning, which can detect and map various key points of the human body from images or video frames in real time. In detail, the OpenPose uses an advanced deep learning model and can detect various key points of the human body with high precision, including the head, wrists, shoulders, knees, etc. That is, by extracting the key points of the rope skipping key frames in the time series through the OpenPose-based human posture key point feature extraction module, the human posture can be segmented from the background, the key point information of the whole body can be detected, and the key point coordinates of the human body in the rope skipping process can be effectively extracted, which can then be used for further motion analysis, time series feature extraction and calculation of the number of rope skipping times.
[0026] In an embodiment of the present application, the human body key point coordinate arrangement timing capture module 140 is used to arrange the time series of the multiple human body key point coordinates into a time series of human body key point coordinates and then input the time series capture model of the human body key point coordinates to obtain a time series of human body key point coordinate timing feature vectors. Figure 3 : is a block diagram of a human body key point coordinate arrangement timing capture module in a rope skipping intelligent counting system based on posture estimation according to an embodiment of the present application. Specifically, in the embodiment of the present application, Figure 3 As shown, the human body key point coordinate arrangement timing capture module 140 includes: a key point sample arrangement unit 141, used to arrange the time series of the multiple human body key point coordinates into the time series of the human body key point coordinates according to the sample dimension; a key point timing feature capture unit 142, used to input the time series of the human body key point coordinates into the TCN-based human body key point coordinate timing capture model to obtain the time series of the human body key point coordinate timing feature vector.
[0027] Specifically, the key point sample arrangement unit 141 is used to arrange the time series of the multiple human key point coordinates into the time series of the human body key point coordinates according to the sample dimension. Accordingly, considering that the time series of the multiple human body key point coordinates are arranged into the time series of the human body key point coordinates according to the sample dimension, the dynamic changes of the whole body information of the human body at multiple time points in the process of rope skipping can be fully captured, so as to identify the periodicity and continuity of the action, and conveniently compare the changes of the whole body key points of the human body between different time points, analyze the change trend of the action, and thus achieve more accurate and efficient rope skipping action analysis and counting.
[0028] Specifically, the key point time series feature capture unit 142 is used to input the time series of the human body key point coordinates into the human body key point coordinate time series capture model based on TCN to obtain the time series of the human body key point coordinate time series feature vector. It should be understood that, considering that there is a time series mutual correlation relationship between the human body key point coordinates in the time series of the human body key point coordinates, and TCN is a deep learning model suitable for time series data, which uses the principle of convolutional neural network (CNN) to deal with time series prediction problems. Therefore, in the technical solution of the present application, the time series of the human body key point coordinates is input into the human body key point coordinate time series capture model based on TCN to obtain the time series of the human body key point coordinate time series feature vector. That is, the human body key point coordinate time series capture model based on TCN can capture the time series correlation and dynamic characteristics in the human body key point coordinates, and learn the short-term and long-term time dependence, which is very important for understanding the coherence of human body movements, and obtains the time series of the human body key point coordinate time series feature vector with more time series correlation.
[0029] In the embodiment of the present application, the human body key point coordinate difference calculation module 150 is used to calculate the position difference between two adjacent human body key point coordinate time series feature vectors in the time series of the human body key point coordinate time series feature vector to obtain the time series of the human body key point coordinate time series change feature vector. Accordingly, considering that in order to be able to more accurately compare and analyze the human body posture time series change between two adjacent human body key point coordinate time series feature vectors, in the technical solution of the present application, the position difference between two adjacent human body key point coordinate time series feature vectors in the time series of the human body key point coordinate time series feature vector is calculated to obtain the time series of the human body key point coordinate time series change feature vector. That is, by calculating the difference, the change of the human body key point posture between consecutive time points can be highlighted, which is helpful to identify the start and end of the rope skipping action. And the difference can emphasize the time series change of the human body key point position, so that the model pays more attention to the dynamic characteristics of the action. Secondly, the rope skipping action has periodicity, and this periodicity can be more easily detected by the difference, so as to count the number of rope skipping.
[0030] In the embodiment of the present application, the key point coordinate time series fusion module 160 is used to fuse the time series of the time series change feature vectors of the key point coordinates of the whole body of the human body to obtain the full time series change feature vectors of the key point coordinates of the whole body of the human body. It should be understood that in order to integrate the key information from different time points of the body parts, so as to obtain a comprehensive, multi-dimensional feature representation, in the technical solution of the present application, the time series of the time series change feature vectors of the key point coordinates of the whole body of the human body are fused to obtain the full time series change feature vectors of the key point coordinates of the whole body of the human body, so that the information of the key points of the whole body of the human body in the entire time domain can be integrated, and a comprehensive human motion feature representation is provided to obtain the full time series change feature vectors of the key point coordinates of the whole body of the human body, and provide an overall explanation of the rope skipping action, thereby improving the overall and comprehensive understanding of the action, and providing better data input for the subsequent rope skipping counting processing.
[0031] In the embodiment of the present application, the skipping times generation module 170 is used to obtain the skipping times based on the full temporal change feature vector of the coordinates of the key points of the human body. Specifically, in the embodiment of the present application, the judgment result generation module includes: the skipping times generation module includes: a coordinate full temporal change feature optimization unit, which is used to perform feature optimization on the full temporal change feature vector of the coordinates of the key points of the human body to obtain an optimized full temporal change feature vector of the coordinates of the key points of the human body; a skipping times decoding unit, which is used to input the optimized full temporal change feature vector of the coordinates of the key points of the human body into a skipping times counter based on a decoder to obtain a decoded value of the skipping times.
[0032] Specifically, the coordinate full time series change feature optimization unit is used to optimize the feature vector of the full time series change of the coordinates of the whole body key points to obtain the optimized feature vector of the full time series change of the coordinates of the whole body key points. Specifically, in the embodiment of the present application, the coordinate full time series change feature optimization unit is used to: perform a mapping consistency measurement based on the differential adjustment of the instance perturbation on the feature vector of the full time series change of the coordinates of the whole body key points to obtain the optimized feature vector of the full time series change of the coordinates of the whole body key points. In particular, in the technical solution of the present application, it is considered that the neural network model can automatically learn the complex patterns and relationships in the data when it performs feature mining on the input data based on the network parameter matrix. However, relying solely on automatically extracted features may not be enough to cope with all situations, especially when there are special instances or disturbances in the data. In this case, if the inherent implicit association between the network output features and the network parameter matrix can be further considered, the behavior and decision-making process of the model can be more deeply understood. By analyzing this inherent association, the prior information of the neural network model can be used to modulate the feature manifold and optimize the distribution of the feature vector in the feature space. This modulation helps to reduce local disturbances caused by special instances or outliers, improve the robustness of the model to abnormal data, and enhance the generalization ability of the model in unknown or complex situations. In addition, this modulation also helps to improve the interpretability of the model, allowing developers to better understand how the model generates rope skipping counts based on input features. In the application scenario of rope skipping counting, the prior information of the neural network model is used to modulate the feature manifold to ensure the accuracy and reliability of rope skipping counting, and to maintain stable performance even in the face of complex or disturbed video data. Based on this, in one embodiment of the present application, a mapping consistency measurement based on instance perturbation differential adjustment is performed on the full temporal change feature vector of the human body's key point coordinates to obtain the optimized human body's full temporal change feature vector of the key point coordinates.
[0033] More specifically, the coordinate full temporal change feature optimization unit is used to: calculate the mean and variance of the full temporal change feature vector of the coordinates of the key points of the human body; calculate the difference between the eigenvalue of the predetermined position of the full temporal change feature vector of the coordinates of the key points of the human body and the mean of the full temporal change feature vector of the coordinates of the key points of the human body to obtain a first intermediate calculation value; multiply the network parameter matrix by the full temporal change feature vector of the coordinates of the key points of the human body, and then subtract the network parameter matrix from the full temporal change feature vector of the coordinates of the key points of the human body by the position to obtain a first intermediate vector, and calculate the Frobenius norm of the first intermediate vector to obtain a first norm value; add the first norm value to the full temporal change feature vector of the coordinates of the key points of the human body The square root of the variance of the eigenvector is taken to obtain a second intermediate calculation value; the first intermediate calculation value is divided by the second intermediate calculation value to obtain a third intermediate calculation value; the squares of each position in the full temporal change eigenvector of the human body key point coordinates are calculated and then accumulated to obtain a first total value; the square root of the first total value is taken and then divided by the scale of the network parameter matrix to calculate the natural exponential function value to obtain a fourth intermediate calculation value, wherein the scale of the network parameter matrix is equal to the product of the width value of the network parameter matrix and the height value thereof; the fourth intermediate calculation value is multiplied by the weighted hyperparameter and then added to the third intermediate calculation value to obtain the eigenvalue of the predetermined position of the optimized full temporal change eigenvector of the human body key point coordinates.
[0034] In the above preferred embodiment, specifically, the coordinate full time series change feature optimization unit is used to: process the full time series change feature vector of the coordinates of the key points of the human body with the following consistency measurement formula to obtain the optimized full time series change feature vector of the coordinates of the key points of the human body; wherein the consistency measurement formula is: ;in, is the full time series change feature vector of the coordinates of the key points of the human body, is the network parameter matrix, represents the weighted hyperparameter, is the first in the full time series change feature vector of the key point coordinates of the human body The characteristic value of is the scale of the network parameter matrix, which is equal to the product of the width value and the height value of the network parameter matrix. is the length of the full time series change feature vector of the key point coordinates of the human body, The Frobenius norm of the feature vector representing the full temporal change of the coordinates of the key points of the human body, represents the mean value of the feature set composed of all eigenvalues of the full time series change feature vector of the coordinates of the key points of the human body, represents the variance of the feature set composed of all eigenvalues of the full time series change feature vector of the coordinates of the key points of the human body, is the first of the optimized human body key point coordinates full time series change feature vectors The characteristic value of .
[0035] That is, in the technical solution of the present application, considering that when the neural network model performs feature mining on the input data based on the network parameter matrix, if the inherent implicit association between the network output features and the network parameter matrix can be further considered, the prior information of the neural network model can be further used to modulate the characteristic manifold of the characteristic vector of the full temporal change of the coordinates of the key points of the human body to reduce the possibility of special instance disturbances of its eigenvalues. Based on this, in a preferred embodiment, a mapping consistency measurement based on differential adjustment of instance disturbance is performed on the characteristic vector of the full temporal change of the coordinates of the key points of the human body, which first maps the characteristic vector of the full temporal change of the coordinates of the key points of the human body to the high-dimensional feature space where the network parameter matrix is located, and uses the compatibility perception factor between the mapping result and the original input as the intrinsic implicit association descriptor between the network output features and the network parameter matrix. Then, the statistical characteristics of the feature group of the full temporal change feature vector of the human body key point coordinates are used as the inherent modal information of the full temporal change feature vector of the human body key point coordinates, and the intrinsic latent correlation descriptor is used as the instance perturbation differential adjustment variable to dynamically standardize the attribute values of each position of the full temporal change feature vector of the human body key point coordinates. At the same time, the class-standardized modulus length data of the full temporal change feature vector of the human body key point coordinates is used as a bias term to constrain the acceptable range of the group distribution of the full temporal change feature vector of the human body key point coordinates. In this way, the attribute influence of the interference distribution of abnormal samples is suppressed as much as possible, and the expression accuracy of the full temporal change feature vector of the human body key point coordinates is improved.
[0036] Specifically, the rope skipping number decoding unit is used to input the optimized human body key point coordinates full time series change feature vector into the rope skipping counter based on the decoder to obtain the decoded value of the rope skipping number. That is, the decoded value of the rope skipping number is obtained intelligently by decoding the optimized human body key point coordinates full time series change feature vector. In this way, the system can automatically calculate the number of rope skipping, reduce the need for manual counting, and improve the efficiency of counting. At the same time, by analyzing the key frames and key points of the human body during the rope skipping process, the system can more accurately identify the rope skipping action, reduce counting errors, and provide monitoring for the user's health and fitness.
[0037] In summary, the rope skipping intelligent counting system 100 based on posture estimation according to the embodiment of the present application is explained, which obtains the rope skipping video of the human body object, and extracts multiple rope skipping key frames from it to obtain the time series of the rope skipping key frames, and uses the key frame image analysis and processing technology based on deep learning to perform human body key point feature extraction and timing analysis on the time series of the rope skipping key frames, so as to intelligently obtain the number of rope skipping according to the difference between the timing features of the whole body key points of each two adjacent time points. In this way, the system can automatically calculate the number of rope skipping, reduce the need for manual counting, and improve the efficiency of counting. At the same time, by analyzing the key frames and human key points during rope skipping, the system can more accurately identify the rope skipping action, reduce counting errors, and provide monitoring for the health and fitness of users.
[0038] Figure 4 FIG. 1 is a flow chart of an intelligent rope skipping counting method based on posture estimation according to an embodiment of the present application. Figure 4 As shown, according to the embodiment of the present application, the intelligent counting method for skipping rope based on posture estimation includes: S110, obtaining a skipping rope video of a human body object; S120, extracting key frames from the skipping rope video of the human body object to obtain a time series of skipping rope key frames; S130, inputting the time series of the skipping rope key frames into a human body posture key point feature extraction module to obtain a time series of multiple human body key point coordinates; S140, arranging the time series of the multiple human body key point coordinates into a time series of human body key point coordinates and then inputting the human body key point coordinate timing capture model to obtain a time series of human body key point coordinate timing feature vectors; S150, calculating the position difference between two adjacent human body key point coordinate timing feature vectors in the time series of human body key point coordinate timing feature vectors to obtain a time series of human body key point coordinate timing change feature vectors; S160, fusing the time series of human body key point coordinate timing change feature vectors to obtain a human body key point coordinate full timing change feature vector; S170, obtaining the number of skipping ropes based on the human body key point coordinate full timing change feature vector.
[0039] Here, those skilled in the art can understand that the specific operations of each step in the above-mentioned rope skipping intelligent counting method based on posture estimation have been described in the above reference. Figures 1 to 3 The description of the rope skipping intelligent counting system based on posture estimation has been introduced in detail, and therefore, its repeated description will be omitted.
[0040] It can be understood by a person skilled in the art that all or part of the steps of the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a readable storage medium. When the program is executed, the steps of the above-mentioned method embodiment are executed; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disks or optical disks. Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present invention can be essentially or in other words, the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. And the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, disks or optical disks.
[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit of the technical solution of the present invention.
Claims
1. A rope skipping intelligent counting system based on posture estimation, characterized in that: include: A human subject rope skipping video acquisition module, used to acquire a rope skipping video of a human subject; A rope skipping process key frame extraction module, used for extracting key frames from the rope skipping video of the human object to obtain a time sequence of rope skipping key frames; A human body posture key point coordinate extraction module, used for inputting the time series of the rope skipping key frames into a human body posture key point feature extraction module to obtain a time series of multiple human body key point coordinates; A human body key point coordinate arrangement timing capture module is used to arrange the time series of the multiple human body key point coordinates into a time series of human body key point coordinates and then input the time series capture model of human body key point coordinates to obtain a time series of human body key point coordinate timing feature vectors; A human body key point coordinate difference calculation module is used to calculate the position difference between two adjacent human body key point coordinate time series feature vectors in the time series of the human body key point coordinate time series feature vectors to obtain a time series of human body key point coordinate time series change feature vectors; A key point coordinate time series fusion module is used to fuse the time series of the time series change feature vectors of the key point coordinates of the whole body of the human body to obtain the full time series change feature vectors of the key point coordinates of the whole body of the human body; The rope skipping times generating module is used to obtain the rope skipping times based on the full time series change feature vector of the coordinates of the key points of the human body.
2. The rope skipping intelligent counting system based on posture estimation according to claim 1 is characterized in that: The rope skipping process key frame extraction module is used to: Key frames are extracted from the rope skipping video of the human object at fixed time intervals to obtain a time sequence of the rope skipping key frames.
3. The rope skipping intelligent counting system based on posture estimation according to claim 2 is characterized in that: The human body posture key point coordinate extraction module is used for: The time series of the rope skipping key frames is input into a human posture key point feature extraction module based on OpenPose to obtain the time series of the coordinates of the multiple human key points.
4. The rope skipping intelligent counting system based on posture estimation according to claim 3 is characterized in that: The human body key point coordinate arrangement timing capture module includes: A key point sample arrangement unit, used for arranging the time series of the plurality of human body key point coordinates into the time series of the human body whole body key point coordinates according to the sample dimension; The key point time series feature capture unit is used to input the time series of the human body key point coordinates into the human body key point coordinate time series capture model based on TCN to obtain the time series of the human body key point coordinate time series feature vector.
5. The rope skipping intelligent counting system based on posture estimation according to claim 4 is characterized in that: The rope skipping times generating module comprises: A coordinate full time series change feature optimization unit, used for optimizing the feature vector of the full time series change of the coordinates of the key points of the human body to obtain an optimized feature vector of the full time series change of the coordinates of the key points of the human body; The rope skipping times decoding unit is used to input the optimized human body key point coordinates full time series change feature vector into a rope skipping counter based on a decoder to obtain a decoded value of the rope skipping times.
6. The rope skipping intelligent counting system based on posture estimation according to claim 5 is characterized in that: The coordinate full time series change characteristic optimization unit is used for: The mapping consistency measurement based on instance perturbation differential adjustment is performed on the full temporal change feature vector of the coordinates of the key points of the human body to obtain the optimized full temporal change feature vector of the coordinates of the key points of the human body.
7. The rope skipping intelligent counting system based on posture estimation according to claim 6 is characterized in that: The coordinate full time series change characteristic optimization unit is used for: Calculate the mean and variance of the full time series change feature vector of the key point coordinates of the human body; Calculate the difference between the characteristic value of the predetermined position of the full time series change characteristic vector of the human body key point coordinates and the mean value of the full time series change characteristic vector of the human body key point coordinates to obtain a first intermediate calculation value; Multiplying the network parameter matrix by the full temporal variation feature vector of the human body key point coordinates, and then subtracting the matrix from the full temporal variation feature vector of the human body key point coordinates by position to obtain a first intermediate vector, and calculating the Frobenius norm of the first intermediate vector to obtain a first norm value; Add the first norm value to the variance of the full time series change characteristic vector of the key point coordinates of the whole body of the human body and then take the square root to obtain a second intermediate calculation value; dividing the first intermediate calculated value by the second intermediate calculated value to obtain a third intermediate calculated value; Calculating the square of each position in the full time series change feature vector of the coordinates of the key points of the human body and then accumulating and summing them to obtain a first total value; Calculate a natural exponential function value after taking the square root of the first sum value and dividing it by the scale of the network parameter matrix to obtain a fourth intermediate calculation value, wherein the scale of the network parameter matrix is equal to the product of the width value of the network parameter matrix and the height value thereof; The fourth intermediate calculated value is multiplied by the weighted hyperparameter and then added to the third intermediate calculated value to obtain the characteristic value of the predetermined position of the optimized human body key point coordinate full time series change characteristic vector.
8. A method for intelligent rope skipping counting based on posture estimation, characterized in that: include: Get a video of a human subject skipping rope; Extract key frames from the rope skipping video of the human subject to obtain a time sequence of rope skipping key frames; Inputting the time sequence of the rope skipping key frames into a human body posture key point feature extraction module to obtain a time sequence of multiple human body key point coordinates; Arranging the time series of the plurality of human body key point coordinates into a time series of human body key point coordinates and then inputting the time series capture model of human body key point coordinates to obtain a time series of human body key point coordinate time series feature vectors; Calculating the position difference between two adjacent time series feature vectors of the human body key point coordinates in the time series of the human body key point coordinates to obtain a time series of time series change feature vectors of the human body key point coordinates; The time series of the time series change feature vectors of the key points coordinates of the whole body of the human body are integrated to obtain the full time series change feature vectors of the key points coordinates of the whole body of the human body; The number of rope skipping is obtained based on the full time-series change feature vector of the coordinates of the key points of the human body.
9. The method for intelligent rope skipping counting based on posture estimation according to claim 8, characterized in that: The time series of the rope skipping key frames is input into a human body posture key point feature extraction module to obtain a time series of multiple human body key point coordinates, including: The time series of the rope skipping key frames is input into a human posture key point feature extraction module based on OpenPose to obtain the time series of the coordinates of the multiple human key points.
10. The method for intelligent rope skipping counting based on posture estimation according to claim 9, characterized in that: Arranging the time series of the plurality of human body key point coordinates into a time series of human body key point coordinates and then inputting the human body key point coordinate time series capture model to obtain a time series of human body key point coordinate time series feature vectors, including: Arrange the time series of the multiple human body key point coordinates into the time series of the human body key point coordinates according to the sample dimension; The time series of the human body key point coordinates is input into the human body key point coordinate timing capture model based on TCN to obtain the time series of the human body key point coordinate timing feature vector.
Citation Information
Cited By
Rope skipping counting method and device based on action classification, medium and electronic equipment
CN119919744A
Skipping Rope Counting Method, Device, Medium and Electronic Device Based on Action Classification
CN119919744B
