Intelligent running posture correction method based on deep learning

Abstract

The invention relates to the technical field of artificial intelligence machine learning and sports crossing, in particular to an intelligent running posture correction method based on deep learning. Comprising the steps of data set construction; constructing a human body posture estimation model; constructing a running posture detection model, acquiring a real-time running video stream transmitted by a user terminal, and processing the video stream frame by frame; extracting a feature matrix of the current frame; obtaining an overturned feature matrix F0 '; the probability Possibility of a certain wrong running posture and the severity of a certain wrong running posture of the runner are calculated through an exponential weighted moving average method, and running posture evaluation and correction suggestions are provided; and updating the running posture evaluation and correction opinions until the video stream is finished. The method can be widely applied to the fields of exercise health management, running posture correction and the like, the exercise performance of runners can be effectively improved, exercise injuries are reduced, and the positive effect of improving scientificity and normalization of exercise training is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of the intersection of artificial intelligence machine learning and sports, and in particular to an intelligent running posture correction method based on deep learning. Background Art

[0002] With the growing awareness of fitness and the widespread adoption of healthy lifestyles, running, as a low-cost and accessible form of exercise, has gradually become an important daily exercise option for the general public. However, as the number of runners continues to grow, the impact of running injuries on people is also becoming increasingly significant. Research shows that the occurrence of running injuries is largely unrelated to external factors such as running duration, exercise frequency, and the venue of exercise. The main cause is incorrect running posture.

[0003] Incorrect running posture may lead to serious sports injuries, such as ankle injury, meniscus injury, and even irreversible injuries such as stress fractures. Especially when the body weight is carried in the wrong posture, the body fails to effectively use the weight-bearing parts, but instead puts excessive load on the joints, ligaments, muscles and other parts that should not bear too much pressure, thereby causing injury. Common incorrect postures include striding, hyperextension of the knees, knees buckling, head-up running and sitting running, etc. These incorrect postures are prone to cause sports injuries in different parts of the body. For example, hyperextension of the knees and knees buckling may cause excessive force on the knee joint, increasing the risk of meniscus injury; striding will cause excessive impact on the lower limbs, leading to iliotibial band friction syndrome; and head-up running and sitting running may put excessive pressure on the spine, affecting body stability.

[0004] Adopting the correct running form is crucial for reducing sports injuries. To help runners, especially beginners, reduce running injuries, it is particularly important to promptly correct common incorrect posture errors. Currently, human posture estimation technology, using computer vision and deep learning algorithms, can achieve real-time human posture detection, showing great potential for detecting and correcting incorrect running posture. Blazepose is an efficient, lightweight convolutional neural network architecture that accurately maps two-dimensional key points into three-dimensional space, suitable for real-time human posture estimation in mobile devices and embedded systems. Although some existing models have improved accuracy and efficiency by increasing computational efficiency, they still fail to function properly when target key points are occluded and can only recognize simple movements. Some running posture monitoring methods can only detect basic arm swing and are limited to specific treadmill scenarios where a person attaches positioning points, making them ineffective in real-life scenarios. Existing technologies generally lack effective correction capabilities for different running posture errors and are not widely adaptable to diverse application scenarios.

[0005] Therefore, in order to reduce sports injuries among runners, it has become an urgent technical need to study an effective method to detect and identify runners' incorrect postures, and provide correction suggestions based on the relationship between key points of the human body, so as to apply it to different scenarios and help runners correct their incorrect postures in different scenarios. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent running posture correction method based on deep learning, which solves the current problem of difficulty in accurately detecting and identifying runners' incorrect running postures, so as to achieve the purpose of providing more accurate correction suggestions based on real-time detection in different scenarios, thereby effectively reducing the risk of running injuries.

[0007] The present invention provides an intelligent running posture correction method based on deep learning, which is characterized by comprising the following steps:

[0008] Step 1: Dataset construction phase. Collect and manually annotate data. Augment existing data by geometric flipping and affine transformation to obtain human pose estimation dataset O * , analyze the existing running posture types, construct the dataset according to the rules of horizontal comparison dataset, and obtain the running posture detection dataset O;

[0009] Step 2: Human body posture estimation model training phase, using human body posture estimation dataset O * Train the human pose estimation neural network with Blazepose as the core to obtain the human pose estimation model;

[0010] Step 3: In the running posture detection model construction phase, the sample o in the running posture detection dataset O is l Input the human body posture estimation model one by one, l represents the sample number, and obtain the three-dimensional human body key point P of the sample l , build running posture detection model C, record c in running posture detection model C l Perform alignment and cleaning, detect and remove outliers;

[0011] Step 4: Enter the detection phase, obtain the real-time running video stream transmitted by the user terminal, and process the video stream frame by frame;

[0012] Step 5, feature extraction stage, input the current frame into the human posture estimation model, calculate the 3D human key points of the current frame, extract the feature matrix of the current frame, obtain the feature matrix F0 of the current frame, and extract the single record c in the running posture detection model C. l The characteristic matrix F l The characteristic matrix D that constitutes a certain running posture m , wherein the video to be detected is divided into several frames and then processed frame by frame, and the frame to be processed is the current frame;

[0013] Step 6, classification stage, flip the feature matrix of the current frame along the x-axis to obtain the flipped feature matrix F0′. First calculate the feature matrix F0 of the current frame and the sum of the feature matrix F0 and the single record c l The characteristic matrix F l The Minkowski distance between the two, and then calculate the flipped feature matrix F0′ and the single record c l The characteristic matrix F l The Minkowski distance between the two running positions is calculated, the double queue method is used to process the data, and the voting method is used to obtain the confidence value of a certain running posture.

[0014] Step 7: Evaluation. Using the confidence value of a particular running posture, the exponentially weighted moving average method is used to calculate the probability and severity of a runner's incorrect running posture. This provides running posture evaluation and correction advice.

[0015] Step 8: If the video stream has not ended, return to step 4 and update the running posture evaluation and correction suggestions until the video stream ends.

[0016] Furthermore, in step one, the method of geometrically flipping and affine transformation to augment existing data is to perform transformation operations on the original image data, including but not limited to geometric adjustment, elastic deformation, grid distortion, image filtering such as box filtering, Gaussian filtering, median filtering, bilateral filtering, and random brightness and contrast adjustment of color channels.

[0017] Furthermore, in step 1, the rule of the running posture detection dataset O is to divide the original data image into two running posture states: flying and landing, and construct a comparison dataset.

[0018] Furthermore, in step 2, the human body posture estimation neural network structure with Blazepose as the core is:

[0019] The backbone network uses a lightweight convolutional neural network to extract deep features of images;

[0020] Heat map sub-network, generating 3D human key points P l A heat map is used to represent the likelihood of various parts of the human body;

[0021] Regression subnetwork, predicting 3D human key points P l The precise coordinates of the location;

[0022] Partial affinity field, used to connect 3D human key points P l , and predict the positions of the occluded joints to form a complete skeleton of the human body.

[0023] Furthermore, in step 3, the calculation rule of the 3D human key points is: a single image and its annotation, each image has a width of W, a height of H, and a number of channels of G, represented by a vector V∈R W×H×G , V refers to the quantity, R refers to the matrix, the key point p l Normalized, the original coordinates of each key point are Normalization is performed as follows,

[0024]

[0025] in is the normalized key point The coordinate of the i-th key point on the x-axis, is the normalized key point The coordinate of the i-th key point on the y-axis, where is the normalized key point The coordinate of the i-th key point on the z-axis, i = 1, 2, 3, 4, ..., 33, all 33 three-dimensional human key points P l ,

[0026]

[0027] Furthermore, in step 3, the alignment and cleaning rules of the 3D human key points are as follows: the dataset O aligns each sample o l The corresponding record c in the running posture detection model C l , after alignment, delete all samples that have no corresponding o l or record c l .

[0028] Furthermore, in step 3, the rule for detecting and eliminating outliers is to detect a record c in the running posture detection model C. l With other records c other Perform a comparison and mark out records that do not meet expectations, including missing or abnormal key point data caused by severe occlusion, and delete all records marked as outliers.

[0029] Furthermore, in step five, the feature extraction process is to record the 23 relative positions between the 33 key points according to the connection relationship of the human joints corresponding to the key points as 23 feature vectors. Each feature vector has three dimensions, forming a feature matrix. The process of extracting the feature matrix from 33 three-dimensional human key points can be expressed as FeatExtract(P l )=R (3*33) →R (3*23) ,

[0030] The feature matrix of the current frame F0∈R 3×23 ,

[0031]

[0032] where x1~x 23 is the component of each feature on the x-axis in the feature matrix F0 of the current frame, y1~y 23 is the component of each feature on the y-axis in the feature matrix F0 of the current frame, z1~z 23 is the component of each feature on the z-axis in the feature matrix F0 of the current frame,

[0033] Extract the running posture detection model O into the c of a single record from the N records of a certain running posture l Feature matrix F l The characteristic matrix D that constitutes the mth running posture m , m refers to the mth wrong running posture, recorded as

[0034] D m =[F1,F2,F3,......,F N ]∈R 3×23×N

[0035]

[0036] Among them, X1~X 23 For a single record c l The characteristic matrix F l The component of each feature on the x-axis, Y1~Y 23 For a single record c l The characteristic matrix F l The component of each feature on the y-axis, Z1~Z 23 For a single record c l The characteristic matrix F l The component of each feature in on the z-axis.

[0037] Furthermore, in step 6, the flipped feature matrix F0′ can be expressed as,

[0038] Feature matrix of the current frame

[0039] Flipped feature matrix

[0040] Furthermore, in step 6, the feature matrix F0 of the current frame is compared with the single record c in the model. l The characteristic matrix F l The generation rule of the Minkowski distance between is:

[0041]

[0042] where weight x is the axis weight of the 23 eigenvectors on the x-axis, weight y is the axis weight of the 23 features on the y-axis, weight z is the axis weight of the 23 features on the z-axis, q is the norm of the Minkowski distance, j refers to the j-th eigenvector, j = 1, 2, 3, 4, ..., 23.

[0043] Furthermore, in step six, the dual-queue method implements different functions, wherein the first queue is used to implement the function of excluding abnormal data to avoid the data that has a significant impact on the classification results, and the second queue is used to implement the function of accurate classification after eliminating abnormal data, including:

[0044] The first queue calculates R in sequence 3×23 Under the matrix, the feature matrix F0 of the current frame and its flipped feature matrix F0′ are compared with the single record c in the running posture correction model. l The characteristic matrix F l The Minkowski distance of , and multiply it by the corresponding axis weight, take the maximum value of all recorded calculation results as the maximum distance, expressed as,

[0045]

[0046] Among them, max{} is the function for finding the maximum value, weight j (j=1,2,3,4,....,23) is the weight corresponding to each feature,

[0047] Take the feature matrix F0 of the current frame and its flipped feature matrix F0′ to get the smaller of the two maximum distances, denoted as distance max ,This smaller value indicates that the current frame and the running posture detection model are in the same direction, reflecting the distance between the current frame and a certain running posture in the running posture detection model, which is expressed as,

[0048] distance max =min{max{|F l -F0|},max{|F l -F0′|}}

[0049] Where min{} is the function to find the minimum value,

[0050] When an abnormality occurs at a test point, the maximum distance max It will amplify abnormal errors and make abnormal test points easier to eliminate;

[0051] The second queue calculates R in turn 3×23Under the matrix, the feature matrix F0 of the current frame and its flipped feature matrix F0′ are compared with the single record c in the running posture detection model. l The characteristic matrix F l Minkowski distance, multiplied by the corresponding axis weight, and the arithmetic mean of all recorded calculation results is taken as the average distance average ,

[0052] Take the smaller value of the two average distances obtained from the original feature matrix F0 and the flipped feature matrix F0′. The smaller value indicates that the current frame and the running posture detection model are in the same direction.

[0053]

[0054] Among them, average{} is the function for finding the average value.

[0055] Furthermore, in step 6, the voting rule is to calculate the feature matrix F0 of the current frame and the single record c in the N records of a certain running posture. l The characteristic matrix F l , the obtained N average distances are arranged from small to large, and the frequency of a certain running posture in the first k average distances is counted as the confidence value of the existence of a certain running posture in the current frame. By traversing each running posture, the confidence value of each running posture in the current frame can be obtained.

[0056] Furthermore, in step seven, the confidence level is smoothed using the exponentially weighted moving average method, which is specifically implemented as follows:

[0057]

[0058] Where μ is the smoothing coefficient, r is the current frame number, fps is the total frame number, value r is the confidence of the fps-r frame, value r ′ is the confidence after smoothing.

[0059] Furthermore, in step seven, the calculation formula for the probability of a runner having a certain incorrect running posture is:

[0060]

[0061] Among them, α represents the probability adjustment constant, which is used to give different influencing factors to various running postures, and λ represents the probability normalization factor, which is used to normalize the final output probability Possibility to 0%. 100%, n valid Indicates all current valid frames, n repeat It represents the number of times a certain running posture appears repeatedly in all valid frames. The calculation formula is:

[0062]

[0063] n repeat =∑repeat(value r ′),(r∈n valid )

[0064] Where repeat() is a function that calculates the membership of the fps-rth frame.

[0065] Furthermore, in step seven, the calculation formula for the severity of a certain incorrect running posture of the runner is:

[0066]

[0067] Where β represents the severity adjustment constant, which is used to give different influencing factors to various running postures, and η represents the severity normalization factor, which is used to normalize the severity of the final output to 0%. 100%, n value It represents the sum of the confidence of a certain running posture in all valid frames, and its calculation formula is:

[0068] n value =∑value r ,(r∈n valid ).

[0069] This invention provides an intelligent running posture correction method based on deep learning. It designs an effective dataset structure and improves dataset quality by classifying and comparing raw image data, addressing the diversity and rapid changes in running postures. By training a model based on Blazepose for human posture estimation, 33 three-dimensional key points are accurately extracted. Combining a feature matrix extraction method with the Minkowski distance metric, the method successfully identifies and classifies running postures. To further optimize the recognition effect, the invention employs a flip matrix and a double queue method to enhance the model's robustness to abnormal data and avoid errors caused by factors such as shooting angle. An exponentially weighted moving average method is employed to achieve responsiveness to new data. Finally, the user's running posture is evaluated based on the probability and severity of a particular incorrect running posture, significantly improving the real-time and accuracy of running posture correction suggestions. The invention can be widely used in fields such as sports health management and running posture correction, effectively improving runners' athletic performance and reducing sports injuries, thereby enhancing the scientific nature and standardization of sports training. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 Flowchart for the implementation of the present invention;

[0071] Figure 2 Schematic diagram of the positions of 33 key points of a three-dimensional human body according to the present invention;

[0072] Figure 3 Schematic diagram of the relationship between the 23 eigenvectors of the present invention;

[0073] Figure 4 The confidence value and smoothed confidence value of the present invention are r ′The curve about the frame number;

[0074] Figure 5 This is a histogram of the accuracy of cross-detection of various running postures of the present invention. DETAILED DESCRIPTION

[0075] like Figure 1 As shown, the present invention provides an intelligent running posture correction method based on deep learning, which is specifically implemented through the following steps.

[0076] Step 1, dataset construction phase, collects and annotates image data, performs augmentation operations on image data, including but not limited to geometric adjustment, elastic deformation, grid distortion, image filtering, box filtering, Gaussian filtering, median filtering, bilateral filtering, and random brightness and contrast adjustment of color channels, to obtain the human pose estimation dataset O * . Analyze the existing running posture types, divide the original data images into two running posture states: flying and landing, construct a comparative dataset, and obtain the running posture detection dataset O. Compared with discrete datasets, comparative datasets can avoid the learning and detection difficulties caused by the rapid changes in human posture during running. Among them, the running posture types include,

[0077] Striding: Striding causes the center of gravity to fluctuate, which consumes more energy. The knees are also overextended, causing the heels to land on the ground, which puts too much impact on the knees.

[0078] Sitting while running: Sitting while running will lead to insufficient extension and kicking, low running efficiency, and increase spinal pressure, which may cause low back pain;

[0079] Hyperextension of the knees: The incorrect running posture of hyperextension of the knees will cause the landing point to be in front of the center of gravity, and the heel will touch the ground, which will directly transfer the force of the ground to the knees, causing excessive force on the knees. In the long run, it will cause knee pain;

[0080] Running with your head tilted back: Running with your head tilted back will over-tense your back muscles, shift your center of gravity backward, and put unnecessary pressure on your knees.

[0081] Crossed arm swing: The wrong running posture of crossing the arms will cause the torso to twist too much, making the body difficult to balance and putting the knee at risk of injury;

[0082] Running with your feet turned outward: Running with your feet turned outward can put abnormal stress on the ankles, calves, and knees, increasing the risk of knee and ankle injuries.

[0083] Knees turning inward: The wrong running posture with knees turning inward will greatly increase the pressure on the knees and calves, which can easily lead to knee and ankle pain;

[0084] Hunched back: The incorrect running posture with hunched back will increase the pressure on the cervical spine and cause discomfort and pain in the shoulders and neck; it will affect the smoothness of breathing and reduce the stability of the torso.

[0085] Step 2: Human body posture estimation model training phase, using human body posture estimation dataset O * Train the human pose estimation neural network with Blazepose as the core to obtain the human pose estimation model. The human pose estimation neural network structure with Blazepose as the core specifically includes:

[0086] The backbone network uses a lightweight convolutional neural network to extract deep features of images;

[0087] Heat map sub-network, generating 3D human key points P l A heat map is used to represent the likelihood of various parts of the human body;

[0088] Regression subnetwork, used to calculate the 3D human key points P l The precise coordinates of the location;

[0089] Partial affinity field, used to connect 3D human key points P l , and predict the positions of the occluded joints to form a complete skeleton of the human body.

[0090] Step 3: In the running posture detection model construction phase, the sample o in the running posture detection dataset O is l Input the human posture estimation model one by one, l represents the number of samples, and each sample o l Including the image and its annotation, the image width is W, the height is H, the number of channels is G, represented by a vector V∈R W×H×G , V refers to the quantity, R refers to the matrix, the key point p l Normalized, the original coordinates of each key point are Normalization is performed as follows:

[0091]

[0092] in is the normalized key point The coordinate of the i-th key point on the x-axis, is the normalized key point The coordinate of the i-th key point on the y-axis, where is the normalized key point The coordinate of the i-th key point on the z-axis, i = 1, 2, 3, 4, ..., 33, all 33 three-dimensional human key points P l , Construct running posture detection model C.

[0093] like Figure 2 As shown, the 33 3D human key points P of the sample are obtained. l , respectively are nose 0, inner side of left eye 1, left eye 2, outer side of left eye 3, inner side of right eye 4, right eye 5, outer side of right eye 6, left ear 7, right ear 8, left mouth 9, right mouth 10, left shoulder 11, right shoulder 12, left elbow 13, right elbow 14, left wrist 15, right wrist 16, left little finger 17, right little finger 18, left finger 19, right finger 20, left thumb 21, right thumb 22, left hip joint 23, right hip joint 24, left knee 25, right knee 26, left ankle 27, right ankle 28, left heel 29, right heel 30, left index finger 31, right index finger 32.

[0094] Align record c in running posture detection model C l The sample o of the corresponding dataset O l , after alignment, delete all samples that have no corresponding o l or record c l Ensure that the data in the dataset and the records in the running posture detection model are synchronized to avoid mismatches between the dataset data and the key point data.

[0095] Compare each record c in the running posture detection model C one by one l With other records c other , mark out records that do not meet expectations, including missing or abnormal key point data caused by severe occlusion, and delete all records marked as outliers. Ensure the rationality of the data and avoid a small number of abnormal data affecting the results.

[0096] Step 4: Enter the detection phase, obtain the real-time running video stream transmitted by the user terminal, and process the video stream frame by frame.

[0097] Step 5, feature extraction stage, input the current frame into the human posture estimation model, calculate the 3D human key points of the current frame, and record the 23 relative positions between these 33 key points according to the connection relationship of the human joints corresponding to the key points as 23 feature vectors. Figure 3As shown, the 23 eigenvectors are left hip joint 23-right hip joint 24, left shoulder 11-right shoulder 12, left shoulder 11-left elbow 13, right shoulder 12-right elbow 14, left elbow 13-left wrist 15, right elbow 14-right wrist 16, left hip joint 23-left knee 25, right hip joint 24-right knee 26, left knee 25-left ankle 27, right knee 26-right ankle 28, left shoulder 11-left wrist 15, right shoulder 12-right The vector formed by wrist 16, left hip joint 23—left ankle 27, right hip joint 24—right ankle 28, left wrist 15—left hip joint 23, right wrist 16—right hip joint 24, left shoulder 11—left ankle 27, right shoulder 12—right ankle 28, left elbow 13—right elbow 14, left knee 25—right knee 26, left wrist 15—right wrist 16, left ankle 27—right ankle 28, nose 0—left shoulder 11—right shoulder 12.

[0098] Each eigenvector has three dimensions, forming a feature matrix. The process of extracting the feature matrix from 33 three-dimensional human key points can be expressed as: l )=R (3*33) →R (3*23) .

[0099] The feature matrix of the current frame F0∈R 3×23 ,

[0100]

[0101] The video to be detected is divided into several frames and processed frame by frame. The frame to be processed is the current frame, x1~x 23 is the component of each feature on the x-axis in the feature matrix F0 of the current frame, y1~y 23 is the component of each feature on the y-axis in the feature matrix F0 of the current frame, z1~z 23 is the component of each feature on the z-axis in the feature matrix F0 of the current frame.

[0102] Extract the running posture detection model O into the c of a single record from the N records of a certain running posture l Feature matrix F l The characteristic matrix D that constitutes the mth running posture m , m refers to the mth wrong running posture, recorded as

[0103] D m =[F1,F2,F3,......,F N ]∈R 3×23×N

[0104]

[0105] Among them, X1~X 23 For a single record cl The characteristic matrix F l The component of each feature on the x-axis, Y1~Y 23 For a single record c l The characteristic matrix F l The component of each feature on the y-axis, Z1~Z 23 For a single record c l The characteristic matrix F l The component of each feature in on the z-axis.

[0106] Step 6, classification stage, flip the feature matrix of the current frame along the x-axis to obtain the flipped feature matrix F0′, which is expressed as:

[0107] Feature matrix of the current frame

[0108] Flipped feature matrix

[0109] First calculate the feature matrix F0 of the current frame and the single record c l The characteristic matrix F l The Minkowski distance between the two, and then calculate the flipped feature matrix F0′ and the single record c l The characteristic matrix F l The Minkowski distance between

[0110] Among them, the Minkowski distance is calculated by the following formula:

[0111]

[0112] where weight x is the axis weight of the 23 eigenvectors on the x-axis, weight y is the axis weight of the 23 features on the y-axis, weight z is the axis weight of the 23 features on the z-axis, q is the norm of the Minkowski distance, j refers to the j-th eigenvector, j = 1, 2, 3, 4, ..., 23.

[0113] A dual-queue method is used to process data, and the two queues implement different functions. The first queue is used to exclude abnormal data to avoid data that has a significant impact on the classification results. The second queue is used to achieve accurate classification after eliminating abnormal data.

[0114] The first queue calculates R in sequence 3×23 Under the matrix, the feature matrix F0 of the current frame and its flipped feature matrix F0′ are compared with the single record c in the running posture correction model. l The characteristic matrix F lThe Minkowski distance of is calculated and multiplied by the corresponding axis weight. The maximum value of all recorded calculation results is taken as the maximum distance, which can be expressed as,

[0115]

[0116] Among them, max{} is the function to find the maximum value, weight j (j=1,2,3,4,....,23) is the weight corresponding to each feature.

[0117] In order to reduce the impact of the difference in shooting direction on the classification results, the smaller value of the two maximum distances obtained by taking the feature matrix F0 of the current frame and its flipped feature matrix F0′ is recorded as distance max , this smaller value indicates that the current frame and the running posture detection model are in the same direction, which can reflect the distance between the current frame and a certain running posture in the running posture detection model. The specific implementation is,

[0118] distance max =min{max{|F l -F0|},max{|F l -F0′|}}

[0119] Where min{} is the function to find the minimum value,

[0120] When an abnormality occurs at a test point, the maximum distance max It will amplify abnormal errors and make abnormal test points easier to eliminate;

[0121] The second queue calculates R in turn 3×23 Under the matrix, the feature matrix F0 of the current frame and its flipped feature matrix F0′ are compared with the single record c in the running posture detection model. l The characteristic matrix F l The Minkowski distance of , and multiply it by the corresponding axis weight. Take the arithmetic mean of all the record calculation results as the average distance average .

[0122] As with the first queue, to avoid the influence of the difference in shooting direction, the smaller value of the two average distances obtained by the original feature matrix F0 and the flipped feature matrix F0′ is taken. The smaller value indicates that the current frame is in the same direction as the running posture detection model.

[0123] distance average =min{average{|F l -F0|},average{|F l -F0′|}},

[0124]

[0125] Among them, average{} is the function for finding the average value.

[0126] Compared with the first queue, the second queue weakens the impact of extreme values on the overall situation, but assigns corresponding weights to each dimension of each feature, making it possible to more accurately and reasonably obtain the fit between the current frame and a certain running posture in the running posture detection model.

[0127] The feature matrix F0 of the current frame is compared with a single record c in N records of a certain running posture. l The characteristic matrix F l , the obtained N average distances are arranged from small to large, and the frequency of a certain running posture in the first k average distances is counted as the confidence value of the existence of a certain running posture in the current frame. By traversing each running posture, the confidence value of each running posture in the current frame can be obtained.

[0128] Where k is the number of nearest neighbors. Experiments show that the best effect is achieved when k is 12.

[0129] Step 7, evaluation stage, through the confidence value of a certain running posture, use the exponentially weighted moving average method to calculate the smoothed confidence value r ′, and provide running posture evaluation and correction suggestions based on this, which can be expressed as the following formula:

[0130]

[0131] Where μ is the smoothing coefficient, r is the current frame number, fps is the total frame number, value r is the confidence of the fps-r frame, value r ′ is the confidence after smoothing. Through experiments, it is determined that μ = 0.35 is the most effective value.

[0132] like Figure 4 As shown, the confidence value and the smoothed confidence value r ′In the curve about the number of frames, the confidence value after smoothing r The exponentially weighted moving average method can smooth time series data, producing stable time-series-based gesture recognition confidence evolution data, enabling efficient tracking and predictive analysis of time series data. This can address the problem of rapid fluctuations in gesture classification confidence, where extreme maximum and minimum values have a significant impact on the results.

[0133] The probability of a runner having a certain incorrect running posture can be expressed as follows:

[0134]

[0135] Among them, α represents the probability adjustment constant, which is used to give different influencing factors to various running postures, and λ represents the probability normalization factor, which is used to normalize the final output probability Possibility to 0%. 100%, n valid Indicates all current valid frames, n repeat It represents the number of times a certain running posture appears repeatedly in all valid frames. The calculation formula is:

[0136]

[0137] n repeat =∑repeat(value′ r ),(r∈n valid )

[0138] Where repeat() is a function that calculates the membership of the fps-rth frame.

[0139] Calculate the severity of a runner's incorrect running posture using the following formula:

[0140]

[0141] Where β represents the severity adjustment constant, which is used to give different influencing factors to various running postures, and η represents the severity normalization factor, which is used to normalize the severity of the final output to 0%. 100%, n value It represents the sum of the confidence of a certain running posture in all valid frames, and its calculation formula is:

[0142] n value =∑value r ,(r∈n valid ).

[0143] Step 8: If the video stream has not ended, return to step 4 and update the running posture evaluation and correction suggestions until the video stream ends.

[0144] Below, the present invention further describes and illustrates the effects of the present invention by showing experimental data.

[0145] Table 1 Test results of various running postures accuracy, precision, recall rate, and F1 comprehensive evaluation indicators

[0146]

[0147] As shown in the table above, the deep learning-based intelligent running posture correction method of the present invention is effective in detecting certain incorrect running postures. To reduce errors, 500 samples of each running posture were randomly selected for testing, and the accuracy, precision, recall, and F1 score were calculated.

[0148] Accuracy refers to the proportion of correctly predicted samples to all samples. The calculation formula is:

[0149] Precision refers to the proportion of samples that are actually incorrect running postures among all samples that are predicted to be incorrect running postures. The calculation formula is:

[0150]

[0151] Recall refers to the proportion of samples that are correctly predicted to be a certain wrong running posture among all samples that are actually a certain wrong running posture. The calculation formula is:

[0152]

[0153] The F1 comprehensive evaluation index F1Score is the harmonic mean of precision and recall, which is used to comprehensively evaluate the performance of the model. The calculation formula is:

[0154]

[0155] Among them, TP is true positive, which indicates the number of correctly predicted incorrect running postures, TN is true negative, which indicates the number of correctly predicted incorrect running postures, FP is false positive, which indicates the number of incorrectly predicted incorrect running postures, and FN is false negative, which indicates the number of incorrectly predicted incorrect running postures.

[0156] Multiple metrics can comprehensively evaluate the effectiveness of the classification method of the present invention, especially when detecting complex problems, where a single accuracy rate may not effectively reflect the model's performance. Verification shows that the present invention's intelligent running posture correction method based on deep learning has an accuracy rate exceeding 90% in detecting all types of incorrect running postures, an average precision of 77.84%, an average recall of 78.15%, and an average F1 score of 74.14%.

[0157] like Figure 5As shown in the figure, 400 samples of each running posture were collected from seven volunteers, for a total of 4,000 samples. A mixed cross-test was performed on various running postures, generating an accuracy histogram. The figure shows the test results for different running postures under different conditions, including comparative data for various postures. The horizontal axis represents the incorrect running posture categories predicted by the method of the present invention, including striding, sitting running, knee hyperextension, sideways running, hunched back, no arm swing, excessive arm swing, crossed arms, pigeon-toed, and pigeon-toed. The vertical axis represents the actual incorrect running posture categories of the samples, and the height indicates the number of samples detected for each running posture under different conditions, which can be used to represent the predicted accuracy. Different conditions were used for classification testing for different running postures. Different color blocks and light-dark contrasts are used in the figure to represent categories such as "striding," "sitting running," and "knee hyperextension." When testing the accuracy of each incorrect running posture category, other incorrect running posture categories were also included. The figure shows the accuracy of each incorrect running posture category when it is interfered with by other incorrect running posture categories. For example, the horizontal axis is a column of striding running, which represents 400 samples whose actual running posture is striding running, and the vertical axis is the number of samples predicted by the method of the present invention as a certain incorrect running posture. Among them, the vast majority can be correctly predicted as striding running, and a small number are predicted as excessive arm swing. This is because striding running is generally accompanied by excessive arm swing, and there are almost no samples predicted as other incorrect running postures. It can be seen that the method of the present invention has a good effect in predicting incorrect running postures. Figure 5 As can be seen, the number of samples detected in certain incorrect running posture categories is significantly higher than in other categories, indicating that these postures are more frequent in the sample data. Furthermore, the classification and recognition of the samples are generally accurate, with only a few samples misidentified and no significant outliers. This also demonstrates that the present invention can effectively identify and detect various incorrect running postures in a wide range of application scenarios.

[0158] The present invention can be applied to a treadmill and, in combination with an equipped camera, can capture a runner's running video and display correction suggestions on the screen. At the same time, the present invention can also be applied to a mobile phone, can be lightweight, and can be deployed on the mobile phone in the form of an app. The mobile phone camera can capture running videos, correct incorrect running postures anytime and anywhere, and provide a certain degree of real-time motion correction. It can also be deployed in other sports scenes for application as needed. For example, an IoT camera can be installed on the side of a runway. When someone passes by, the runner's personal information can be obtained through facial recognition technology, and the runner's running video can be captured. Cloud computing technology can be used to accurately analyze the running posture, and correction suggestions can be fed back to the user terminal in real time. The application range is wide.

Claims

1. An intelligent running posture correction method based on deep learning, characterized in that: The following steps are included: Step 1: Dataset construction phase. Collect and manually annotate data. Augment existing data by geometric flipping and affine transformation to obtain human pose estimation dataset O * , analyze the existing running posture types, construct the dataset according to the rules of horizontal comparison dataset, and obtain the running posture detection dataset O; Step 2: Human body posture estimation model training phase, using human body posture estimation dataset O * Train the human pose estimation neural network with Blazepose as the core to obtain the human pose estimation model; Step 3: In the running posture detection model construction phase, the sample o in the running posture detection dataset O is l Input the human body posture estimation model one by one, l represents the sample number, and obtain the three-dimensional human body key point P of the sample l , build running posture detection model C, record c in running posture detection model C l Perform alignment and cleaning, detect and remove outliers; Step 4: Enter the detection phase, obtain the real-time running video stream transmitted by the user terminal, and process the video stream frame by frame; Step 5, feature extraction stage, input the current frame into the human posture estimation model, calculate the 3D human key points of the current frame, extract the feature matrix of the current frame, obtain the feature matrix F0 of the current frame, and extract the single record c in the running posture detection model C. l The characteristic matrix F l The characteristic matrix D that constitutes a certain running posture m , wherein the video to be detected is divided into several frames and then processed frame by frame, and the frame to be processed is the current frame; Step 6, classification stage, flip the feature matrix of the current frame along the x-axis to obtain the flipped feature matrix F0′. First calculate the feature matrix F0 of the current frame and the sum of the feature matrix F0 and the single record c l The characteristic matrix F l The Minkowski distance between the two, and then calculate the flipped feature matrix F0′ and the single record c l The characteristic matrix F l The Minkowski distance between the two running positions is calculated, the double queue method is used to process the data, and the voting method is used to obtain the confidence value of a certain running posture. Step 7: Evaluation. Using the confidence value of a particular running posture, the exponentially weighted moving average method is used to calculate the probability and severity of a runner's incorrect running posture. This provides running posture evaluation and correction advice. Step 8: If the video stream has not ended, return to step 4 and update the running posture evaluation and correction suggestions until the video stream ends.

2. The intelligent running posture correction method based on deep learning according to claim 1, characterized in that: The method of geometric flipping and affine transformation to augment existing data is to perform transformation operations on the original image data, including but not limited to geometric adjustment, elastic deformation, grid distortion, box filtering, Gaussian filtering, median filtering, bilateral filtering and random brightness and contrast adjustment of color channels in image filtering; the rule of the running posture detection dataset O is to divide the original data image into two running posture states: taking off and landing, and construct a comparison dataset.

3. The intelligent running posture correction method based on deep learning according to claim 2, characterized in that: The human pose estimation neural network structure with Blazepose as the core is as follows: the backbone network uses a lightweight convolutional neural network to extract deep features of the image; Heat map sub-network, generating 3D human key points P l A heat map is used to represent the likelihood of various parts of the human body; Regression subnetwork, predicting 3D human key points P l The precise coordinates of the location; Partial affinity field, used to connect 3D human key points P l , and predict the positions of the occluded joints to form a complete skeleton of the human body.

4. The method for correcting running posture based on deep learning according to claim 3, characterized in that: The calculation rule of 3D human key points is: a single image and its annotation, each image has a width of W, a height of H, and a number of channels of G, represented by a vector V∈R W×H×G , V refers to the quantity, R refers to the matrix, the key point p l Normalized, the original coordinates of each key point are Normalization is performed as follows, in is the normalized key point The coordinate of the i-th key point on the x-axis, is the normalized key point The coordinate of the i-th key point on the y-axis, where is the normalized key point The coordinate of the i-th key point on the z-axis, i = 1, 2, 3, 4, ..., 33, all 33 three-dimensional human key points P l , The alignment and cleaning rules of 3D human key points are as follows: dataset O aligns each sample o l The corresponding record c in the running posture detection model C l , after alignment, delete all samples that have no corresponding o l or record c l ; The rule for detecting and removing outliers is to detect a record c in the running posture detection model C. l With other records c other Perform a comparison and mark out records that do not meet expectations, including missing or abnormal key point data caused by severe occlusion, and delete all records marked as outliers.

5. The method for correcting running posture based on deep learning according to claim 4, characterized in that: The feature extraction process is to record the 23 relative positions between the 33 key points according to the connection relationship of the human joints corresponding to the key points as 23 feature vectors, each of which has three dimensions. The process of extracting the feature matrix from 33 3D human key points can be expressed as: l )=R (3*33) →R (3*23) , The feature matrix of the current frame F0∈R 3×23 , where x1~x 23 is the component of each feature on the x-axis in the feature matrix F0 of the current frame, y1~y 23 is the component of each feature on the y-axis in the feature matrix F0 of the current frame, z1~z 23 is the component of each feature on the z-axis in the feature matrix F0 of the current frame, Extract the running posture detection model O into the c of a single record from the N records of a certain running posture l Feature matrix F l The characteristic matrix D that constitutes the mth running posture m , m refers to the mth wrong running posture, recorded as <h2 style=";text-align:left;direction:ltr">D<h2 style=";text-align:left;direction:ltr"> m <h2 style=";text-align:left;direction:ltr"> (F1,F2,F3,......,F<h2 style=";text-align:left;direction:ltr"> N <h2 style=";text-align:left;direction:ltr"> ]∈R<h2 style=";text-align:left;direction:ltr"> 3×23×N Among them, X1~X 23 For a single record c l The characteristic matrix F l The component of each feature on the x-axis, Y1~Y 23 For a single record c l The characteristic matrix F l The component of each feature on the y-axis, Z1~Z 23 For a single record c l The characteristic matrix F l The component of each feature in on the z-axis.

6. The method for correcting running posture based on deep learning according to claim 5, characterized in that: The flipped feature matrix F0′ can be expressed as, Feature matrix of the current frame Flipped feature matrix The feature matrix F0 of the current frame and the single record c in the model l The characteristic matrix F l The generation rule of the Minkowski distance between is: where weight x is the axis weight of the 23 eigenvectors on the x-axis, weight y is the axis weight of the 23 features on the y-axis, weight z is the axis weight of the 23 features on the z-axis, q is the norm of the Minkowski distance, j refers to the j-th eigenvector, j = 1, 2, 3, 4, ..., 23.

7. The method for correcting running posture based on deep learning according to claim 6, characterized in that: The double-queue method realizes different functions. The first queue is used to eliminate abnormal data to avoid data that has a significant impact on the classification results. The second queue is used to achieve accurate classification after eliminating abnormal data, including: The first queue calculates R in sequence 3×23 Under the matrix, the feature matrix F0 of the current frame and its flipped feature matrix F0′ are compared with the single record c in the running posture correction model. l The characteristic matrix F l The Minkowski distance of , and multiply it by the corresponding axis weight, take the maximum value of all recorded calculation results as the maximum distance, expressed as, Among them, max{} is the function for finding the maximum value, weight j (j=1,2,3,4,....,23) is the weight corresponding to each feature, Take the feature matrix F0 of the current frame and its flipped feature matrix F0′ to get the smaller of the two maximum distances, denoted as distance max ,This smaller value indicates that the current frame and the running posture detection model are in the same direction, reflecting the distance between the current frame and a certain running posture in the running posture detection model, which is expressed as, distance max =min{max{|F l -F0|},max{|F l -F0′|}} Where min{} is the function to find the minimum value, When an abnormality occurs at a test point, the maximum distance max It will amplify abnormal errors and make abnormal test points easier to eliminate; The second queue calculates R in turn 3×23 Under the matrix, the feature matrix F0 of the current frame and its flipped feature matrix F0′ are compared with the single record c in the running posture detection model. l The characteristic matrix F l Minkowski distance, multiplied by the corresponding axis weight, and the arithmetic mean of all recorded calculation results is taken as the average distance average , Take the smaller value of the two average distances obtained from the original feature matrix F0 and the flipped feature matrix F0′. The smaller value indicates that the current frame and the running posture detection model are in the same direction. distance average =min{average{|F l -F0|},average{|F l -F0′|}}, Among them, average{} is the function for finding the average value.

8. The method for correcting running posture based on deep learning according to claim 7, characterized in that: The voting rule is to calculate the feature matrix F0 of the current frame and compare it with a single record c in N records of a certain running posture. l The characteristic matrix F l , the obtained N average distances are arranged from small to large, and the frequency of a certain running posture in the first k average distances is counted as the confidence value of the existence of a certain running posture in the current frame. By traversing each running posture, the confidence value of each running posture in the current frame can be obtained.

9. The method for correcting running posture based on deep learning according to claim 8, characterized in that: The confidence level is smoothed using the exponentially weighted moving average method, which is implemented as follows: Where μ is the smoothing coefficient, r is the current frame number, fps is the total frame number, value r is the confidence of the fps-r frame, value r ′ is the confidence after smoothing; The calculation formula for the probability of a runner having a certain incorrect running posture is: Among them, α represents the probability adjustment constant, which is used to give different influencing factors to various running postures, and λ represents the probability normalization factor, which is used to normalize the final output probability Possibility to n valid Indicates all current valid frames, n repeat It represents the number of times a certain running posture appears repeatedly in all valid frames. The calculation formula is: n repeat =∑repeat(value′ r ),(r∈n valid ) Where repeat() is a function that calculates the membership of the fps-r frame; The calculation formula for the severity of a runner's incorrect running posture is: Where β represents the severity adjustment constant, which is used to give different influencing factors to various running postures; η represents the severity normalization factor, which is used to normalize the final output severity to 0% □ 100%, and n value It represents the sum of the confidence of a certain running posture in all valid frames, and its calculation formula is: n value =∑value r ,(r∈n valid )。

Images