Physical training posture correction system and method

Abstract

The invention provides a physical training posture correction system and method, and the method comprises the steps: obtaining a physical training posture image, carrying out the human posture estimation of the physical training posture image based on an improved HRNet network structure, obtaining a thermodynamic diagram of key points of a human skeleton, and carrying out the decoding operation of the obtained thermodynamic diagram of the key points of the human skeleton, human body posture features are extracted based on the human body skeleton key point coordinates; comparison evaluation is carried out on the extracted human body posture features and standard action posture features based on a DTW evaluation model; non-standard actions are extracted; the non-standard actions are displayed to trainees for posture correction. The posture detection of physical training can be realized, whether a nonstandard training posture exists or not is judged according to the posture detection, if yes, the nonstandard training posture is displayed to trainees for correction, and use is convenient.

Description

Technical Field

[0001] The present invention relates to the technical field of sports training, and in particular to a sports training posture correction system and method. Background Art

[0002] In sports training, the accuracy and standardization of posture are crucial to improving training effects and reducing sports injuries. Existing sports training posture correction methods usually rely on the experience of coaches or use basic visual feedback systems. Traditional posture correction detection methods have insufficient accuracy and require wearing a large number of sensors to achieve the correction function, which is inconvenient to use. Therefore, it is very necessary to design a sports training posture correction system and method. Summary of the invention

[0003] In order to overcome the deficiencies of the prior art, the object of the present invention is to provide a sports training posture correction system and method.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] The present invention provides a sports training posture correction system, comprising:

[0006] An image acquisition module, used for acquiring sports training posture images;

[0007] Human body posture estimation module, used for human body posture estimation;

[0008] A decoding module, used for decoding the human posture estimation result into the coordinates of key points of human skeleton;

[0009] A human posture feature extraction module is used to extract human posture features based on the coordinates of key points of human skeleton;

[0010] An irregular action judgment module is used to extract irregular actions based on human posture features;

[0011] The display module is used to display irregular movements to training personnel.

[0012] The present invention also provides a method for correcting a sports training posture, comprising:

[0013] Step 100: Acquire a sports training posture image;

[0014] Step 200: performing human posture estimation on the sports training posture image based on the improved HRNet network structure to obtain a heat map of key points of the human skeleton;

[0015] Step 300: decoding the acquired heat map of the key points of the human skeleton to obtain the coordinates of the key points of the human skeleton;

[0016] Step 400: extracting human body posture features based on the coordinates of key points of the human skeleton;

[0017] Step 500: Based on the DTW evaluation model, the extracted human body posture features are compared and evaluated with the standard action posture features to extract the non-standard actions;

[0018] Step 600: Show the irregular movements to the trainer for posture correction.

[0019] Preferably, in step 200, human posture estimation is performed on the sports training posture image based on the improved HRNet network structure to obtain a thermal map of key points of the human skeleton, specifically:

[0020] Construct a human posture estimation model based on the improved HRNet network structure;

[0021] Train the human posture estimation model based on a preset training set;

[0022] The sports training posture images are input into the trained human posture estimation model to obtain the heat map of the key points of the human skeleton.

[0023] Preferably, the specific and improved HRNet network structure is as follows:

[0024] The two low-resolution sub-networks in the fourth stage of the traditional HRNet network structure were pruned, and its high-resolution network branches were retained to reduce excessive calculation of the model. Two resolution blocks were added between the first and second stages of the traditional HRNet network structure to enhance the ability to extract low-resolution features. The CBAM attention mechanism was added after the network switching unit of the traditional HRNet network structure to improve the model's ability to extract important features in the channel dimension and spatial dimension.

[0025] Preferably, in step 300, the obtained heat map of the key points of the human skeleton is decoded, specifically:

[0026] The heat map of the key points of the human skeleton is obtained, and the heat map information is converted based on the moving pixel distance decoding. The coordinate decoding is achieved by moving the hot spot probability coordinates obtained at the highest point of the heat map to the hot spot probability coordinates of the second highest point by 1 / 4 of the length.

[0027] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0028] The present invention provides a sports training posture correction system and method, the system includes an image acquisition module, a human body posture estimation module, a decoding module, a human body posture feature extraction module, an irregular action judgment module, and a display module; the method includes acquiring a sports training posture image, performing human body posture estimation on the sports training posture image based on an improved HRNet network structure, acquiring a heat map of key points of a human body skeleton, decoding the acquired heat map of key points of a human body skeleton, obtaining coordinates of key points of a human body skeleton, extracting human body posture features based on the coordinates of key points of a human body skeleton, comparing and evaluating the extracted human body posture features with standard action posture features based on a DTW evaluation model, extracting irregular actions, displaying the irregular actions to a trainer, and performing posture correction. The present invention can realize posture detection of sports training, and judge whether there is an irregular training posture based on it, and if so, display it to the trainer for correction, so as to facilitate use. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0030] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0031] Figure 2 It is a schematic diagram of the traditional HRNet network structure;

[0032] Figure 3 It is a schematic diagram of the Basic residual unit structure;

[0033] Figure 4 Schematic diagram of feature fusion structure;

[0034] Figure 5 This is a schematic diagram of the improved HRNet network structure;

[0035] Figure 6 Schematic diagram of the CBAM attention mechanism structure;

[0036] Figure 7 To construct a schematic diagram of the human skeleton;

[0037] Figure 8 This is a schematic diagram of the limb angle model;

[0038] Fig. 9 It is a schematic diagram of the limb combination structure;

[0039] Fig.10Schematic diagrams showing examples of different limb orientations;

[0040] Fig.11 Schematic diagram of limb orientation characteristics. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The purpose of the present invention is to provide a sports training posture correction system and method, which can realize sports training posture detection and determine whether there is an irregular training posture. If so, it will be displayed to the training personnel for correction to facilitate use.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] The present invention provides a sports training posture correction system, comprising:

[0045] An image acquisition module, used for acquiring sports training posture images;

[0046] Human body posture estimation module, used for human body posture estimation;

[0047] A decoding module, used for decoding the human posture estimation result into the coordinates of key points of human skeleton;

[0048] A human posture feature extraction module is used to extract human posture features based on the coordinates of key points of human skeleton;

[0049] An irregular action judgment module is used to extract irregular actions based on human posture features;

[0050] The display module is used to display irregular movements to training personnel.

[0051] like Figure 1 As shown, the present invention also provides a method for correcting a sports training posture, comprising:

[0052] Step 100: Acquire a sports training posture image;

[0053] Step 200: performing human posture estimation on the sports training posture image based on the improved HRNet network structure to obtain a heat map of key points of the human skeleton;

[0054] Step 300: decoding the acquired heat map of the key points of the human skeleton to obtain the coordinates of the key points of the human skeleton;

[0055] Step 400: extracting human body posture features based on the coordinates of key points of the human skeleton;

[0056] Step 500: Based on the DTW evaluation model, the extracted human body posture features are compared and evaluated with the standard action posture features to extract the non-standard actions;

[0057] Step 600: Show the irregular movements to the trainer for posture correction.

[0058] In step 100, the sports training posture images may be collected by taking photos or the like, and this application does not impose any limitation on this.

[0059] In step 200, human posture estimation is performed on the sports training posture image based on the improved HRNet network structure to obtain a thermal map of key points of the human skeleton, specifically:

[0060] Construct a human posture estimation model based on the improved HRNet network structure;

[0061] Train the human posture estimation model based on a preset training set;

[0062] The sports training posture images are input into the trained human posture estimation model to obtain the heat map of the key points of the human skeleton;

[0063] The following is a detailed introduction to this part:

[0064] First, the traditional HRNet network structure is introduced. HRNet predicts the key points of human skeleton by combining convolutional neural network with heat map regression. Most human posture estimation networks refer to the pyramid residual model to extract multi-scale features from the input image. This kind of network front-end model has too many layers, and the feature information is easily lost when it is transmitted between too many layers. The HRNet network structure can generate high-resolution feature maps with rich semantic information. The convolutional layers of each subnet also use skip connections, which contain multiple residual units, and can effectively avoid overfitting problems.

[0065] The traditional HRNet network structure is as follows Figure 2As shown in the figure, the input of HRNet is directly connected to two 3×3 convolutions. The image resolution is reduced to 1 / 4 of the original resolution through convolution operation, and the output regression heat map resolution is the same as this resolution. The HRNet network structure can be divided into four stages. The first stage consists of a high-resolution sub-network. The second, third, and fourth stages reduce the high-resolution image by downsampling and transfer it to the low-resolution sub-network, which not only reduces the resolution, but also doubles the number of channels. However, the upper backbone network always maintains high-resolution feature transmission. The number of network branches included in different stages is different, and information fusion is performed multiple times at different scales to effectively avoid information loss.

[0066] The first stage of the HRNet network contains 4 residual units, which are similar to the residual units in the ResNet network model. They are all composed of Bottleneck residual units. The specific structure of Bottleneck is as follows: Figure 3 As shown in (a), the input feature map is convolved with sizes of 1×1, 3×3, and 1×1 to output a new feature map. The input feature map is fused with the feature map output by the Bottleneck residual unit. In the second stage, there are 2 multi-resolution blocks, each of which consists of 4 Basic residual units. The structure of the Basic residual unit is as follows: Figure 3 As shown in (b), the input feature map is output as a new feature map after two interconnected 3×3 convolutions. The feature map formed by the Basic residual unit is fused with the input feature map. There are 4 and 3 multi-resolution blocks in the third and fourth stages respectively, and the resolution block structure is the same as that of the first stage.

[0067] The feature scale of the HRNet network input image is a 256-size RGB color image. Before entering the first stage, the image will undergo two convolutions to reduce the feature scale of the input image so that it can be passed to the next layer. The feature scale is reduced from 256 to 64. The processed data then enters the first stage with 4 Bottleneck modules. The first stage outputs 256 feature maps and passes them to the second stage for processing. In the second stage, in addition to a high-resolution network, there is a low-resolution sub-network branch. After being processed by the Basic residual unit, the high-resolution sub-network will obtain 32 64-scale feature maps. The low-resolution sub-network will obtain 64 feature maps of 32 scales. There are two low-resolution sub-networks in the third stage. The data processing method is the same as the previous stage. The high-resolution sub-network in the third stage will obtain 32 feature maps of 64 scales. The two low-resolution sub-networks will obtain 64 feature maps of 32 scales and 128 feature maps of 16 scales respectively. After the fourth stage, 32 feature maps of 64 scales, 64 feature maps of 32 scales, 28 feature maps of 16 scales and the latest 256 feature maps of 8 scales will be obtained. Finally, the multi-resolution scale features obtained in the fourth stage are fused as the final heat map output;

[0068] The HRNet network structure is connected and feature fused through exchange units at each stage. The structure is as follows: Figure 4 As shown in the figure, the feature map undergoes 3×3 convolution with a step size of 2, and performs 2x, 4x or 8x downsampling operations to add a low-resolution sub-network. The number of channels of the newly added low-resolution sub-network is twice the number of channels of the previous sub-network, and the resolution is reduced to 1 / 2 of the resolution of the previous sub-network. The upsampling uses 1×1 upsampling convolution to restore the low-resolution feature channel;

[0069] The nearest neighbor interpolation algorithm is used in the upsampling of the exchange unit of HRNet. When the image is transformed, some pixels cannot be directly assigned values. The pixel coordinates of the two-dimensional image are limited to integers. When performing feature fusion, the size of the feature map needs to be expanded, but there will be some pixels in the feature map that cannot be directly mapped. Therefore, the nearest neighbor interpolation algorithm is used to mark the points closest to the pixels that cannot be mapped from the original feature map pixel matrix, and then confirm the specific values ​​of the pixels that cannot be mapped. The HRNet network structure performs multiple upsampling and downsampling operations on the feature map, so that the high-resolution backbone network can obtain a wider receptive field feature from the remaining low-resolution sub-networks. Through multiple multi-resolution scale fusion and high-resolution feature mapping of the exchange unit, the large-scale high-resolution information branch will make up for the problem that the small-scale low-resolution branch does not fully express the detailed information of the human posture, and the high-resolution representation of the local information can help the low-resolution representation provide a more accurate global description, such as whether there is a lack of local key points.

[0070] However, the calculation of the traditional HRNet is too large, so the present invention improves and optimizes it. The optimized network model is as follows Figure 5 As shown in the figure, the specific optimization process is:

[0071] First, the fourth stage, which accounts for the largest proportion of network parameters, is streamlined. Two low-resolution sub-networks in the fourth stage are pruned, and its high-resolution network branch is retained to reduce the pressure of excessive calculation on the overall model. However, simply removing the two low-resolution sub-networks will reduce the model's extraction of low-dimensional features and reduce the number of cross-resolution fusions. In order to compensate for the performance degradation caused by removing the network branch, two resolution blocks are added between the first and second stages to enhance the extraction performance of low-resolution features. At the same time, the CBAM attention mechanism is added after the network exchange unit to improve the network's extraction of important features in the channel dimension and spatial dimension.

[0072] Introduce the CBAM attention mechanism:

[0073] The dual attention module (Convolutional Block Attention Module, CBAM) proposed by Woo et al. that integrates spatial attention and channel attention designs the two pooling operations of global average pooling and global maximum pooling into the channel attention mechanism, so that the channel attention mechanism can help the network highlight the use of key position feature information. At the same time, combined with the spatial attention mechanism, the network can focus on key information from both channel and space aspects. The convolutional attention module CBAM can well retain the key information in the image. The channel attention module and the spatial attention module are interconnected in various ways, but after a large number of experimental verifications, the effect of serially connecting the spatial attention module through the channel attention module is relatively the best. The structure of the CBAM module is as follows: Figure 6 As shown;

[0074] The CBAM module uses maximum pooling and average pooling in both the channel attention module and the spatial attention module. It can analyze not only the overall semantic information but also the local semantic information. CBAM is a lightweight module that can be easily embedded in a convolutional neural network. For example, if there is an intermediate layer feature map F, the output after CBAM is as follows:

[0075]

[0076]

[0077] Where Mc is the attention feature extraction in the channel dimension, M s It is the attention feature extraction in the spatial dimension. After the feature map is extracted in the channel dimension, it is multiplied with the original feature map to obtain F'. After the feature map is extracted in the spatial dimension, F' is multiplied with itself to finally obtain the output F".

[0078] The channel attention mechanism in CBAM uses maximum pooling and global average pooling when compressing feature maps, and obtains two description methods. The calculation process is shown in the following formula:

[0079]

[0080] go through Global max pooling and After global average pooling, the channel information is filtered through a shared network consisting of two fully connected layers MLP, and the overall channel attention map M c , the calculation process is:

[0081] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))

[0082] Different from the channel attention module, the spatial attention module pays more attention to the position information. The feature extraction calculation process of the spatial dimension is:

[0083]

[0084] The present invention provides an embodiment, and the preset data set can be selected from the COCOtrain2017 data set, and other data sets can also be selected according to specific needs, and this application does not impose any limitation on this.

[0085] In step 300, the obtained heat map of the key points of the human skeleton is decoded, specifically:

[0086] Get the heat map of the key points of the human skeleton, convert the heat map information based on the moving pixel distance decoding, and realize the coordinate decoding by moving the hot spot probability coordinates obtained at the highest point of the heat map to the hot spot probability coordinates of the second highest point by 1 / 4 of the length. The moving pixel distance decoding method is introduced in detail:

[0087] For high-resolution input images, most network models will first downsample the image to a small-resolution image before training the network. When the key point heat map obtained by the trained model is converted to the key point coordinates, the recovery process often produces quantization errors due to the different resolutions. The main idea of ​​decoding the key point heat map based on the moving pixel distance is to move the highest response coordinate of the key point detected by the heat map to the second highest response coordinate position by 1 / 4 of the distance to reduce the quantization error generated during the conversion process. This method has higher detection accuracy than directly selecting the highest response coordinate for decoding. The specific operations are as follows:

[0088] Assume that the highest response value in the key point heat map is H1, the second highest response value is H2, P represents the predicted key point position in the training model, and the P value is:

[0089]

[0090] Since each training model uses different resolutions when preprocessing the image, corresponding changes need to be made during restoration. α is the multiple by which the image resolution is reduced during model preprocessing. The final conversion from the key point heat map to the coordinates P1 of the key point in the image is:

[0091] P1=α*P

[0092] The key point coordinate decoding based on the moving pixel distance can effectively compensate for the pixel deviation caused by the image downsampling in the network model.

[0093] Step 400, extracting human posture features based on the coordinates of key points of the human skeleton, specifically:

[0094] First, the human skeleton spatial coordinate system needs to be reconstructed: when the human skeleton key points are detected in a two-dimensional image through a human posture estimation network, the pixel position coordinates of the skeleton key points are usually established in a Cartesian coordinate system with the upper left corner of the original image, and the coordinates of a certain pixel point in the upper left corner are used as the origin of the coordinate system to describe other pixels. However, in actual applications, it is found that when the human body has different relative positions and different resolution scales in the video, the coordinate information data of the skeleton point will be inconsistent, and relatively standard movements may be misjudged as irregular movements. Therefore, it is necessary to reconstruct the reference position of the coordinate system to reduce the possibility of misjudgment caused by different relative positions and different resolution scales. By analyzing the bone structure and combining the human body movement mechanism, the nose key point with serial number 0 is on the line connecting the human body's trunk, and the human spine trunk can well express various states of movement. At the same time, the recognition accuracy of the skeleton point No. 0 is relatively stable, so the key point No. 0 is selected as the origin of the new coordinate system, the horizontal right direction is the X-axis direction of the new coordinate system, and the vertical downward direction is the Y-axis direction of the new coordinate system. Figure 7The coordinate diagram of the human skeleton in the new coordinate system is shown;

[0095] After reconstructing the reference coordinate system, the corresponding original key point coordinates must also be changed accordingly. iL =(x iL ,y iL ) represents the coordinates of the i-th human skeleton key point in the original coordinate system, M iN =(x iN ,y iN ) represents the coordinates of the key points of the human skeleton under the newly created coordinate system. The key point of the nose with serial number 0 is used as the origin of the coordinate system under the new coordinate system. Its coordinates in the new coordinate system are M 0N =(0,0), and the remaining 16 key coordinates of the human skeleton are calculated by taking the coordinates of the key point with serial number 0 as reference, as shown in the following formula:

[0096] M iN =(x iN ,y iN )=M iL -M 0N =(x iL -x 0N ,y iL -y 0N )

[0097] Feature extraction of human skeleton limb angles:

[0098] When the human body moves, the limb angle formed by adjacent bone points can well represent the movement posture. The cosine value of the angle between the template posture and the limb segment vector of the same posture to be detected is calculated using the coordinates of the bone joint points. The cosine value of the angle is used to indirectly judge the angle between the two limbs with the same posture, thereby obtaining the similarity between the template posture and the posture to be detected, such as Figure 8 As shown, it shows the joint combination of the right arm of the human body, in which the limb angle model is formed by three bone points;

[0099] This part of the model is composed of 4 bone key points. Three adjacent key points can form a limb angle. The model in the figure shows the limb angle with the right elbow key point No. 8 as the vertex. In order to express the human skeleton model more accurately and reasonably, the limb angle is expressed in detail. The limb angle with the A bone point as the vertex is expressed as (A D , B, C), and similar expressions with B and C as vertices are (A, B D , C) and (A, B, C D );

[0100] From the shape of the constructed model, it can be seen that it is very convenient and reasonable to use the trigonometric cosine theorem to calculate the limb angle. When the side lengths a, b, and c of the triangle to be solved are known, the angle can be calculated:

[0101] b 2 =a 2 +c 2 -2accosθ

[0102] When the coordinates of the three vertices of the constructed triangle are known, the angle can be calculated using the following vector inner product method:

[0103]

[0104] After the image passes through the HRNet human posture estimation network, the corresponding coordinate information of the specific human skeleton points can be obtained through key point heat map decoding, so it is more convenient and quick to solve the limb angle using the vector inner product method. Figure 8 For example,

[0105] The angle of the limb with A as the vertex is expressed as (A D , B, C), the three vertex coordinates are A = (x1, y1), B = (x2, y2), C = (x3, y3), and the vector is calculated from the three vertex coordinates and Then we can ask (A D , B, C) Limb angle, calculated as follows:

[0106]

[0107] When using limb angles to express movement postures, in order to accurately divide the specific components of the human skeleton represented by each group of angles, the human skeleton is divided into 6 limb components according to the distribution and corresponding characteristics of the limb angles, such as Fig. 9 The figure shows the human skeleton division diagram. The six limb combinations and the limb angles contained in the limb combinations are summarized as follows:

[0108] The first part of the limb combination is composed of three bone key points, numbered 0, 5, and 6. The three bone points represent the nose, left shoulder, and right shoulder respectively. The first part of the limb combination mainly expresses the positional relationship between the head and the shoulder and the limb trunk, and contains three limb angles, which are (0 D ,5,6),(0,5 D ,6),(0,5,6 D );

[0109] The second part of the limb combination is composed of four key bone points, numbered 5, 7, 9, and 11. The four bone points represent the left shoulder, left elbow, left wrist, and left hip respectively. The second part of the limb combination mainly expresses the movement posture of the left arm and the relative position of the left arm to the main body trunk. It contains four limb angles, which are (5 D ,7,9),(5,7 D ,9),(5,7,9 D )、(5 D , 7, 11);

[0110] The third part of the limb combination is composed of four key bone points, numbered 6, 8, 10, and 12. The four bone points represent the right shoulder, right elbow, right wrist, and right hip respectively. The third part of the limb combination mainly expresses the movement posture of the right arm and the relative position of the right arm to the main body trunk. It contains four limb angles, which are (6 D ,8,10),(6,8 D , 10), (6, 8, 10 D )、(6 D , 8, 12);

[0111] The fourth part of the limb combination is composed of four skeletal key points, numbered 5, 6, 11, and 12. The four skeletal key points represent the left shoulder, right shoulder, left hip, and right hip, respectively. The fourth part of the limb combination mainly expresses the movement posture of the human body trunk. It originally contains 12 limb angles, but due to redundant calculations, after removing the composite angles, 4 limb angles that actually participate in the calculation are selected from the 8 limb angles, which are (5 D ,6,11),(5,6 D ,12),(5,6 D ,11)、(5 D , 6, 12);

[0112] The fifth part of the limb combination is composed of three key bone points, numbered 11, 13, and 15. The three bone points represent the left hip, left knee, and left ankle respectively. The fifth part of the limb combination mainly expresses the movement posture of the left leg. Two limb angles are selected, which are (11 D , 13, 15), (11, 13 D , 15);

[0113] The sixth limb combination consists of three key bone points, numbered 12, 14, and 16. The three bone points represent the right hip, right knee, and right ankle, respectively. The sixth limb combination mainly expresses the movement posture of the right leg. Two limb angles are selected, which are (12 D , 14, 16), (12, 14 D , 16);

[0114] In order to more accurately represent the relative position of the left and right leg limbs and the main body, add (5, 11 D , 13) and (6, 12 D ,14) Limb angle auxiliary judgment,All human bodies contain 21 limb angle feature information in total, and all limb angle features are integrated into a set V as a limb angle feature set.

[0115] Feature extraction of limb combination center vector:

[0116] By analyzing the divided limb combinations and the overall human skeleton structure, it is found that in the model composed of the second, third, fifth and sixth limb combinations that express the movement posture of human limbs, there is a phenomenon that the angles are the same but the limbs point in different directions, such as Fig.10 As shown, it represents the movement posture of the left arm of the upper body of the human body;

[0117] It can be clearly seen from the figure that according to the division of the left arm limb combination, the two actions have the same bone angle, but they obviously do not belong to the same action. The difference lies in the different directions of the forearm limbs. If only the limb angle is used for judgment, there will be similar misjudgment possibilities for the limb combination of the human body. In order to eliminate such misjudgment, the angle between the center point vector of the human arm limb combination part and the center point vector of the leg limb combination part and the defined human body medial axis vector is used as the judgment feature to solve the problem of the same limb angle but different directions. The specific calculation steps are as follows:

[0118] 1. Calculate the center point coordinates of the arm limb combination, leg limb combination and head limb combination. When calculating the center point coordinates of the two double-arm limb combinations, remove the 11-left hip and 12-right hip bone points that were originally divided into the left and right double-arm limb combinations to reduce the amount of calculation. After processing, these five limb combinations are composed of three bone key points, so assuming that the coordinates of the three bone key points contained in one of the limb combinations are A=(x1,y1), B=(x2,y2), C=(x3,y3), and the center point H=(x0,y0), the coordinates of the center point H can be calculated as follows:

[0119]

[0120] 2. According to the reconstructed human skeleton coordinate system, the skeleton coordinate point with serial number 0 is taken as the coordinate origin, and the vector from the skeleton point with serial number 0 pointing to the center point of the head and limb combination is defined as the human body medial axis vector, and the vector from the skeleton point with serial number 0 pointing to the center point of the limb combination is recorded as the limb vector;

[0121] 3. Let the human body's medial axis vector be The limb vector is Calculate the angle between two vectors:

[0122]

[0123] like Fig.11 The limb vector diagram of the two-arm limb combination is shown. According to the above calculation steps, a total of four limb orientation angle features can be obtained, and the four features are combined into a set S.

[0124] In step 500, the DTW-based evaluation model compares and evaluates the extracted human posture features with the standard action posture features to extract non-standard actions, specifically:

[0125] When using the DTW algorithm to match human motion postures, the Euclidean distance between the two comparison sequences can be used to determine the similarity between a single frame of motion during the entire motion process. The larger the Euclidean distance, the lower the similarity, and the smaller the Euclidean distance, the higher the similarity. The regular path distance length is used to represent the similarity of the overall motion. Similarly, the smaller the regular path, the higher the similarity, and the larger the regular path, the lower the similarity. However, using the distance length as the final evaluation rule is not intuitive enough, and the algorithm evaluation mechanism needs to be redefined;

[0126] Through the above extraction of human skeleton posture feature information, we obtain 21 limb angle information used to represent the 6 limb combinations and 4 feature information used to represent the limb orientations. The two feature information used to represent the motion posture are fused into a set T containing 25 feature information:

[0127] T=[θ1,θ2,θ3…θ 25 ]

[0128] The standard template action posture feature sequence is set as A = {L1, L2, L3...L n}, the action posture feature sequence to be matched is set as B = {K1, K2, K1…K m}, set A and set B represent human body posture feature vectors, where each single vector contains 25 posture feature information L1 = (x i1 ,x i2 …x i25 ), K j =(y j1 ,y j2 …y j25 ), calculate the Euclidean distance to represent the similarity between the i-th frame of the action to be matched and the j-th frame of the standard template action:

[0129]

[0130] In order to convert the distance into a more intuitive scoring mechanism, the obtained Euclidean distance is normalized using a commonly used normalization algorithm;

[0131] After the single-frame posture similarity evaluation index is successfully converted, the method of calculating the regularized path distance will also change. The changed calculation formula is shown as follows:

[0132] D(i,j)=value(i,j)+max[D(i-1,j),D(i,j-1),D(i-1,j-1)]

[0133] The final similarity score between the to-be-matched action sequence with a length of m and the standard template sequence is as follows. The similarity evaluation score with a full score of 100 is divided into three stages: less than 60 points is unqualified, greater than or equal to 60 points and less than 90 points is qualified, and greater than or equal to 90 points is excellent. The similarity of the motion posture to be matched is evaluated;

[0134]

[0135] Set the single frame score below 60 points as an unqualified single frame, and set the posture feature sequence of an unqualified single frame of a certain action sequence to be matched as P = (x1, x2…x 25 )), the standard template posture sequence corresponding to the single frame is Q = (y1, y2…y 25 , calculate the angle deviation of each limb in the two frames, and the ratio of the limb angle deviation to the standard limb angle is recorded as L, and D k The final judgment is made based on the value of, as shown below:

[0136]

[0137] D k =1-I k

[0138] When D k When the value of is less than the set threshold, the k-th limb angle of the single frame in the sequence to be matched belongs to an irregular action angle, and an irregular action is obtained;

[0139] The irregular movements are displayed to the trainers through the display module for posture correction.

[0140] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0141] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A sports training posture correction system, characterized in that: include: An image acquisition module, used for acquiring sports training posture images; Human body posture estimation module, used for human body posture estimation; A decoding module, used for decoding the human posture estimation result into the coordinates of key points of human skeleton; A human posture feature extraction module is used to extract human posture features based on the coordinates of key points of human skeleton; An irregular action judgment module is used to extract irregular actions based on human posture features; The display module is used to display irregular movements to training personnel.

2. A method for correcting posture during sports training, characterized in that: include: Step 100: Acquire a sports training posture image; Step 200: performing human posture estimation on the sports training posture image based on the improved HRNet network structure to obtain a heat map of key points of the human skeleton; Step 300: decoding the acquired heat map of the key points of the human skeleton to obtain the coordinates of the key points of the human skeleton; Step 400: extracting human body posture features based on the coordinates of key points of the human skeleton; Step 500: Based on the DTW evaluation model, the extracted human body posture features are compared and evaluated with the standard action posture features to extract the non-standard actions; Step 600: Show the irregular movements to the trainer for posture correction.

3. The method according to claim 2, characterized in that In step 200, human posture estimation is performed on the sports training posture image based on the improved HRNet network structure to obtain a thermal map of key points of the human skeleton, specifically: Construct a human posture estimation model based on the improved HRNet network structure; Train the human posture estimation model based on a preset training set; The sports training posture images are input into the trained human posture estimation model to obtain the heat map of the key points of the human skeleton.

4. The method according to claim 2, characterized in that: The specific and improved HRNet network structure is as follows: The two low-resolution sub-networks in the fourth stage of the traditional HRNet network structure were pruned, and its high-resolution network branches were retained to reduce excessive calculation of the model. Two resolution blocks were added between the first and second stages of the traditional HRNet network structure to enhance the ability to extract low-resolution features. The CBAM attention mechanism was added after the network switching unit of the traditional HRNet network structure to improve the model's ability to extract important features in the channel dimension and spatial dimension.

5. The method according to claim 2, characterized in that: In step 300, the obtained heat map of the key points of the human skeleton is decoded, specifically: The heat map of the key points of the human skeleton is obtained, and the heat map information is converted based on the moving pixel distance decoding. The coordinate decoding is achieved by moving the hot spot probability coordinates obtained at the highest point of the heat map to the hot spot probability coordinates of the second highest point by 1 / 4 of the length.

Images