3D (three-dimensional) human body posture estimation method aiming at walking race action extraction in RGB (red, green and blue) video

Abstract

The invention discloses a 3D (three-dimensional) human body posture estimation method aiming at walking race action extraction in an RGB (red, green and blue) video, which belongs to the technical field of crossing of computer vision and sports science, and specifically comprises the following steps: obtaining a 2D human body joint point matrix corresponding to each frame in the RGB video based on a 2D human body posture estimation network; constructing 3D human body posture estimation input data containing skeleton length prior information; and constructing and training a 3D human body posture estimation network, and finally realizing the 3D position of the human body joint point of each frame in the RGB video to be estimated under the camera coordinate system. According to the method, the skeleton length prior information is introduced into the 3D human body posture estimation network architecture, so that the posture estimation error is reduced, stable walking race action parameters are calculated according to the output 3D posture, the walking race action in the RGB video is extracted, and the extraction accuracy is high.

Description

Technical Field

[0001] The present invention belongs to the interdisciplinary technical field of computer vision and sports science, and in particular relates to a 3D human body posture estimation method for extracting race walking actions in RGB videos. Background Art

[0002] Motion capture technology based on posture estimation aims to track and record human motion in real time without delaying the digitization process. It has been widely used for quantitative purposes in several sports disciplines and activities, such as technical and competition evaluation, and is the main source of data to help researchers gain knowledge of human motion mechanics. Motion capture systems are mainly divided into three categories: optical systems, non-optical systems, and markerless systems. Optical motion capture systems track markers placed on human landmarks to triangulate their 3D positions. Although optical motion capture systems have been used in sports, the system requires long preparation and post-processing in the laboratory to clear all corrupted data, which is time-consuming and costly. Non-optical systems use sensors attached to body parts to send or read their position and orientation data in space, but any metal objects near the sensors may introduce distortion in the magnetic field, making their measurement accuracy susceptible to external environmental influences. At the same time, this method requires the subject to carry accessories, which will affect their free and unhindered movement. Markerless systems do not require subjects to wear special equipment for tracking. Only camera observation and algorithm analysis can achieve real-time and accurate capture of human motion. In recent years, with the widespread popularity of deep learning technology, many revolutionary technologies have been born in the field of markerless motion capture, and human pose estimation (HPE) technology based on deep learning has been more widely studied.

[0003] The meaning of human posture estimation technology is: to perceive the human motion state in the image through a computer vision system, and to represent it in space in the form of a human skeleton, and to analyze the image or video sequence provided by the imaging device to obtain relevant information of the human posture of the current frame, such as the spatial position of the human skeleton point, the category of the skeleton point, the direction of movement, etc. Human posture estimation technology can be divided into two categories according to the output data dimension: two-dimensional HPE and three-dimensional HPE (3D human posture estimation). Three-dimensional HPE describes the human posture more comprehensively and better meets practical needs. The international research time on three-dimensional HPE is relatively short. Most of the existing research work is for three-dimensional HPE of monocular images. Before the emergence of deep learning methods, most of them use pattern recognition to match the human model with the human body in the video, but the effect is generally poor and depends on manual annotation. After the emergence of deep learning, 3D human posture estimation technology has developed rapidly, the accuracy has increased rapidly, and it can be applied to real life. Compared with traditional motion capture methods, 3D human posture estimation technology does not rely on annotation, and only uses cameras to collect input, which has obvious cost advantages. Therefore, the present invention uses 3D human posture estimation technology to extract walking action in RGB video.

[0004] For the scenario of extracting race walking motion in RGB video, 3D human pose estimation can extract the 3D motion pose of the human body from the video, thereby deriving the specific motion parameters required for race walking motion extraction and analysis. However, for the specific form of race walking, traditional 3D human pose estimation technology faces several major challenges. On the one hand, race walking is a highly standardized and complex sport involving complex leg and upper body movements, so highly accurate pose information is required to accurately evaluate the performance of athletes. On the other hand, for the same human body in the same video, there is often a certain error in the 3D human pose of each frame output by the network, including errors in the length and direction of the human skeleton, which will lead to deviations in the final output results. Summary of the invention

[0005] Aiming at the problems existing in the traditional 3D human posture estimation technology in the scenario of extracting race walking action in RGB video, the present invention provides a 3D human posture estimation method for extracting race walking action in RGB video, so as to achieve high-accuracy posture information estimation.

[0006] In order to achieve the above purpose, the technical method adopted by the present invention is as follows:

[0007] A 3D human body posture estimation method for extracting race walking action in RGB video includes the following steps:

[0008] Step 1: Construct a training set consisting of multiple RGB videos containing race walkers. Based on the 2D human posture estimation network, estimate the pixel coordinate values ​​of the human joints in each frame of the RGB video in the pixel coordinate system. After distortion correction and internal reference normalization, obtain the N×2-dimensional 2D human joint matrix corresponding to each frame; where N is the total number of human joints;

[0009] Step 2: Construct 3D human posture estimation input data, specifically:

[0010] Step 2.1: Concatenate the 2D human joint point matrices of adjacent frames in the RGB video to obtain a 2D posture sequence of T×N×2 dimensions, denoted as Where T is the sequence length;

[0011] Step 2.2, construct a human skeleton graph with human joints as points and bones between adjacent human joints as edges; and number the N human joints with the hip joint as 0;

[0012] Step 2.3: Use the shortest path algorithm to determine the bones that need to be passed between the hip joint and any human joint point in the human skeleton diagram, and obtain the maximum number of bones M that need to be passed; since the lengths of the bones of the race walker in the RGB video are known, the bone length set B corresponding to the bones passed between the hip joint and each human joint point is obtained in the order of numbering. 0，i = {B 0，m , …, B n，i}, i = 0, 1, ..., N-1, m, n ∈ [0, 1, ..., N-1]; where i is the number of the human joint point; m and n are the numbers of the human joint points that need to be passed between the hip joint and the i-th human joint point; B 0，m B is the bone length between the hip joint and the mth human joint point; n，i B is the bone length between the nth human joint point and the ith human joint point; 0，0 ={0};

[0013] Step 2.4: Set B by bone length 0，i = {B 0，m , …, B n，i} is a row vector, and a 2D bone length matrix of N×M dimensions corresponding to each frame in the RGB video is constructed; where the last blank position of each row that is less than M bone lengths is filled with 0, so that all elements in the first row are 0;

[0014] Step 2.5: Concatenate the 2D bone length matrices of adjacent frames in the RGB video to obtain a T×N×M dimensional bone length prior sequence, denoted as

[0015] Step 2.6: 2D pose sequence and bone length prior sequence Splice to get 3D human posture estimation input data

[0016] Step 3: Build a 3D human posture estimation network to train the corresponding RGB videos in the training set. As input, the coordinates of each human joint point in each frame of each RGB video in the camera coordinate system are the estimation target, and minimizing the loss is the training goal. The 3D human pose estimation network is trained to obtain the trained 3D human pose estimation network;

[0017] Step 4: After the RGB video to be estimated is processed in step 2, the 3D human posture estimation input data corresponding to the RGB video to be estimated is obtained. Input into the trained 3D human pose estimation network, and output the 3D positions of N human joints in each frame of the RGB video to be estimated in the camera coordinate system.

[0018] Furthermore, according to the 3D positions of N human joints in each frame of the output RGB video to be estimated in the camera coordinate system, the motion parameters required for the race walking action are calculated, including the angle between the upper and lower arms, the angle between the upper arm and the shoulder, the angle between the upper and lower legs, and the angle between the two thighs.

[0019] Furthermore, the 3D human pose estimation network adopts a space-time-cross-attention structure, including a first linear projection layer, a codec and a linear regression layer in sequence; wherein the codec includes a D-layer encoder structure and a D-layer decoder structure stacked in sequence, and D is an even number; the encoder structure includes a spatial Transformer block and a temporal Transformer block in sequence; the decoder structure is specifically a cross-attention Transformer block;

[0020] First, input it into the first linear projection layer to project the third-dimensional data into high-dimensional features to obtain feature data d m is the feature dimension; at the same time, Linearly projected into feature data in the first layer of cross-attention Transformer block After D layers of alternating spatial Transformer blocks and temporal Transformer blocks, They are processed together in the first cross-attention Transformer block, and after processing by D layers of cross-attention Transformer blocks and linear regression layers, the 3D positions of N human joints in the camera coordinate system are output.

[0021] Furthermore, the input of the spatial Transformer block in the first layer of the encoder structure is embedded with a spatial position encoding, specifically a learnable parameter matrix.

[0022] Furthermore, the temporal position coding is embedded inside the first layer of the temporal Transformer block of the encoder structure, specifically, the rotational position coding of the improved Attention weight matrix is ​​improved by setting a window of size ω in the position matrix of the original Attention weight, the value of ω does not exceed the network training length, and the absolute position coding exceeding ω is uniformly set to ω, thereby modifying the position matrix of the improved Attention weight, that is:

[0023]

[0024] Furthermore, the loss function L used in training in step 3 is a joint loss function, and the formula is:

[0025] L=L MPJPE +λ t L time +λ m L Bone

[0026]

[0027]

[0028]

[0029] Where, L MPJPE is the average position error of the human joints; L time is the time consistency loss; L Bone is the bone vector loss; λ and λ m All are loss weight coefficients; is the 3D coordinate of the i-th human joint point corresponding to the t-th frame of the sequence; is the 3D coordinate of the i-th human joint point corresponding to the t-1th frame of the sequence; is the real 3D coordinate of the i-th human joint point corresponding to the t-th frame of the sequence, which is a known quantity in each RGB video; is the estimated bone length of the ith bone corresponding to the tth frame of the sequence; is the actual bone length of the ith bone in the tth frame of the sequence.

[0030] Further, Calculated by Euclidean distance.

[0031] Furthermore, N=17, and the joints of the human body are hip joint, right hip joint, right knee joint, right ankle joint, left hip joint, left knee joint, left ankle joint, thoracic spine joint, neck joint, nose joint, head joint, left shoulder joint, left elbow joint, left hand joint, right shoulder joint, right elbow joint and right hand joint. In this case, M is 5.

[0032] Furthermore, the 2D human pose estimation network in step 1 is specifically a publicly available network for 2D human pose estimation, such as a ViT (Vision Transformer) network, a stacked hourglass network, an Openpose network, and the like.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] Aiming at the situation that the bone length of athletes can be obtained more conveniently in the race walking training scenario, the present invention proposes a 3D human posture estimation method for extracting race walking actions in RGB videos. Based on the 3D human posture estimation network architecture, combined with visual imaging and other knowledge, bone length prior information is introduced into the 3D human posture estimation network architecture to reduce the error of posture estimation. Stable race walking action parameters are calculated according to the output 3D posture, so as to realize the extraction of race walking actions in RGB videos with high extraction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 This is a flow chart of the 3D human posture estimation method for extracting race walking action in RGB video proposed in Example 1;

[0037] Figure 2 A schematic diagram of a human skeleton constructed in Example 1;

[0038] Figure 3 This is a schematic diagram of the structure of the 3D human posture estimation network constructed in Example 1. DETAILED DESCRIPTION

[0039] In order to further understand the present invention, preferred embodiments of the present invention are described below in conjunction with examples, but it should be understood that these descriptions are only for further illustrating the features and advantages of the present invention, rather than limiting the claims of the invention.

[0040] Example 1

[0041] This embodiment proposes a 3D human posture estimation method for extracting race walking action in RGB video. The process is as follows: Figure 1 As shown, the following steps are included:

[0042] Step 1: Construct a training set consisting of multiple RGB videos containing race walkers. Based on the 2D human posture estimation network, estimate the pixel coordinate values ​​of the human joints in each frame of the RGB video in the pixel coordinate system. After distortion correction and internal reference normalization, obtain the N×2-dimensional 2D human joint matrix corresponding to each frame; where N is the total number of human joints. The specific process is as follows:

[0043] Get the camera intrinsic parameters of the camera that shoots RGB video (with arbitrary resolution), including a 3×3 projection matrix and a distortion matrix consisting of multiple meridional distortion parameters and tangential distortion parameters;

[0044] The camera internal parameters of the camera are obtained by using Zhang's calibration method or other methods, or by consulting the camera manufacturer;

[0045] The Zhang calibration method is a method for calculating the optimal camera internal parameters by taking multiple shots of a chessboard at different angles through a camera; the chessboard is a calibration board composed of black and white squares, the side lengths of the black and white squares are uniform and known, and it is used as a calibration object for camera calibration (an object mapped from the real world to the digital image);

[0046] Based on the 2D human posture estimation network, the pixel coordinate values ​​of the human joint points in each frame of the RGB video are estimated in the pixel coordinate system to obtain the N×2-dimensional 2D human joint point matrix X before correction. 2D ; Wherein, N=17 is the total number of human joints;

[0047] The 2D human body posture estimation network can be implemented using any network or method. The estimation result of the 2D posture estimation will affect the final estimation result to a certain extent. This embodiment uses the ViT network;

[0048] According to the distortion matrix, the 2D human joint point matrix X before correction 2D To perform distortion correction, you can use the distortion correction function in the open source toolbox such as Opencv to correct the pixel coordinate position of the distorted human joint points to the correct pixel coordinate position, and obtain the N×2-dimensional corrected 2D human joint point matrix X 2D-DC ;

[0049] Based on the inverse matrix of the projection matrix, the corrected 2D human joint point matrix X 2D-DCPerform internal parameter normalization to obtain the N×2-dimensional 2D human joint point matrix X after internal parameter normalization. 2D-No , the operation formula is:

[0050]

[0051] In the formula, is the inverse matrix of the projection matrix, f x is the actual physical length of each pixel in the horizontal direction of the normalized coordinate system of the camera imaging, in meters, y is the actual physical length of each pixel in the ordinate direction in the normalized coordinate system of the camera imaging, in meters, c x is the pixel horizontal coordinate of the imaging center, c y is the pixel ordinate of the imaging center; u and v are the length direction coordinates and height direction coordinates of each human joint point in the 2D human joint point matrix after correction, respectively. The value range of u is [0, w], and the value range of v is [0, h]; X / Z and Y / Z are the length direction coordinates and height direction coordinates of each human joint point in the 2D human joint point matrix after internal reference normalization, respectively;

[0052] Step 2: Construct 3D human posture estimation input data, specifically:

[0053] Step 2.1: Concatenate the 2D human joint point matrices of adjacent frames in the RGB video to obtain a 2D posture sequence of T×N×2 dimensions, denoted as Where T is the sequence length;

[0054] Step 2.2: Use human joints as points and bones between adjacent human joints as edges to construct a human skeleton graph, and number the N human joints with the hip joint as 0, such as Figure 2 As shown, including: No. 0 hip joint, No. 1 right hip joint, No. 2 right knee joint, No. 3 right ankle joint, No. 4 left hip joint, No. 5 left knee joint, No. 6 left ankle joint, No. 7 thoracic spine joint point, No. 8 neck joint point, No. 9 nose joint point, No. 10 head joint point, No. 11 left shoulder joint point, No. 12 left elbow joint point, No. 13 left hand joint point, No. 14 right shoulder joint point, No. 15 right elbow joint point, No. 16 right hand joint point;

[0055] Step 2.3: Use the shortest path algorithm to determine the bones that need to be passed between the hip joint and any human joint point in the human skeleton diagram, and obtain the maximum number of bones M that need to be passed; since the lengths of the bones of the race walker in the RGB video are known, the bone length set B corresponding to the bones passed between the hip joint and each human joint point is obtained in the order of numbering. 0，i = {B 0，m , …, B n，i}, i = 0, 1, ..., N-1, m, n ∈ [0, 1, ..., N-1]; where i is the number of the human joint point; m and n are the numbers of the human joint points that need to be passed between the hip joint and the i-th human joint point; B 0，m B is the bone length between the hip joint and the mth human joint point; n，i B is the bone length between the nth human joint point and the ith human joint point; 0，0 ={0};

[0056] Step 2.4: Set B by bone length 0，i = {B 0，m , …, B n，i} is a row vector, and a 2D bone length matrix of N×M dimensions corresponding to each frame in the RGB video is constructed; where the last blank position of each row that is less than M bone lengths is filled with 0, so that all elements in the first row are 0;

[0057] Step 2.5: Concatenate the 2D bone length matrices of adjacent frames in the RGB video to obtain a T×N×M dimensional bone length prior sequence, denoted as

[0058] Step 2.6: 2D pose sequence and bone length prior sequence Splice to get 3D human posture estimation input data

[0059] Step 3: Build Figure 3 The 3D human pose estimation network shown in the figure adopts a space-time-cross attention structure, including a first linear projection layer, a codec, and a linear regression layer in sequence; wherein the codec includes a D-layer encoder structure and a D-layer decoder structure stacked in sequence, and D is an even number; the encoder structure includes a spatial Transformer block and a temporal Transformer block in sequence; the decoder structure is specifically a cross attention Transformer block;

[0060] First, input it into the first linear projection layer to project the third-dimensional data into high-dimensional features to obtain feature data d m is the feature dimension; at the same time, Linearly projected into feature data in the first layer of cross-attention Transformer block After D layers of alternating spatial Transformer blocks and temporal Transformer blocks, They are processed together in the first cross-attention Transformer block, and after processing by D layers of cross-attention Transformer blocks and linear regression layers, the 3D positions of N human joints in the camera coordinate system are output.

[0061] The input of the spatial Transformer block in the first layer of the encoder structure is embedded with a spatial position encoding, specifically a learnable parameter matrix.

[0062] The temporal position encoding is embedded inside the first layer of the temporal Transformer block of the encoder structure. Specifically, the rotational position encoding of the improved Attention weight matrix is ​​modified by setting a window of size ω in the position matrix of the original Attention weight, the value of ω does not exceed the network training length, and the absolute position encoding exceeding ω is uniformly set to ω, and then the position matrix of the improved Attention weight is modified, that is:

[0063]

[0064] Step 4: Take the RGB video corresponding to the training set As input, the coordinates of each human joint point in each frame of each RGB video in the camera coordinate system are the estimation target, and minimizing the loss is the training goal. The 3D human pose estimation network is trained to obtain the trained 3D human pose estimation network;

[0065] Among them, the loss function L used in training is the joint loss function, and the formula is:

[0066] L=L MPJPE +λ t L time +λ m L Bone

[0067]

[0068]

[0069]

[0070] Where, L MPJPE is the average position error of the human joints; L time is the time consistency loss; L Bone is the bone vector loss; t and λ m All are loss weight coefficients; is the 3D coordinate of the i-th human joint point corresponding to the t-th frame of the sequence; is the 3D coordinate of the i-th human joint point corresponding to the t-1th frame of the sequence; is the real 3D coordinate of the i-th human joint point corresponding to the t-th frame of the sequence, which is a known quantity in each RGB video; is the estimated bone length of the ith bone corresponding to the tth frame of the sequence, which is calculated by Euclidean distance; is the actual bone length of the ith bone in the tth frame of the sequence;

[0071] Step 5: After the RGB video to be estimated is processed in step 2, the 3D human posture estimation input data corresponding to the RGB video to be estimated is obtained. Input into the trained 3D human pose estimation network, and output the 3D positions of N human joints in each frame of the RGB video to be estimated in the camera coordinate system.

[0072] According to the 3D positions of N human joints in each frame of the output RGB video to be estimated in the camera coordinate system, the motion parameters required for the race walking action are calculated, including the angle between the upper and lower arms, the angle between the upper arm and the shoulder, the angle between the upper and lower legs, and the angle between the two thighs.

[0073] For example, calculate the angle θ between the thigh and the thigh. According to the 3D positions of the right hip joint, right knee joint and right ankle joint output in step 5, the spatial coordinates are A(x1, y1, z1), B(x2, y2, z2) and C(x3, y3, z3), respectively. We can get vector AB = (x2-x1, y2-y1, z2-z1) and vector AC = (x3-x1, y3-y1, z3-z1), and then calculate cosθ = (AB·AC) / (|AB|*|AC|).

[0074] The above embodiments are intended to provide a better understanding of the present invention, but are not limited to the optimal implementation mode and do not constitute a limitation on the content and protection scope of the present invention. Any product identical or similar to the present invention obtained by anyone under the inspiration of the present invention or by combining the present invention with the features of other prior arts shall be within the protection scope of the present invention.

Claims

1. A 3D human posture estimation method for extracting race walking action in RGB video, characterized in that: The following steps are involved: Step 1: Construct a training set consisting of multiple RGB videos containing race walkers. Based on the 2D human posture estimation network, estimate the pixel coordinate values ​​of the human joints in each frame of the RGB video in the pixel coordinate system. After distortion correction and internal reference normalization, obtain the N×2-dimensional 2D human joint matrix corresponding to each frame; where N is the total number of human joints; Step 2: Construct 3D human posture estimation input data, specifically: Step 2.1: Concatenate the 2D human joint point matrices of adjacent frames in the RGB video to obtain a 2D posture sequence of T×N×2 dimensions, denoted as Where T is the sequence length; Step 2.2, construct a human skeleton graph with human joints as points and bones between adjacent human joints as edges; and number the N human joints with the hip joint as 0; Step 2.3: Use the shortest path algorithm to determine the bones that need to be passed between the hip joint and any human joint point in the human skeleton diagram, and obtain the maximum number of bones M that need to be passed; obtain the bone length set B corresponding to the bones passed between the hip joint and each human joint point in the order of numbering. 0,i = {B 0,m ,…,B n,i },i=0,1,...,N-1,m,n∈[0,1,...,N-1]; where i is the number of the human joint; m and n are the numbers of the human joints that need to be passed between the hip joint and the i-th human joint; B 0,m B is the bone length between the hip joint and the mth human joint point; n,i B is the bone length between the nth human joint point and the ith human joint point; 0,0 ={0}; Step 2.4: Set B by bone length 0,i = {B 0,m ,…,B n,i } is a row vector, and a 2D bone length matrix of N×M dimensions corresponding to each frame in the RGB video is constructed; where each row fills the last blank position of less than M bone lengths with 0; Step 2.5: Concatenate the 2D bone length matrices of adjacent frames in the RGB video to obtain a T×N×M dimensional bone length prior sequence, denoted as Step 2.6: 2D pose sequence and bone length prior sequence Splice to get 3D human posture estimation input data Step 3: Build a 3D human posture estimation network to train the corresponding RGB videos in the training set. As input, the coordinates of each human joint point in each frame of each RGB video in the camera coordinate system are the estimation target, and minimizing the loss is the training goal. The 3D human pose estimation network is trained to obtain the trained 3D human pose estimation network; Step 4: After the RGB video to be estimated is processed in step 2, the 3D human posture estimation input data corresponding to the RGB video to be estimated is obtained. Input into the trained 3D human pose estimation network, and output the 3D positions of N human joints in each frame of the RGB video to be estimated in the camera coordinate system.

2. The 3D human posture estimation method for extracting race walking action in RGB video according to claim 1, characterized in that: According to the 3D positions of N human joints in each frame of the output RGB video to be estimated in the camera coordinate system, the motion parameters required for the race walking action are calculated.

3. The 3D human posture estimation method for extracting race walking action in RGB video according to claim 1, characterized in that: The 3D human pose estimation network adopts a space-time-cross attention structure, including a first linear projection layer, a codec and a linear regression layer in sequence; wherein the codec includes a D-layer encoder structure and a D-layer decoder structure stacked in sequence; the encoder structure includes a spatial Transformer block and a temporal Transformer block in sequence; the decoder structure is specifically a cross attention Transformer block; First, input it into the first linear projection layer to project the third-dimensional data into high-dimensional features to obtain feature data d m is the feature dimension; at the same time, Linearly projected into feature data in the first layer of cross-attention Transformer block After D layers of alternating spatial Transformer blocks and temporal Transformer blocks, They are processed together in the first cross-attention Transformer block, and after processing by D layers of cross-attention Transformer blocks and linear regression layers, the 3D positions of N human joints in the camera coordinate system are output.

4. The 3D human body posture estimation method for extracting race walking action in RGB video according to claim 3, characterized in that: The input to the spatial Transformer block in the first layer of the encoder structure is embedded with a spatial position encoding, specifically a learnable parameter matrix.

5. The 3D human posture estimation method for extracting race walking action in RGB video according to claim 3, characterized in that: The temporal position encoding is embedded inside the first layer of the temporal Transformer block of the encoder structure. Specifically, the rotational position encoding of the improved Attention weight matrix is ​​modified by setting a window of size ω in the position matrix of the original Attention weight, the value of ω does not exceed the network training length, and the absolute position encoding exceeding ω is uniformly set to ω, and then the position matrix of the improved Attention weight is modified, that is:

6. The 3D human body posture estimation method for extracting race walking action in RGB video according to claim 3, characterized in that: The loss function L used in training in step 3 is the joint loss function, and the formula is: L=L MPJPE +λ t L time +λ m L Bone Where, L MPJPE is the average position error of the human joints; L time is the time consistency loss; L Bone is the bone vector loss; t and λ m All are loss weight coefficients; is the 3D coordinate of the i-th human joint point corresponding to the t-th frame of the sequence; is the 3D coordinate of the i-th human joint point corresponding to the t-1th frame of the sequence; is the real 3D coordinate of the i-th human joint point corresponding to the t-th frame of the sequence, which is a known quantity in each RGB video; is the estimated bone length of the ith bone corresponding to the tth frame of the sequence; is the actual bone length of the ith bone in the tth frame of the sequence.

7. The 3D human body posture estimation method for extracting race walking action in RGB video according to claim 6, characterized in that: Calculated by Euclidean distance.

8. The 3D human posture estimation method for extracting race walking action in RGB video according to claim 1, characterized in that: N=17, the joints of the human body are hip joint, right hip joint, right knee joint, right ankle joint, left hip joint, left knee joint, left ankle joint, thoracic spine joint, neck joint, nose joint, head joint, left shoulder joint, left elbow joint, left hand joint, right shoulder joint, right elbow joint and right hand joint. At this time, M is 5.

Images