A 3D human pose estimation method for extracting a race walk action in an RGB video

Abstract

The application discloses a 3D human posture estimation method for race walk action extraction in an RGB video and belongs to the technical field of computer vision and sports science, in particular to the following: based on a 2D human posture estimation network, a 2D human joint matrix corresponding to each frame in the RGB video is obtained; 3D human posture estimation input data containing bone length prior information is constructed; a 3D human posture estimation network is constructed and trained, and finally the 3D position of a human joint in each frame of the RGB video to be estimated under a camera coordinate system is realized. The application introduces bone length prior information in the 3D human posture estimation network architecture, thereby reducing the posture estimation error, calculating stable race walk action parameters according to the output 3D posture, realizing race walk action extraction in the RGB video, and having high extraction accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and sports science, and particularly relates to a 3D human pose estimation method for race walking action extraction in an RGB video. BACKGROUND

[0002] Motion capture technology based on pose estimation aims to track and record human motion in real time without the need for a delayed digitization process, and has been widely used for quantitative purposes in several kinematic disciplines and activities, such as technical and competition evaluation, and is the main source of data for researchers to obtain knowledge of human kinematics. Motion capture systems are mainly divided into three categories: optical systems, non-optical systems and markerless systems. Optical motion capture systems track markers placed on human landmarks to triangulate their 3D positions. Although optical motion capture systems have been used in sports, the system requires long preparation and post-processing in the laboratory to remove all damaged data, which is time-consuming and costly. Non-optical systems use sensors attached to body parts to send or read their position and orientation data in space, but any metal object near the sensor can introduce distortion in the magnetic field, making its measurement accuracy susceptible to external environmental influences. At the same time, this method requires subjects to carry accessories, which will affect their free and unobstructed movement. Markerless systems do not require subjects to wear special equipment for tracking, and can achieve real-time and accurate capture of human motion through camera observation and algorithm analysis only. In recent years, with the widespread popularity of deep learning technology, many revolutionary technologies have emerged in the field of markerless motion capture, and human pose estimation (HPE) technology based on deep learning has received more extensive research.

[0003] The meaning of human pose estimation technology is that the human motion state in the image is perceived by a computer vision system, and is represented in the form of a human skeleton in space. Through analysis of the image or video sequence provided by the imaging device, relevant information of the current frame human pose is obtained, such as the spatial position of the human skeleton point, the category to which the skeleton point belongs, and the motion direction. Human pose estimation technology can be divided into two categories according to the dimension of the output data: two-dimensional HPE and three-dimensional HPE (3D human pose estimation). Three-dimensional HPE describes the human pose more comprehensively and better meets the actual needs. Internationally, the research on three-dimensional HPE is relatively short. Most of the existing research work is on monocular image three-dimensional HPE. Before the emergence of deep learning methods, most of the pattern recognition methods were used to match the human body model with the human body in the video, but the effect was generally poor and depended on manual annotation. After the emergence of deep learning, 3D human pose estimation technology has developed rapidly, and the precision has improved rapidly. It can be applied to actual life. Compared with the traditional motion capture method, the 3D human pose estimation technology does not depend on labeling and only uses a camera to collect input, which has obvious cost advantage. Therefore, the present application uses 3D human pose estimation technology to extract the running action in the RGB video.

[0004] For the extraction of running action in RGB video, 3D human pose estimation can extract the 3D pose of human motion from the video, thereby deriving the specific action parameters required for running action extraction and analysis. For this specific form of running, traditional 3D human pose estimation technology faces several major challenges. On the one hand, running is a highly standardized and complex movement that involves complex leg and upper body movements, so highly accurate pose information is needed to accurately assess the performance of athletes. On the other hand, for the same human body in the same video, each frame of 3D human pose output by the network often has some errors, including human skeleton length error and human skeleton direction error, which will cause deviation in the final output result. SUMMARY

[0005] In view of the problems of traditional 3D human pose estimation technology in the extraction of running action in RGB video, the present application provides a 3D human pose estimation method for extracting running action in RGB video, which realizes high-accuracy pose information estimation.

[0006] In order to achieve the above purpose, the technical method adopted by the present application is as follows:

[0007] A 3D human pose estimation method for extracting running action in RGB video, comprising the following steps:

[0008] Step 1, construct a training set composed of multiple RGB videos containing race runners, estimate the pixel coordinate values of the human joint points in each frame of the RGB video in the pixel coordinate system based on a 2D human pose estimation network, and after distortion correction and intrinsic normalization processing, obtain the corresponding N×2-dimensional 2D human joint matrix of each frame; wherein N is the total number of human joint points;

[0009] Step 2, construct 3D human pose estimation input data, specifically:

[0010] Step 2.1, splice the 2D human joint matrices of adjacent frames in the RGB video to obtain a T×N×2-dimensional 2D pose sequence, denoted as Where T is the sequence length;

[0011] Step 2.2, construct a human skeleton graph with human joint points as points and the bones between adjacent human joint points as edges; and number the N human joint points with the hip joint as 0;

[0012] Step 2.3, determine the bones required to pass between the hip joint and any human joint in the human skeleton graph using the shortest path algorithm, and obtain the maximum number of bones M required to pass; since the lengths of all bones of the race runner in the RGB video are known, the bone length set B corresponding to the bones passing between the hip joint and each human joint is obtained in order, i.e. 0，i ={B 0，m ,…,B n，i},i=0,1,...,N-1,m,n∈[0,1,...,N-1]; wherein i is the number of human joint points; m and n are the numbers of human joint points required to pass between the hip joint and the ith human joint; B 0，m is the bone length between the hip joint and the mth human joint; B n，i is the bone length between the nth human joint and the ith human joint; B 0，0 ={0};

[0013] Step 2.4, construct a 2D bone length matrix of N×M dimensions corresponding to each frame in the RGB video with the bone length set B 0，i ={B 0，m ,…,B n，i} as a row vector; wherein the last blank position of each row with less than M bones is filled with 0, and the elements of the first row are all 0;

[0014] Step 2.5, splice the 2D bone length matrices of adjacent frames in the RGB video to obtain a T×N×M-dimensional bone length prior sequence, denoted as

[0015] Step 2.6, the 2D pose sequence Bone length prior sequence The data is stitched together to obtain the 3D human pose estimation input data.

[0016] Step 3: Construct a 3D human pose estimation network, using the corresponding RGB videos in the training set. As input, the coordinates of each human joint in each frame of each RGB video in the camera coordinate system are the estimation target, and minimizing the loss is the training objective. The 3D human pose estimation network is trained to obtain the trained 3D human pose estimation network.

[0017] Step 4: After processing the RGB video to be estimated in Step 2, the 3D human pose estimation input data corresponding to the RGB video to be estimated is obtained. Input the data into the trained 3D human pose estimation network, and output the 3D position of N human joints in the camera coordinate system for each frame of the RGB video to be estimated.

[0018] Furthermore, based on the 3D positions of N human joints in the camera coordinate system of each frame in the output RGB video to be estimated, the motion parameters required for the race walking motion are calculated, including the angle between the upper and lower arms, the angle between the upper arm and shoulder, the angle between the thigh and calf, and the angle between the two thighs.

[0019] Furthermore, the 3D human pose estimation network adopts a space-time-cross-attention structure, including a first linear projection layer, an encoder-decoder, and a linear regression layer in sequence; wherein, the encoder-decoder includes a stacked D-layer encoder structure and a D-layer decoder structure, where D takes an even number; the encoder structure includes a spatial Transformer block and a temporal Transformer block in sequence; the decoder structure is specifically a cross-attention Transformer block;

[0020] First, the data is input into the first linear projection layer, which projects the third-dimensional data onto the high-dimensional features to obtain the feature data. d m For feature dimensions; at the same time, The feature data is linearly projected into the first-layer cross-attention Transformer block. After alternating spatial and temporal Transformer blocks in layer D, and The points are processed together in the first cross-attention Transformer block. After processing by the D-layer cross-attention Transformer block and the linear regression layer, the 3D positions of N human joints in the camera coordinate system are output.

[0021] Further, the input embedding space position encoding of the first layer spatial Transformer block of the encoder structure is specifically a learnable parameter matrix.

[0022] Further, the internal embedding time position encoding of the first layer time Transformer block of the encoder structure is specifically a rotation position encoding of the improved Attention weight matrix, which is obtained by setting a window of size ω in the position matrix of the original Attention weight, ω not exceeding the network training length, and uniformly setting the absolute position encoding exceeding ω as ω, and then modifying the position matrix of the improved Attention weight, i.e.

[0023]

[0024] Further, the loss function L used in step 3 during training is a joint loss function, and the formula is:

[0025] L=L MPJPE +λ t L time +λ m L Bone

[0026]

[0027]

[0028]

[0029] In the formula, L MPJPE is the average position error of the human joint; L time is the time consistency loss; L Bone is the skeleton vector loss; λ and λ m are loss weight coefficients; is the 3D coordinate of the i-th human body joint corresponding to the t-th frame of the sequence; is the 3D coordinate of the i-th human body joint corresponding to the t-1-th frame of the sequence; is the real 3D coordinate of the i-th human body joint corresponding to the t-th frame of the sequence, which is a known quantity in each RGB video; is the estimated bone length of the i-th bone corresponding to the t-th frame of the sequence; is the real bone length of the i-th bone corresponding to the t-th frame of the sequence.

[0030] Further, which is calculated by the Euclidean distance.

[0031] Further, N=17, each human joint is respectively hip joint, right hip joint, right knee joint, right ankle joint, left hip joint, left knee joint, left ankle joint, thoracic spine joint, neck joint, nose joint, head joint, left shoulder joint, left elbow joint, left hand joint, right shoulder joint, right elbow joint and right hand joint, at this time M takes 5.

[0032] Further, the 2D human pose estimation network in step 1 is specifically a network for 2D human pose estimation disclosed, such as a ViT (Vision Transformer) network, a stacked hourglass network, an Openpose network and the like.

[0033] Compared with the prior art, the beneficial effects of the present application are as follows:

[0034] In view of the fact that the skeleton length of an athlete can be conveniently obtained in a walking training scene, the present application proposes a 3D human pose estimation method for walking action extraction in an RGB video, on the basis of a 3D human pose estimation network architecture, in combination with visual imaging knowledge, the skeleton length prior information is introduced into the 3D human pose estimation network architecture, so as to reduce the error of pose estimation, according to the output 3D pose, the stable walking action parameters are calculated, the walking action extraction in the RGB video is realized, and the extraction accuracy is high. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0036] Figure 1 The flowchart of the 3D human pose estimation method for walking action extraction in the RGB video proposed for embodiment 1 is shown in the figure;

[0037] Figure 2 The human skeleton diagram constructed in embodiment 1 is shown in the figure;

[0038] Figure 3 The structure diagram of the 3D human pose estimation network constructed in embodiment 1 is shown in the figure. DETAILED DESCRIPTION

[0039] In order to further understand the present application, the preferred embodiments of the present application will be described below in combination with the embodiments, but it should be understood that these descriptions are only for further illustrating the features and advantages of the present application, and are not limiting on the claims of the present application.

[0040] Embodiment 1

[0041] The embodiment proposes a 3D human pose estimation method for race walking action extraction in an RGB video, and a flowchart is shown in the figure, which includes the following steps: Figure 1

[0042] Step 1, a training set composed of multiple RGB videos containing race walking athletes is constructed, based on a 2D human pose estimation network, the pixel coordinate values of the human joint nodes in each frame of the RGB video in the pixel coordinate system are estimated, after distortion correction and intrinsic normalization processing, the corresponding N×2-dimensional 2D human joint node matrix of each frame is obtained; wherein N is the total number of human joint nodes; the specific process is as follows:

[0043] The camera intrinsic parameters of the camera shooting the RGB video (resolution is arbitrary) are obtained, including a 3×3 projection matrix, and a distortion matrix composed of multiple longitudinal distortion parameters and tangential distortion parameters;

[0044] The camera intrinsic parameters of the camera are obtained by Zhang's calibration method or by consulting the camera manufacturer;

[0045] The Zhang's calibration method is a method of calculating the optimal camera intrinsic parameters by shooting a chessboard at different angles by a camera multiple times; the chessboard is a calibration board composed of black and white squares, the side length of the black and white squares is uniform, and the side length is known, which is used as a calibration object (an object mapped from the real world to a digital image) for camera calibration;

[0046] Based on the 2D human pose estimation network, the pixel coordinate values of the human joint nodes in each frame of the RGB video in the pixel coordinate system are estimated, and an N×2-dimensional pre-correction 2D human joint node matrix X 2D is obtained; wherein N=17 is the total number of human joint nodes;

[0047] The 2D human pose estimation network can be realized by using any network or method, and the estimation result of 2D pose estimation will affect the final estimation result to some extent, and the embodiment uses a ViT network;

[0048] According to the distortion matrix, the pre-correction 2D human joint node matrix X 2D is subjected to a distortion correction (Distortion Correction) operation, and the pixel coordinate position of the human joint node after distortion can be corrected to the correct pixel coordinate position by using the distortion correction function in the open source tool box such as Opencv, and an N×2-dimensional post-correction 2D human joint node matrix X 2D-DC is obtained;

[0049] Based on the inverse matrix of the projection matrix, the post-correction 2D human joint node matrix X 2D-DC ​The intrinsic parameter normalization operation is performed to obtain an N×2-dimensional 2D human joint matrix X after intrinsic parameter normalization 2D-No , and the operation formula is:

[0050]

[0051] In the formula, is the inverse matrix of the projection matrix, f x is the actual physical length of each pixel in the horizontal coordinate direction of the normalized coordinate system of the camera imaging, with the unit of meter, f y is the actual physical length of each pixel in the vertical coordinate direction of the normalized coordinate system of the camera imaging, with the unit of meter, c x is the pixel horizontal coordinate of the imaging center, c y is the pixel vertical coordinate of the imaging center; u and v are the length direction coordinate and height direction coordinate of each human joint in the corrected 2D human joint matrix respectively, the value range of u is [0, w], and the value range of v is [0, h]; X / Z and Y / Z are the length direction coordinate and height direction coordinate of each human joint in the 2D human joint matrix after intrinsic parameter normalization respectively;

[0052] Step 2, constructing 3D human pose estimation input data, specifically:

[0053] Step 2.1, splicing the 2D human joint matrices of adjacent frames in the RGB video to obtain a 2D pose sequence of T×N×2 dimension, denoted as wherein T is the sequence length;

[0054] Step 2.2, constructing a human skeleton graph with human joints as points and the bones between adjacent human joints as edges, and numbering the N human joints with the hip joint as No. 0, as shown in Figure 2 , including: No. 0 hip joint, No. 1 right hip joint, No. 2 right knee joint, No. 3 right ankle joint, No. 4 left hip joint, No. 5 left knee joint, No. 6 left ankle joint, No. 7 thoracic spine joint, No. 8 neck joint, No. 9 nose joint, No. 10 head joint, No. 11 left shoulder joint, No. 12 left elbow joint, No. 13 left hand joint, No. 14 right shoulder joint, No. 15 right elbow joint, and No. 16 right hand joint;

[0055] Step 2.3, determining the bones required to pass through between the hip joint and any human joint in the human skeleton graph by using the shortest path algorithm, and obtaining the maximum number M of bones required to pass through; since the lengths of the bones of the track and field athletes in the RGB video are known, the bone length set B 0，i corresponding to the bones passing through between the hip joint and each human joint is obtained in the order of numbering 0，m , …, B n，i}, i = 0, 1, …, N-1, m, n ∈ [0, 1, …, N-1]; where i is the number of human joint; m and n are the numbers of human joints required to pass between the hip joint and the ith human joint; B 0，m is the bone length between the hip joint and the mth human joint; B n，i is the bone length between the nth human joint and the ith human joint; B 0，0 = {0};

[0056] Step 2.4, taking the bone length set B 0，i = {B 0，m , …, B n，i} as a row vector, construct a 2D bone length matrix of N x M dimensions corresponding to each frame in the RGB video; where the last blank position of each row is filled with 0 if there are not enough M bone lengths, then the elements of the first row are all 0;

[0057] Step 2.5, splice the 2D bone length matrices of adjacent frames in the RGB video to obtain a bone length prior sequence of T x N x M dimensions, denoted as

[0058] Step 2.6, splice the 2D pose sequence and the bone length prior sequence to obtain the 3D human pose estimation input data

[0059] Step 3, construct a 3D human pose estimation network as shown in Figure 3 , which adopts a space-time-cross attention structure, including a first linear projection layer, a coding-decoding encoder, and a linear regression layer in turn; where the coding-decoding encoder includes a D-layer encoder structure and a D-layer decoder structure stacked in turn, and D is an even number; the encoder structure includes a spatial Transformer block and a temporal Transformer block in turn; the decoder structure is specifically a cross-attention Transformer block;

[0060] First input to the first linear projection layer, project the data of the third dimension to high-dimensional features to obtain feature data d m is the feature dimension; at the same time, In the first cross-attention Transformer block, linearly projected into feature data After D layers of alternating spatial Transformer blocks and temporal Transformer blocks, and The points are processed together in the first cross-attention Transformer block. After processing by the D-layer cross-attention Transformer block and the linear regression layer, the 3D positions of N human joints in the camera coordinate system are output.

[0061] The input to the first-layer spatial Transformer block of the encoder structure is embedded with spatial position encoding, specifically a learnable parameter matrix.

[0062] Temporal position encoding is embedded inside the first-layer temporal Transformer block of the encoder structure. Specifically, this involves improving the rotational position encoding of the Attention weight matrix. A window of size ω is set within the original Attention weight position matrix, where the value of ω does not exceed the network training length. Absolute position encodings exceeding ω are uniformly set to ω, thus modifying the position matrix of the Attention weights to obtain the improved matrix.

[0063]

[0064] Step 4: Using the RGB videos corresponding to each video in the training set... As input, the coordinates of each human joint in each frame of each RGB video in the camera coordinate system are the estimation target, and minimizing the loss is the training objective. The 3D human pose estimation network is trained to obtain the trained 3D human pose estimation network.

[0065] The loss function L used during training is the joint loss function, and its formula is:

[0066] L = L MPJPE +λ t L time +λ m L Bone

[0067]

[0068]

[0069]

[0070] In the formula, L MPJPE L represents the average positional error of human joints. time For time consistency loss; L Bone For the bone vector loss; λ t and λ m All are loss weighting coefficients; Let be the 3D coordinates of the i-th human joint point corresponding to the t-th frame of the sequence; Let be the 3D coordinates of the i-th human joint point corresponding to the (t-1)-th frame of the sequence; is the real 3D coordinate of the i-th human body joint corresponding to the t-th frame of the sequence, and is a known quantity in each RGB video; is the estimated bone length of the i-th bone corresponding to the t-th frame of the sequence, which is specifically calculated by the Euclidean distance; is the real bone length of the i-th bone corresponding to the t-th frame of the sequence;

[0071] Step 5: After the to-be-estimated RGB video is processed in sequence by step 2, the 3D human pose estimation input data corresponding to the to-be-estimated RGB video is obtained is input into the trained 3D human pose estimation network, and the 3D positions of N human body joints in each frame of the to-be-estimated RGB video in the camera coordinate system are output.

[0072] According to the output 3D positions of N human body joints in each frame of the to-be-estimated RGB video in the camera coordinate system, the action parameters required for the race walking action are calculated, including the upper and lower arm angle, the upper arm and shoulder angle, the thigh angle, the two thigh angles, etc.

[0073] For example, the thigh angle θ is calculated, according to the 3D positions of the right hip joint, the right knee joint and the right ankle joint output by step 5, the spatial coordinates are A(x1, y1, z1), B(x2, y2, z2) and C(x3, y3, z3), respectively, to obtain vector AB=(x2-x1, y2-y1, z2-z1) and vector AC=(x3-x1, y3-y1, z3-z1), and then cosθ=(AB·AC) / (|AB|*|AC|) is calculated.

[0074] The above embodiments are for better further understanding of the present application and are not limited to the best mode of the present application, and do not limit the content and protection scope of the present application. Any product obtained by the inspiration of the present application or by combining the present application with other prior art features, which is the same as or similar to the present application, is within the protection scope of the present application.

Claims

1. A 3D human pose estimation method for extracting walking motions from RGB videos, characterized in that, Includes the following steps: Step 1: Construct a training set consisting of multiple RGB videos containing race walkers. Based on a 2D human pose estimation network, estimate the pixel coordinates of human joints in each frame of the RGB video in the pixel coordinate system. After distortion correction and intrinsic parameter normalization, obtain the N×2 2D human joint matrix corresponding to each frame; where N is the total number of human joints. Step 2: Construct 3D human pose estimation input data, specifically as follows: Step 2.1: Concatenate the 2D human joint point matrices of adjacent frames in the RGB video to obtain a T×N×2 2D pose sequence, denoted as... Where T is the sequence length; Step 2.2: Construct a human skeleton diagram with human joints as points and the bones between adjacent human joints as edges; and number the N human joints with the hip joint as number 0. Step 2.3: Using the shortest path algorithm, determine the bones that need to be traversed between the hip joint and any other joint in the human skeleton diagram, and obtain the maximum number of bones M that need to be traversed; obtain the set of bone lengths B corresponding to the bones traversed between the hip joint and each other joint in numerical order. 0,i ={B 0,m ,…,B n,i }, i = 0, 1, ..., N-1, m, n ∈ [0, 1, ..., N-1]; where i is the number of the human joint; m and n are the numbers of the human joints that the hip joint needs to pass through to reach the i-th human joint; B 0,m B is the bone length between the hip joint and the m-th human joint. n,i B is the bone length between the nth and ith human joints; 0,0 ={0}; Step 2.4, using bone length as the basis for assembly B 0,i ={B 0,m ,…,B n,i Let} be a row vector, and construct an N×M 2D bone length matrix corresponding to each frame in the RGB video; where each row is padded with 0s at the last blank position if it is less than M bone lengths; Step 2.5: Concatenate the 2D bone length matrices of adjacent frames in the RGB video to obtain a T×N×M dimensional bone length prior sequence, denoted as... Step 2.6: Process the 2D pose sequence Bone length prior sequence The data is stitched together to obtain the 3D human pose estimation input data. Step 3: Construct a 3D human pose estimation network, using the corresponding RGB videos in the training set. As input, the coordinates of each human joint in each frame of each RGB video in the camera coordinate system are the estimation target, and minimizing the loss is the training objective. The 3D human pose estimation network is trained to obtain the trained 3D human pose estimation network. Step 4: After processing the RGB video to be estimated in Step 2, the 3D human pose estimation input data corresponding to the RGB video to be estimated is obtained. Input the data into the trained 3D human pose estimation network, and output the 3D position of N human joints in the camera coordinate system for each frame of the RGB video to be estimated.

2. The 3D human pose estimation method for extracting walking motions from RGB video according to claim 1, characterized in that, Based on the 3D positions of N human joints in the camera coordinate system of each frame in the output RGB video to be estimated, the motion parameters required for the race walking motion are calculated.

3. The 3D human pose estimation method for extracting walking motions from RGB video according to claim 1, characterized in that, The 3D human pose estimation network adopts a space-time-cross-attention structure, including a first linear projection layer, an encoder-decoder, and a linear regression layer in sequence; wherein, the encoder-decoder includes a D-layer encoder structure and a D-layer decoder structure stacked in sequence; the encoder structure includes a spatial Transformer block and a temporal Transformer block in sequence; the decoder structure is specifically a cross-attention Transformer block; First, the data is input into the first linear projection layer, which projects the third-dimensional data onto the high-dimensional features to obtain the feature data. d m For feature dimension; at the same time, The feature data is linearly projected into the first-layer cross-attention Transformer block. After alternating spatial and temporal Transformer blocks in layer D, and The points are processed together in the first cross-attention Transformer block. After processing by the D-layer cross-attention Transformer block and the linear regression layer, the 3D positions of N human joints in the camera coordinate system are output.

4. The 3D human pose estimation method for extracting walking motions from RGB video according to claim 3, characterized in that, The input to the first-level spatial Transformer block of the encoder structure is embedded with spatial position encoding, specifically a learnable parameter matrix.

5. The 3D human pose estimation method for extracting walking motions from RGB video according to claim 3, characterized in that, Temporal position encoding is embedded inside the first-layer temporal Transformer block of the encoder structure. Specifically, this involves improving the rotational position encoding of the Attention weight matrix. A window of size ω is set within the original Attention weight position matrix, where the value of ω does not exceed the network training length. Absolute position encodings exceeding ω are uniformly set to ω, thus modifying the position matrix of the Attention weights to obtain the improved matrix.

6. The 3D human pose estimation method for extracting walking motions from RGB video according to claim 3, characterized in that, The loss function L used during training in step 3 is the joint loss function, and its formula is: L=L MPJPE +λ t L time +λ m L Bone In the formula, L MPJPE L represents the average positional error of human joints. time For time consistency loss; L Bone For the bone vector loss; λ t and λ m All are loss weighting coefficients; Let be the 3D coordinates of the i-th human joint point corresponding to the t-th frame of the sequence; Let be the 3D coordinates of the i-th human joint point corresponding to the (t-1)-th frame of the sequence; Let be the true 3D coordinates of the i-th human joint point corresponding to the t-th frame of the sequence, and be the known quantities in each RGB video; The estimated bone length is the length of the i-th bone corresponding to the t-th frame of the sequence. Let be the actual bone length of the i-th bone corresponding to the t-th frame of the sequence.

7. The 3D human pose estimation method for extracting walking motions from RGB video according to claim 6, characterized in that, It was obtained by calculation using Euclidean distance.

8. The 3D human pose estimation method for extracting walking motions from RGB video according to claim 1, characterized in that, N=17, and the joints of each human body are the hip joint, right hip joint, right knee joint, right ankle joint, left hip joint, left knee joint, left ankle joint, thoracic spine joint, neck joint, nose joint, head joint, left shoulder joint, left elbow joint, left hand joint, right shoulder joint, right elbow joint, and right hand joint. At this time, M is 5.

Images