A 3D human pose estimation method

Abstract

The application discloses a 3D human posture estimation method, which comprises two components of a coordinate Transformer encoder and a time domain Transformer encoder, and respectively enhances the feature extraction capability of the network from the spatial and temporal angles. Firstly, the coordinate Transformer encoder is used to generate multiple feasible solutions from the input 2D human posture sequence, and a multi-layer perceptron is used to realize information exchange between the multiple feasible solutions; then, a more reliable feasible solution is generated through many-to-one mapping; then, the time domain Transformer encoder is used to capture the global dependence of the input sequence; finally, a regression header is used to output 3D human posture coordinates. The method has the advantages that the network prediction is more accurate, and the problem of limited time window of the traditional inflation time convolution method is avoided.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a 3D human pose estimation method. Background Technology

[0002] 3D (Three Dimensions) human pose estimation is an important research direction in computer vision and pattern recognition, requiring the prediction of 3D keypoint coordinates of a given human image. However, 3D human pose estimation faces numerous challenges. Obtaining 3D human pose estimation data is often limited, with most scenes being indoors and the action paradigms being limited, restricting the diversity of 3D human pose data. In recent years, with the development of deep learning, 3D human pose estimation has made significant progress.

[0003] With the development of deep learning-based 2D human pose estimation technology, advanced 3D human pose estimation often adopts a two-stage approach: first, obtaining a 2D human pose coordinate sequence from an image, and then upscaling the 2D human pose coordinate sequence to 3D to obtain a complete 3D human pose coordinate sequence. However, this approach still has several problems: First, the transformation from a 2D image to a 3D human pose is itself a problem with multiple feasible solutions, meaning that one 2D human pose corresponds to multiple different 2D projections of 3D human poses. Therefore, whether there is a relationship between multiple feasible solutions and how many feasible solutions one 2D human pose corresponds to will have a significant impact on the final result, leading to inaccurate 3D human pose coordinate sequences. Second, current 3D human pose estimation methods based on convolutional neural networks mainly utilize dilated temporal convolutional neural networks. The receptive field of these methods is affected by factors such as the number of network layers, the size of the convolutional kernel, and the dilation coefficient, and generally suffers from a limited time window. Summary of the Invention

[0004] This invention discloses a 3D human pose estimation method to solve the problem of limited network prediction ability caused by using only a single feasible solution, and the problem of limited time window caused by using dilated temporal convolutional neural networks.

[0005] To solve the above-mentioned technical problems, the present invention discloses the following technical solution:

[0006] This invention provides a 3D human pose estimation method, specifically including the following steps: dataset establishment, coordinate transformation, feasible solution generation, repeated execution, channel adjustment, optimal feasible solution acquisition, dependency capture, and coordinate prediction. The dataset establishment step refers to inputting multiple sets of 2D human pose coordinate sequences and the corresponding 3D human pose coordinate sequences for each set of 2D human pose coordinate sequences. The coordinate transformation step connects the horizontal and vertical coordinates of the input 2D human pose coordinate sequence to convert it into a 2D human pose coordinate sequence readable by the Transformer encoder, outputting the first transformed data. The feasible solution generation step inputs the first transformed data into the coordinate Transformer encoder to generate a feasible solution. The repeated execution step refers to repeating the feasible solution generation step at least three times to obtain at least three feasible solutions. The channel adjustment step performs a one-dimensional convolution on each feasible solution to adjust the channels of each feasible solution, thereby obtaining the high-dimensional features of each feasible solution. The optimal feasible solution acquisition step concatenates the multiple feasible solutions along the channel dimension, uses a multilayer perceptron to realize information exchange between the multiple feasible solutions, and obtains an optimal feasible solution through a many-to-one mapping. The high-dimensional feature encoding generation step inputs the optimal feasible solution into the temporal Transformer encoder to obtain a high-dimensional feature encoding containing the global dependencies of the input sequence. The coordinate prediction step uses a one-dimensional convolution as the regression head to adjust the number of feature channels to predict the 3D human pose coordinates and outputs the predicted 3D human pose coordinate sequence.

[0007] Further, the feasible solution generation step specifically includes the following steps: a coordinate embedding step, a first data connection step, a second data connection step, and a feasible solution acquisition step. The coordinate embedding step embeds the first transformed data into a learnable coordinate position and executes the Dropout module. The coordinate position embedding formula is as follows:

[0008]

[0009] in, The input data contains the human pose coordinates for a specific frame. Embed for coordinate position, The data is embedded with coordinate positions; the first data connection step is to... As input, the LayerNorm operation is performed and then fed into a multi-head self-attention mechanism module. The Dropout module is then executed, outputting a first output feature. This first output feature is then connected to the first transformed data via a skip connection. The self-attention calculation formula is as follows:

[0010]

[0011] in, For query, As key, For value, As a dimension, MSA will , , Divided into share, For the number of MSA headers, perform the above steps respectively. The second data connection step involves taking the first output feature as input, performing the LayerNorm operation, and then inputting it into a multilayer perceptron module. The multilayer perceptron module includes multiple linear layers. After each linear layer, the Dropout module is executed to output a second output feature. The first output feature and the second output feature are then connected via skip connections. The calculation formula in the multilayer perceptron module is as follows:

[0012]

[0013] in, For GELU functions, , These are the weights for the two linear layers, , These are the bias terms of two linear layers; the feasible solution acquisition step involves connecting the first transformed data and the second output feature through a skip connection as a sample, and performing the LayerNorm operation on different features of the sample to obtain a feasible solution.

[0014] Further, the optimal feasible solution acquisition step specifically includes the following steps: a feasible solution concatenation step, an information exchange step, and a channel number reduction step. The feasible solution concatenation step involves concatenating multiple feasible solutions along the channel dimension to form a new feasible solution, such that the number of feature channels in the new feasible solution is multiple times that of each feasible solution. The information exchange step involves performing a LayerNorm operation on the new feasible solution and then inputting it into a multilayer perceptron. The multilayer perceptron module includes multiple linear layers and executes two Dropout modules to perform information exchange between each feasible solution, outputting first output data. A skip connection is then used to connect the new feasible solution with the first output data. The calculation formula in the multilayer perceptron module is as follows:

[0015]

[0016] in, For GELU functions, , These are the weights for the two linear layers, , These are the bias terms of two linear layers; the channel reduction step involves performing a BatchNorm operation on the first output data, then using the one-dimensional convolution to complete a many-to-one mapping of the output data, and outputting the optimal feasible solution.

[0017] Furthermore, the dependency capture step specifically includes the following steps: a data transformation step, a time location embedding step, a third data connection step, a fourth data connection step, and a feature encoding acquisition step. The data transformation step transforms the data of the optimal feasible solution to generate second transformed data; the time location embedding step embeds the second transformed data into a learnable time location and executes the Dropout module. The time location embedding formula is as follows:

[0018]

[0019] in, The input data consists of temporal data of human pose from a single frame. Embedded for time location, The data is embedded with the time location; the third data connection step is to... As input, the LayerNorm operation is performed and then fed into the multi-head self-attention mechanism module. The Dropout module is then executed, outputting a third output feature. This third output feature is then connected to the optimal feasible solution via a skip connection. The self-attention calculation formula is as follows:

[0020]

[0021] in, For query, As key, For value, As a dimension, MSA will , , Divided into share, The number of heads in MSA (Multi-Head Self-Attention) is used. MSA is a multi-head self-attention mechanism. The above steps are performed separately. Operations;

[0022] The fourth data connection step involves taking the third output feature as input, performing a LayerNorm operation, and then inputting it into the multilayer perceptron module. The multilayer perceptron module includes multiple linear layers. After each linear layer, the Dropout module is executed to output a fourth output feature. This fourth output feature is then connected to the third output feature via a skip connection. The calculation formula in the multilayer perceptron module is as follows:

[0023]

[0024] in, For GELU functions, , These are the weights for the two linear layers, , These are the bias terms for the two linear layers, The feature encoding acquisition step involves connecting the fourth output feature with the optimal feasible solution via skip connections, and then performing the LayerNorm operation to obtain a high-dimensional feature encoding that includes the global dependencies of the input sequence.

[0025] Furthermore, following the coordinate prediction step, a training step is also included, which is performed on the Human3.6M dataset. This dataset comprises multiple datasets, with some used as the training set and others as the test set. Training lasts for 15 epochs, with Adam selected as the optimizer. The initial learning rate is set to 0.001, and a shrinkage factor of 0.95 is applied to the learning rate after each epoch. The average position error per joint is used as the loss function, as shown in the following formula:

[0026]

[0027] in, Indicates the number of joints; Indicates the first Predicted 3D human pose coordinate sequence for each joint. Indicates the first A sequence of real 3D human pose coordinates for each joint. The position error is represented by the loss function, which has a minimum value. The predicted value of the 3D human pose coordinate sequence corresponding to the minimum value of the loss function is output.

[0028] Furthermore, after the training step, a testing step is also included, in which the trained 3D human pose estimation method and the three methods VideoPose, PoseFormer and MHFormer are used to test the test set of the dataset, and the test results are compared.

[0029] The present invention also provides a storage medium for storing executable program code; a processor reads the executable program code to run a computer program corresponding to the executable program code to perform at least one step in the 3D human pose estimation method.

[0030] The advantages of this invention are that it provides a 3D human pose estimation method that exchanges information from multiple feasible solutions through a many-to-one mapping to generate an optimal feasible solution, making network prediction more accurate. It also utilizes a temporal Transformer encoder to capture the global dependencies of the input sequence, while avoiding the time window limitation problem of traditional dilated temporal convolution methods. Attached Figure Description

[0031] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] Figure 1 Here is a flowchart of the 3D human pose estimation method provided by the present invention;

[0033] Figure 2 A flowchart of the feasible solution generation steps provided by the present invention;

[0034] Figure 3 This is a schematic diagram of the coordinate Transformer encoder of the present invention;

[0035] Figure 4 The flowchart of the optimal feasible solution acquisition steps provided by the present invention;

[0036] Figure 5 A flowchart of the dependency capture steps provided by this invention;

[0037] Figure 6 This is a schematic diagram of the time-domain Transformer encoder of the present invention;

[0038] Figure 7 The image shows the results of testing a jumping scenario with partial body occlusion, as provided by this invention.

[0039] Figure 8 The image provided by this invention shows the results of testing a running scenario with partial body occlusion.

[0040] Figure 9 A visualization comparison of the method provided by this invention with other advanced algorithms on the same dataset.

[0041] Figure 10 A comparison chart of test results of the method provided by this invention with other advanced algorithms on the same dataset;

[0042] Figure 11 This is a schematic diagram of the structure of the storage medium of the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] like Figure 1 As shown, the present invention provides a 3D human pose estimation method, which specifically includes the following steps: step S1) dataset establishment step, step S2) coordinate transformation step, step S3) feasible solution generation step, step S4) repeated execution step, step S5) channel adjustment step, step S6) optimal feasible solution acquisition step, step S7) dependency capture step, step S8) coordinate prediction step, step S9) training step, and step S10) testing step.

[0045] Step S1) Dataset creation step: Input multiple sets of 2D human posture coordinate sequences and the corresponding real 3D human posture coordinate sequences for each set of 2D human posture coordinate sequences.

[0046] Step S2) Coordinate transformation step: Connect the horizontal and vertical coordinates of the input 2D human pose coordinate sequence to convert it into a 2D human pose coordinate sequence readable by the Transformer encoder, and output the first transformation data, for example, convert the original format (batchsize, frames, in_joints, 2) into (batchsize, in_joints*2, frames).

[0047] like Figure 2 and Figure 3 As shown, step S3) feasible solution generation step involves inputting the first transformed data into a coordinate Transformer encoder to generate multiple feasible solutions, specifically including the following steps: step S31) coordinate embedding step, step S32) first data connection step, step S33) second data connection step, and step S34) feasible solution acquisition step.

[0048] Step S31) Coordinate embedding step: The first transformed data is embedded into a learnable coordinate position to preserve the spatial position of each joint, and the Dropout module is executed to suppress overfitting. The coordinate position embedding formula is as follows:

[0049]

[0050] in, The input data contains the human pose coordinates for a specific frame. Embed for coordinate position, The data is embedded in the coordinate position.

[0051] Step S32) First data connection step, connecting the... As input, the LayerNorm operation is performed and then fed into a multi-head self-attention mechanism module. This module has 9 heads and an input data dimension of 27. The Dropout module is then executed to suppress overfitting, outputting a first output feature. This first output feature is then connected to the first transformed data via skip connections. The self-attention calculation formula is as follows:

[0052]

[0053] in, For query, As key, For value, For dimensions. MSA will , , Divided into share, For the number of MSA headers, perform the above steps respectively. Calculation.

[0054] Step S33) Second data connection step: Taking the first output feature as input, after performing the LayerNorm operation, it is input into a multilayer perceptron module. The multilayer perceptron module includes two linear layers with 54 hidden units each, a GELU function, and executes the Dropout module after each linear layer to output a second output feature. The first output feature and the second output feature are then connected via skip connections. The calculation formula in the multilayer perceptron module is as follows:

[0055]

[0056] in, For GELU functions, , These are the weights for the two linear layers, , These are the bias terms for the two linear layers, respectively.

[0057] S34) Feasible solution acquisition step: Connect the first transformed data with the second output feature through a skip connection as a sample, and perform the LayerNorm operation on different features of the sample to obtain a feasible solution.

[0058] Step S4) Repeat the steps to obtain feasible solutions three times to obtain three feasible solutions.

[0059] Step S5) Channel adjustment step: Perform a one-dimensional convolution on each feasible solution to adjust the channels of each feasible solution to 512, obtain the high-dimensional features of each feasible solution, and then use the BatchNorm operation to accelerate network training and convergence, use the ReLU function to increase the non-linear relationship between the layers of the network, and use the Dropout module to suppress overfitting.

[0060] like Figure 4 As shown, step S6) is the optimal feasible solution acquisition step, which involves concatenating the multiple feasible solutions along the channel dimension, using a multilayer perceptron to realize information exchange between the multiple feasible solutions, and obtaining an optimal feasible solution through a many-to-one mapping. Specifically, it includes the following steps: step S61) feasible solution concatenation step, step S62) information exchange step, and step S63) channel number reduction step.

[0061] Step S61) Feasible solution splicing step: splice the multiple feasible solutions along the channel dimension to form a new feasible solution, so that the number of feature channels of the new feasible solution is 3 times that of each feasible solution.

[0062] Step S62) Information exchange step: After performing a LayerNorm operation on the new feasible solution, it is input into a multilayer perceptron. The multilayer perceptron module includes two linear layers with 54 hidden units each, a GELU function, and two Dropout modules. Information exchange is performed between the three feasible solutions, outputting first output data. The new feasible solution is then connected to the first output data via a jump connection. The calculation formula in the multilayer perceptron module is as follows:

[0063]

[0064] in, For GELU functions, , These are the weights for the two linear layers, , These are the bias terms for the two linear layers, respectively.

[0065] S63) Channel reduction step: After performing the BatchNorm operation on the first output data, the one-dimensional convolution is used to complete the many-to-one mapping of the output data, and the number of feature channels of the first output data is adjusted to 1 / 3 of the current number of feature channels, and the optimal feasible solution is output.

[0066] like Figure 5 and Figure 6As shown, step S7) dependency capture step, the optimal feasible solution is input into the temporal Transformer encoder to obtain a high-dimensional feature encoding containing the global dependency of the input sequence, specifically including the following steps: step S71) data transformation step, step S72) temporal position embedding step, step S73) third data connection step, step S74) fourth data connection step and step S75) feature encoding acquisition step.

[0067] Step S71) Data conversion step: Convert the data format of the optimal feasible solution to generate second converted data, for example, convert the data format from (batchsize, dim, frames) to (batchsize, frames, dim).

[0068] Step S72) Temporal Position Embedding Step: The second transformed data is embedded with a learnable temporal position to preserve the temporal position of each frame, and the Dropout module is executed. The temporal position embedding formula is as follows:

[0069]

[0070] in, The input data consists of temporal data of human pose from a single frame. Embedded for time location, This is the data embedded with the time location.

[0071] Step S73) Third data connection step, to connect the... As input, the LayerNorm operation is performed and then fed into the multi-head self-attention mechanism module, which has 8 heads and an input data dimension of 512. The Dropout module is then executed, outputting a third output feature, which is then connected to the optimal feasible solution via skip connections. The self-attention calculation formula is as follows:

[0072]

[0073] in, For query, As key, For value, For dimensions. MSA will , , Divided into share, The number of heads in MSA (Multi-Head Self-Attention) is used. MSA is a multi-head self-attention mechanism. The above steps are performed separately. Operations;

[0074] Step S74) Fourth data connection step: The third output feature is used as input, and after performing a LayerNorm operation, it is input into the multilayer perceptron module, which includes two linear layers with 1024 hidden units each, a GELU function, and a Dropout module is executed after each linear layer to output a fourth output feature. The fourth output feature is then connected to the third output feature through a skip connection. The calculation formula in the multilayer perceptron module is as follows:

[0075]

[0076] in, For GELU functions, , These are the weights for the two linear layers, , These are the bias terms for the two linear layers, It is a multilayer perceptron.

[0077] S75) Feature encoding acquisition step: the fourth output feature is connected to the optimal feasible solution through a skip connection, and then the LayerNorm operation is performed to obtain a high-dimensional feature encoding containing the global dependencies of the input sequence.

[0078] Step S8) Coordinate prediction step: Use a one-dimensional convolution as the regression head to adjust the number of feature channels to predict 3D human pose coordinates and output the predicted 3D human pose coordinate sequence.

[0079] Step S9) Training step: Steps S2), S3), S4), S5), S6), S7), and S8) are trained on the Human 3.6M public dataset. This dataset includes seven 3D-labeled datasets: D1, D2, D3, D4, D5, D6, and D7. D1, D2, D3, D4, and D5 are used as the training set, and D6 and D7 are used as the test set. Training is performed for 15 epochs. Adam is selected as the optimizer, with an initial learning rate of 0.001. A shrinkage factor of 0.95 is applied to the learning rate after each epoch. The average positional error per joint is used as the loss function, as shown in the following formula:

[0080]

[0081] in, Indicates the number of joints; Indicates the first Predicted 3D human pose coordinate sequence for each joint. Indicates the first A sequence of real 3D human pose coordinates for each joint. The position error is represented by the loss function, which has a minimum value. The predicted value of the 3D human pose coordinate sequence corresponding to the minimum value of the loss function is output.

[0082] like Figure 9 and Figure 10 As shown, in step S10), the testing step uses other advanced algorithms to train on the Human 3.6M dataset. During training, the training set, test set partitioning, and optimizer are all consistent with the method of this invention. The length of the input sequence is 243 frames. Two representative scenarios, a simple waving action and a complex jumping action, were selected, such as... Figure 9 As shown, the advantages and disadvantages of this invention are compared with advanced algorithms such as VideoPose, PoseFormer, and MHFormer through visualization. While the VideoPose algorithm detects people, the movements are significantly distorted. Although PoseFormer and MHFormer can display general human movements, they are inferior to this invention in details such as wrists and knees. This invention can not only detect people in complex scenes but also accurately reproduce the bending and rotation of details such as wrists, elbows, and knees. Figure 10 As shown, the performance of various methods on the loss function also demonstrates that the method provided by this invention predicts a smaller position error.

[0083] like Figure 7 and Figure 8 As shown, in order to further examine the performance of the method of the present invention under the condition of half-body occlusion, in addition to the original actions, test scenarios including but not limited to jumping and running were added to the test steps.

[0084] like Figure 11 As shown, this embodiment also provides a storage medium 11, a memory 111 for storing executable program code; a processor 112 reads the executable program code to run a computer program corresponding to the executable program code to perform at least one step in the 3D human pose estimation method.

[0085] The advantages of this invention are that it provides a 3D human pose estimation method that exchanges information from multiple feasible solutions through a many-to-one mapping to generate an optimal feasible solution, making network prediction more accurate. It also utilizes a temporal Transformer encoder to capture the global dependencies of the input sequence, while avoiding the time window limitation problem of traditional dilated temporal convolution methods.

[0086] The above provides a detailed description of a 3D human pose estimation method provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the technical solutions and core ideas of the present invention. Those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D human pose estimation method, characterized in that, Specifically comprising the following steps: A data set establishment step of inputting a plurality of sets of 2D human posture coordinate sequences and a real 3D human posture coordinate sequence corresponding to each set of 2D human posture coordinate sequences; A coordinate conversion step of connecting the horizontal and vertical coordinates of the 2D human posture coordinate sequence to convert it into a 2D human posture coordinate sequence readable by a Transformer encoder and output first conversion data; A feasible solution generation step of inputting the first conversion data into a coordinate Transformer encoder to generate a feasible solution; The feasible solution generation step specifically comprises the following steps: A coordinate embedding step of embedding the first conversion data into a learnable coordinate position and executing a Dropout module, with the coordinate position embedding formula being wherein, is human pose coordinate data for a frame in the input data, is a coordinate position embedding, is data after the coordinate position embedding. a first data connection step of connecting the first transformed data to the first output feature from a previous iteration of the first data connection step, As input, the input is input to a multi-head self-attention mechanism module after performing a LayerNorm operation, then the Dropout module is executed, the first output feature is output, and the first output feature is connected to the first transformed data through a skip connection, and the self-attention calculation formula is wherein, is a query, is a key, is a value, is a dimension, the MSA will , , is divided into parts, is the number of heads of the MSA, the MSA is a multi-head self-attention mechanism, and the above operations are performed respectively; A second data connection step of inputting the first output feature as input, executing the LayerNorm operation, and then inputting it into a multi-layer perceptron module, the multi-layer perceptron module comprising a plurality of linear layers, each of which is followed by the Dropout module, and outputting a second output feature, and connecting the first output feature and the second output feature through a skip connection, with the calculation formula in the multi-layer perceptron module being wherein, is the GELU function, , are the weights of the two linear layers, respectively, , are the bias terms of the two linear layers, respectively. A feasible solution acquisition step of connecting the first conversion data and the second output feature through a skip connection to form a sample, performing the LayerNorm operation on different features of the sample, and obtaining a feasible solution; A repeated execution step of repeating the feasible solution generation step at least three times to obtain at least three feasible solutions; A channel adjustment step of executing a one-dimensional convolution on each feasible solution to adjust the channel of each feasible solution, thereby obtaining a high-dimensional feature of each feasible solution; An optimal feasible solution acquisition step of splicing a plurality of feasible solutions along the channel dimension, exchanging information between the plurality of feasible solutions by using a multi-layer perceptron, and obtaining an optimal feasible solution through a one-to-many mapping; The optimal feasible solution acquisition step specifically comprises the following steps: A feasible solution splicing step of splicing the plurality of feasible solutions along the channel dimension to form a new feasible solution, so that the feature channel number of the new feasible solution is a multiple of that of each feasible solution; An information exchange step of inputting the new feasible solution into a multi-layer perceptron after performing the LayerNorm operation, the multi-layer perceptron module comprising a plurality of linear layers and two Dropout modules, exchanging information between each feasible solution, outputting first output data, and connecting the new feasible solution and the first output data through a skip connection, with the calculation formula in the multi-layer perceptron module being wherein, is a GELU function, , are weights of two linear layers, respectively, , are bias terms of two linear layers, respectively, is a multi-layer perceptron; A channel number reduction step of executing the BatchNorm operation on the first output data and using the one-dimensional convolution to complete a one-to-many mapping of the first output data to output an optimal feasible solution; A dependency relationship capturing step of inputting the optimal feasible solution into a time domain Transformer encoder to obtain a high-dimensional feature code containing global dependency relationships of the input sequence; The dependency relationship capturing step specifically comprises the following steps: Data conversion step, converting data of the optimal feasible solution to generate second converted data; Time position embedding step, embedding the second converted data into a learnable time position and executing a Dropout module, the time position embedding formula is as follows: wherein, is human pose time series data for a frame in the input data, is a time position embedding, is the data after the time position embedding; a third data connection step of connecting the As input, the input is input to the multi-head self-attention mechanism module after performing the LayerNorm operation, then the Dropout module is performed, the third output feature is output, and the third output feature is connected with the optimal feasible solution through the jump connection, and the self-attention calculation formula is: wherein, is a query, is a key, is a value, is a dimension, the MSA will , , be divided into parts, is the number of heads of the MSA, the MSA is a multi-head self-attention mechanism, and the above operations are performed respectively; Fourth data connection step, taking the third output feature as input, executing a LayerNorm operation and inputting into the multi-layer perception module, the multi-layer perception module includes multiple linear layers, the Dropout module is executed after each linear layer, the fourth output feature is output, and the fourth output feature is connected with the third output feature through a skip connection, and the calculation formula in the multi-layer perception module is as follows: wherein, is a GELU function, , are weight values of the two linear layers, respectively, , are bias terms of the two linear layers, respectively, is a multi-layer perceptron; Feature encoding acquisition step, executing the LayerNorm operation on the data connected with the optimal feasible solution through the skip connection to obtain high-dimensional feature encoding containing global dependency of input sequence; Coordinate prediction step, taking a one-dimensional convolution as a regression header to adjust the number of feature channels and output a predicted 3D human pose coordinate sequence.

2. The 3D human pose estimation method of claim 1, wherein, After the coordinate prediction step, the training step is further included, training is performed on the Human3.6M dataset, part of the dataset is taken as a training set and part of the dataset is taken as a test set, a total of 15 epochs are trained, Adam is selected as an optimizer, the initial learning rate is set to 0.001, and a shrinkage factor 0.95 is applied to the learning rate after each epoch, and the average position error of each joint is used as a loss function, the formula is as follows: wherein, represents the number of joints; represents the predicted 3D human pose coordinate sequence of the jthjoint, represents the real 3D human pose coordinate sequence of the jthjoint, is the positional error; The loss function has a minimum value, and the predicted value of the 3D human pose coordinate sequence corresponding to the minimum value of the loss function is output.

3. The 3D human pose estimation method of claim 2, wherein, After the training step, the test step is further included, the test set of the dataset is tested by using the above trained 3D human pose estimation method and VideoPose, PoseFormer and MHFormer three methods, and the test results are compared.

4. A storage medium for storing executable program code; characterized in that, A processor reads the executable program code to run a computer program corresponding to the executable program code to execute at least one step of the 3D human pose estimation method of any one of claims 1-3.

Images