A 3D human pose estimation method for monocular video
By combining prior knowledge of human joint motion with a temporally dilated convolutional model, the accuracy and consistency issues of monocular video 3D human pose estimation in outdoor scenes are solved, achieving high-quality 3D human pose estimation.
Patent Information
- Application Number
- CN202110648734.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-10
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-06-10
AI Technical Summary
Existing monocular video 3D human pose estimation methods have low accuracy in outdoor scenes, lack continuity of motion poses, and cannot effectively utilize the temporal dimension information of the video, resulting in poor estimation results.
We employ a smoothing algorithm based on prior knowledge of human joint motion and a temporal dilated convolution model. Combined with a cascaded pyramid structure two-dimensional pose detector and joint filter-based smoothing and noise reduction, we extract high-dimensional information through temporal dilated convolution and utilize the temporal correlation of video sequences to construct a three-dimensional pose estimation model.
It improves the accuracy and consistency of 3D human pose estimation, and can output natural and high-quality 3D human pose estimation results in complex scenes.
Smart Images

Figure CN113313731B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a posture estimation method, in particular to a three-dimensional human posture estimation method for monocular video, and belongs to the field of computer vision and image processing. Background Art
[0002] Human pose estimation has always been a major research direction in the field of computer vision. As a basic research, it is widely used in many fields such as human-computer interaction, behavior recognition, character tracking, animation production, etc. Early research was limited by factors such as hardware performance, theoretical basis and scarce data sets, and most of the research on human pose estimation was limited to the two-dimensional plane. With the widespread application of human pose estimation technology, the industry has put forward higher requirements for the accuracy and information richness of pose estimation. The existing two-dimensional pose estimation cannot detect the real moving human body in the actual scene well due to the lack of third-dimensional information, which makes three-dimensional human pose estimation one of the research hotspots. Among them, three-dimensional human pose estimation based on monocular images has better environmental universality than multi-eye and depth images, and has lower requirements for image acquisition equipment, so it has a larger engineering application market. For monocular video scenes, in dealing with diverse human motion backgrounds, the current 3D human pose estimation still has the following problems: (1) the video scene is limited to indoors; (2) the 3D pose accuracy is not high; (3) the motion pose lacks coherence. These problems result in the current 3D human pose method having poor estimation effect in various constrained scenes, and it is easy to have unnatural phenomena of discontinuous poses in video scenes, and it cannot make good use of the rich temporal dimension information in the video. Summary of the invention
[0003] In order to solve the above problems, the present invention provides a 3D human posture estimation method for monocular video, which can meet the demand for 3D human posture estimation in monocular video scenes, and at the same time, the method can obtain high-quality posture estimation results.
[0004] In order to achieve the above object, the technical solution of the present invention is as follows: a method for estimating 3D human posture in monocular video, which combines a smoothing algorithm based on prior knowledge of human joint motion, improves the 3D posture effect in the monocular video scene, and comprises the following steps:
[0005] Step 1: For the input monocular human motion video, a 2D posture detector with a cascaded pyramid structure is used to extract 2D joint coordinates and output as a 2D joint sequence. The present invention does not rely on any specific 2D key point detector, which enables the model to be used outdoors with rich scenes. In addition to the stacked hourglass detector, the present invention can also use Mask R-CNN with a ResNet-101-FPN backbone, using its reference implementation in Detectron, and a cascaded pyramid network (CPN) representing the FPN extension. For Mask R-CNN and CPN, start with a pre-trained model on COCO and fine-tune the detector on a 2D projection of Human3.6M. For Mask R-CNN, a ResNet-101 backbone network is used. When fine-tuning the Human3.6M model, the last layer of the key point network is reinitialized, as well as the unconvolution layer that regresses the heat map to learn a new set of key points. For CPN, we use a ResNet-50 backbone with a resolution of 384×288 and reinitialize the final layers (convolutional weights and batch normalization statistics) of Global-Net and Refine-Net.
[0006] Step 2: Use a two-dimensional joint filter based on human joint motion priors to smooth and reduce noise on the two-dimensional joint sequence. In the process of filtering and reducing noise, in order to determine the position of the noise point in the sequence, assume that j represents a joint among the 17 human joints to be predicted, L i Indicates the distance between the adjacent previous and next frames when the extreme point appears in the coordinate transformation diagram of the joint in the real data, K j L1, L2, L3, …, L i When an extreme point appears in the coordinate sequence of the joint point j predicted by the two-dimensional joint, if the distance between the extreme point and the coordinate points of the two frames before and after is greater than K j , then the extreme point is regarded as a noise point. i} represents a two-dimensional posture sequence with N frames, x i is the 2D joint coordinates of the i-th frame, the matrix D is generated by subtracting the 2D posture of the previous frame from the 2D posture of the i-th frame, and the matrix R is generated by subtracting the 2D posture of the next frame from the 2D posture of the i-th frame. According to the noise threshold constraint K, the noise point matrix N can be generated. In order to update the maximum noise point and the minimum noise point, N is used to generate the corresponding extreme value matrix N ex And negate it to get N ex ′, multiply the elements of the corresponding positions of the mean matrix and the extreme noise point matrix and add them to the original matrix to complete the update, and output the joint motion sequence after noise reduction and smoothing.
[0007] Step 3: Perform dimensionality-enhancing convolution on the input two-dimensional joint sequences of several frames, and enlarge the 34 joint data to 1024. The input video sequence is M frames, each frame contains 17×2 joint data, that is, the input data is M×34, and a conventional convolution with a convolution kernel of 3 is performed on it and the output dimension is expanded from 34 to 1024, and batch normalization, ReLU and Dropout are used for conventional processing.
[0008] Step 4: Construct a temporal dilation convolution model, using a multi-layer progressive structure, and achieve precise control of the temporal perception domain through the dilation factor. Notify each layer to use a one-dimensional convolution module to increase the depth, which is conducive to extracting high-dimensional information. The input layer obtains the series (x, y) coordinates of J joints in each frame, and performs a temporal convolution with a kernel size of W and an output channel of C. Then, a block of B temporal dilation convolutions is used, and residual connections are introduced between the B blocks. Each block first executes a kernel size of W and a dilation factor of D = W. B The first layer is a one-dimensional convolution of , followed by a convolution with a kernel size of 1. After the convolution (except the last layer), there is batch normalization, activation function and random dropout. The input layer obtains the concatenated (x, y) coordinates of J joints in each frame and performs a temporal convolution with a kernel size of W and an output channel of C. Then, a block of B temporal dilated convolutions is used, and residual connections are introduced between the B blocks. Each block first performs a kernel size of W and a dilation factor of D = W B A one-dimensional convolution is performed, followed by a convolution with a kernel size of 1. The convolution (except the last layer) is followed by batch normalization, ReLU, and Dropout.
[0009] Step 5: Use the slicing method to perform residual connections in each layer of convolutional structure, retaining the main features and accumulating them to the convolution output while avoiding overfitting. For each layer of dilated convolution units, assuming that the input data dimension is (3R, 1024), the output data dimension after a certain layer of convolution is (R, 1024). In order to accumulate the original data to the output to form a residual connection, the slicing function is used to establish the association, and the slicing area is set to a 1×3 matrix. The elements in the matrix are output at intervals of 3 to ensure that the output dimension matches while retaining the main features.
[0010] Step 6: After constructing the above-mentioned temporal expansion convolution model, in order to ensure that the model can converge normally during the training process, a corresponding loss function needs to be constructed. In order to make the model fully utilize the correlation between multiple frames of the video sequence, in addition to constructing the temporal expansion convolution model itself, a loss function that can learn the approximate linear difference characteristics of adjacent frames of the video sequence is introduced. Assume that T is the number of frames of the input continuous video frame, represents the 3D posture error between the i-th frame and the i-1-th frame predicted by the model, Represents the true error between adjacent frames. The difference between the two types of calculation results represents the degree of posture mismatch between adjacent frames. Then, the inter-frame posture continuity loss function can be constructed as follows:
[0011]
[0012] In addition to processing the last layer of features and calculating the error with the real data as part of the loss function, in order to give full play to the role of the network's lower-level and shallower features that are also valuable in 3D pose estimation, an intermediate supervision scheme is used to introduce the network's intermediate variables as part of the loss function. After each block unit (layer) of the B dilated convolution blocks, the output result of the slicing function used by the residual connection is reused, and the intermediate frame is extracted and directly output as a single frame result, and finally the 3D pose coordinates are output after dimensionality reduction. For the 3D human body pose output by the B temporal dilated convolution units, the error with the real data is calculated and accumulated into the output function of the model. Assume that B represents the number of temporal dilated convolution units, J represents the number of human joints to be predicted, and ω n Represents the weight of the b-th dilated convolutional layer, then the 3D pose estimation loss function with intermediate supervision can be constructed as follows:
[0013]
[0014] By combining the corresponding 3D pose basic error loss function, inter-frame pose continuity loss function and intermediate supervision loss function, the 3D pose estimation joint loss function used in this model can be constructed.
[0015] Loss = L Base +λ1L PCBF +λ2L inter
[0016] Step 7: Use the 3D joint coordinate sequence output by the temporal expansion convolution model to reconstruct the 3D posture result corresponding to the monocular human motion video. During the reconstruction process, the 3D joint coordinates of the pelvis output by the model are placed at the center of the 3D space, the 3D coordinates of the remaining 16 joints are mapped to the 3D space and the associated joints are connected one by one, and finally the visual 3D human posture estimation result corresponding to the input human motion video is output.
[0017] Compared with the prior art, the advantages of the present invention are as follows: 1) Based on the existing two-dimensional posture estimation results, the present invention utilizes the prior knowledge of human motion to eliminate the noise points in the two-dimensional joint sequence, and can obtain a smoother and more natural output sequence; 2) Aiming at the characteristics of monocular video scenes, the present invention proposes a three-dimensional posture estimation algorithm based on temporal dilation convolution, which utilizes the rich time dimension information in the video scene, so that the predicted three-dimensional posture is more consistent with the human motion posture of the input video; 3) The present invention introduces intermediate supervision and slice residual connection in the temporal dilation unit, so that the low-dimensional information in the convolution process is fully retained, thereby improving the accuracy of three-dimensional posture estimation. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 is the overall flow chart of the present invention, Figure 2 It is the intermediate supervision structure in step 6 of the present invention. DETAILED DESCRIPTION
[0019] The present invention is described in detail below in conjunction with the accompanying drawings, and the specific steps are as follows.
[0020] Example 1: Figure 1 As shown, the present invention mainly includes two processes, namely, two-dimensional posture estimation optimization based on joint filters and three-dimensional posture estimation algorithm based on time-series dilation convolution. Each part is described in detail below:
[0021] 1. 2D Posture Estimation Optimization Algorithm Based on Joint Filter
[0022] As shown in the figure, the video of the monocular scene is used as input. In the two-dimensional posture estimation optimization scheme of the present invention, the following process is adopted for calculation:
[0023] Step 1: For the input monocular human motion video, a 2D posture detector is used to extract the 2D joint coordinates and output them as a 2D joint sequence.
[0024] The present invention does not rely on any specific 2D keypoint detector, which enables the model to be used outdoors in rich scenes. In addition to the stacked hourglass detector, the present invention can also use Mask R-CNN with a ResNet-101-FPN backbone, using its reference implementation in Detectron, and a cascaded pyramid network (CPN) representing an extension of FPN. For Mask R-CNN and CPN, start with a pre-trained model on COCO and fine-tune the detector on a 2D projection of Human3.6M. For Mask R-CNN, a ResNet-101 backbone network is used. When fine-tuning the Human3.6M model, the last layer of the keypoint network is reinitialized, as well as the deconvolution layer that regresses the heat map to learn a new set of keypoints. For CPN, the present invention uses a ResNet-50 backbone network with a resolution of 384×288, and reinitializes the final layers of Global-Net and Refine-Net (convolution weights and batch normalization statistics). The present invention always maintains batch normalization when fine-tuning.
[0025] Step 2: Use a priori-based two-dimensional joint filter to smooth and reduce noise on the two-dimensional joint sequence. In the process of filtering and denoising, in order to determine the position of the noise point in the sequence, assume that j represents a joint among the 17 human joints to be predicted, L i Indicates the distance between the adjacent previous and next frames when the extreme point appears in the coordinate transformation diagram of the joint in the real data, K j L1, L2, L3, …, L i The maximum distance value can be used to construct the noise threshold constraint in the following formula:
[0026] K j =max((L1, L2, L3, ..., L i )
[0027] When an extreme point appears in the coordinate sequence of the joint point j predicted by the two-dimensional joint, if the distance between the extreme point and the coordinate points of the previous and next two frames is greater than K j , then the extreme point is regarded as a noise point.
[0028] Assume X = {x1, x2, x3, …, x i} represents a two-dimensional posture sequence with N frames, x i is the 2D joint coordinates of the i-th frame, the matrix D is generated by subtracting the 2D posture of the previous frame from the 2D posture of the i-th frame, and the matrix R is generated by subtracting the 2D posture of the next frame from the 2D posture of the i-th frame. According to the noise threshold constraint K, the noise point matrix N can be generated. In order to update the maximum noise point and the minimum noise point, N is used to generate the corresponding extreme value matrix N ex And negate it to get N ex′, multiply the elements of the corresponding positions of the mean matrix and the extreme noise point matrix and add them to the original matrix to complete the update, then a two-dimensional joint filter as shown below can be constructed to reduce the noise points in the output two-dimensional joint point sequence and increase the smoothness:
[0029]
[0030] Among them, X old is the original two-dimensional posture sequence, X new is the smoothed 2D pose sequence.
[0031] 2. 3D Pose Estimation Algorithm Based on Temporal Dilated Convolution
[0032] Step 3: Perform dimensionality-enhancing convolution on the input two-dimensional joint sequences of several frames to enlarge the dimension of 34 joint data to 1024;
[0033] The input video sequence is 243 frames, each frame contains 17×2 joint data, that is, the input data is 243×34. A regular convolution with a convolution kernel of 3 is performed on it and the output dimension is expanded from 34 to 1024. Batch regularization, ReLU and Dropout are used for regular processing, and the output data is 241×1024.
[0034] Step 4: Construct a time-series dilated convolution model, adopt a multi-layer progressive structure, achieve precise control of the time perception domain through the dilation factor, and inform each layer to use a one-dimensional convolution module to increase the depth, which is conducive to extracting high-dimensional information;
[0035] The input layer obtains the concatenated (x, y) coordinates of J joints in each frame and performs a temporal convolution with a kernel size of W = 3 and an output channel of C = 1024. Then, B = 4 temporal dilated convolution blocks are used, and residual connections are introduced between B blocks. Each block first performs a kernel size of W and a dilation factor of D = W. B The first layer of temporal expansion units has a dilation factor of 3, the second layer has a dilation factor of 9, the third layer has a dilation factor of 27, and the fourth layer has a dilation factor of 81. The convolution kernel of each layer is 3, which enhances the perception of the past and future time directions without introducing more parameters to double the amount of calculation. The output dimension of each convolution unit is 1024, and the dropout loss rate is set to 0.25. After four temporal expansion convolution units, a convolution is performed to obtain 1×51 output data, which represents the three-dimensional coordinates of 17 human joints.
[0036] Step 5: Use the slicing method to perform residual connections in each layer of the convolution structure, retaining the main features while avoiding overfitting and accumulating them into the convolution output;
[0037] For each layer of dilated convolution units, assuming that the input data dimension is (3R, 1024), the output data dimension after a certain layer of convolution is (R, 1024). In order to accumulate the original data into the output to form a residual connection, the slicing function is used to establish the association. The slicing area is set to a 1×3 matrix, and the elements in the matrix are extracted at intervals of 3 to ensure that it matches the output dimension while retaining the main features.
[0038] Step 6: After building the temporal dilation convolution model, in order to ensure that the model can converge normally during the training process, a corresponding loss function needs to be constructed. Assume that T is the number of input continuous video frames, J is the number of human joints, represents the three-dimensional joint point coordinates obtained by the model prediction for the i-th frame, Represents the true value of the 3D joint coordinates of the nth frame, then the 3D posture basic error loss function can be constructed as follows:
[0039]
[0040] In order to make the model fully utilize the correlation between multiple frames of video sequences, in addition to constructing the temporal dilation convolution model itself, a loss function that can learn the approximate linear difference characteristics of adjacent frames in the video sequence is introduced. Assume that T is the number of consecutive input video frames, represents the 3D posture error between the i-th frame and the i-1-th frame predicted by the model, Represents the true error between adjacent frames. The difference between the two types of calculation results represents the degree of posture mismatch between adjacent frames. Then, the inter-frame posture continuity loss function can be constructed as follows:
[0041]
[0042] In addition to processing the last layer of features and calculating the error with the real data as part of the loss function, in order to give full play to the role of the lower-level and shallower features of the network, which are also valuable in 3D pose estimation, an intermediate supervision scheme is used to introduce the intermediate variables of the network as part of the loss function. The output results of the slicing function used by the residual connection are reused, and the intermediate frame is extracted and directly output as a single frame result. Finally, the 3D pose coordinates are output after dimensionality reduction. For the 3D human body pose output by B dilated convolution units, the error with the real data is calculated and accumulated into the output function of the model. Assume that B represents the number of temporal dilated convolution modules, J represents the number of human joints to be predicted, and ω n Represents the weight of the b-th dilated convolutional layer, then the 3D pose estimation loss function with intermediate supervision can be constructed as follows:
[0043]
[0044] Combining the corresponding 3D posture basic error loss function, inter-frame posture continuity loss function and intermediate supervision loss function, the 3D posture estimation loss function used in this model can be constructed:
[0045] Loss = L base +λ1L time +λ2L inter
[0046] Among them, λ1 and λ2 are the weight ratios of inter-frame pose continuity loss and intermediate supervision loss, respectively.
[0047] For the optimizer, Amsgrad is used and trained for 80 epochs, using an exponentially decaying learning rate schedule starting from η=0.001 and using a shrinkage factor α=0.95 for each epoch. Since predicting a large number of adjacent frames during training will produce less desirable results than models that do not utilize temporal information (there are good random samples in the batch). The present invention selects training clips from different video clips to reduce the correlation of training samples. The size of the clip set is set to the width of the receptive field of the temporal dilation convolutional model so that the model predicts a single 3D pose for each training clip.
[0048] The present invention greatly optimizes the model in a single-frame scenario by replacing dilated convolution with strided convolution, where the step size is set to the dilation factor, which is only applied during training. During the prediction process, the temporal dilated convolution model can process the entire sequence and reuse the intermediate states of other three-dimensional frames for faster reasoning. In order to avoid frame loss affecting the effectiveness of the model convolution, the input boundaries of the sequence are padded by replication. In order to obtain more stable operating statistics, the present invention uses a schedule for the batch normalization momentum β: starting from β = 0.1 and then decaying at an exponential rate to reach β = 0.001 in the last epoch.
[0049] Step 7: Use the 3D joint coordinate sequence output by the temporal expansion convolution model to reconstruct the 3D posture result corresponding to the monocular human motion video. During the reconstruction process, the 3D joint coordinates of the pelvis output by the model are placed at the center of the 3D space, the 3D coordinates of the remaining 16 joints are mapped to the 3D space and the associated joints are connected one by one, and finally the visual 3D human posture estimation result corresponding to the input human motion video is output.
[0050] It should be noted that the above embodiments are merely preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Equivalent replacements or substitutions made on the basis of the above technical solutions all fall within the protection scope of the present invention.
Claims
1. A 3D human pose estimation method for monocular video, It is characterized in that The method comprises the following steps: Step 1: For the input monocular human motion video, a cascade pyramid-structured two-dimensional posture detector is used to extract the two-dimensional joint coordinates and output them as a two-dimensional joint sequence; Step 2: Construct a noise threshold constraint based on the extreme points of the two-dimensional joint sequence and the difference between the previous and next frames, and use the threshold filter to smooth and reduce the noise of the two-dimensional joint sequence; Step 3: Perform dimensional convolution on the input two-dimensional joint sequences of several frames; Step 4: Construct a time-series dilated convolution model, using a multi-layer progressive structure to achieve precise control of the temporal perception domain through dilation factors; Step 5: Use the slicing method to perform residual connections in each layer of the convolution structure, retaining the main features while avoiding overfitting and accumulating them into the convolution output; Step 6: Use the inter-frame posture continuity based on human motion rules and the intermediate supervision strategy to improve the underlying perception ability of the model to construct a joint loss function; Step 7: Use the 3D joint coordinate sequence output by the temporal dilation convolution model to reconstruct the 3D posture result corresponding to the monocular human motion video; The specific process of step 4 is as follows: the input layer obtains the serial (x, y) coordinates of J joints in each frame, and performs a temporal convolution with a kernel size of W and an output channel of C. Then, B temporal dilated convolution blocks are used, and residual connections are introduced between the B blocks. Each block first executes a kernel size of W and a dilation factor of D = W. B A one-dimensional convolution is performed, followed by a convolution with a kernel size of 1, except for the last layer of convolution, which is followed by batch normalization, ReLU, and Dropout; The specific process of step 6 is as follows: after constructing the time-series dilated convolution model, in order to ensure that the model can converge normally during the training process, a corresponding loss function needs to be constructed; 5.1 Assume that T is the number of continuous input video frames, S is the total number of samples, Represents the three-dimensional coordinates of a joint predicted by the model; Represents the true value of the three-dimensional coordinates of the corresponding joint; ||·|| 2 Indicates the calculation of the Euclidean distance between two coordinates, then the three-dimensional posture basic error loss function can be constructed as follows: 5.2 Inter-frame pose continuity loss: In order to make the model fully utilize the correlation between multiple frames in the video sequence, in addition to constructing the temporal dilation convolution model itself, a loss function that can learn the approximate linear difference features of adjacent frames in the video sequence is introduced. Assume that T is the number of consecutive input video frames, represents the 3D posture error between the i-th frame and the i-1-th frame predicted by the model, Represents the true error between adjacent frames. The difference between the two types of calculation results represents the degree of posture mismatch between adjacent frames. Then, the inter-frame posture continuity loss function can be constructed as follows: 5.3 Intermediate supervision loss: In addition to processing the last layer of features and calculating the error with the real data as part of the loss function, in order to give full play to the role of low-level and shallow features of the network that are also valuable in 3D posture estimation, an intermediate supervision scheme is used to introduce network intermediate variables as part of the loss function. After each block unit of the B dilated convolution blocks, the output result of the slicing function used by the residual connection is reused, and the intermediate frame is extracted and directly output as a single frame result. Finally, the 3D posture coordinates are output after dimensionality reduction. For the 3D human body posture output by the B temporal dilated convolution blocks, the error with the real data is calculated and accumulated into the output function of the model. Assume that B represents the number of temporal dilated convolution blocks, J represents the number of human joints to be predicted, and ω b Represents the weight of the b-th dilated convolutional layer, then the 3D pose estimation loss function with intermediate supervision can be constructed as follows: Combining the corresponding 3D pose basic error loss function, inter-frame pose continuity loss function and intermediate supervision loss function, the 3D pose estimation joint loss function used in this model can be constructed: Loss=L Base +λ 1 L PCBF +λ 2 L inter where λ 1 , 2 are the weight ratios of inter-frame pose continuity loss and intermediate supervision loss, respectively.
2. The method for estimating 3D human posture from monocular video according to claim 1, It is characterized in that Step 2: The specific process is as follows: 2.1 Noise threshold constraint: In the process of filtering and denoising, in order to determine the position of the noise point in the sequence, assume that j represents a joint among the 17 human joints to be predicted, L i Indicates the distance between the adjacent previous and next frames when the extreme point appears in the coordinate transformation diagram of the joint in the real data, K j For L 1 ,L 2 ,L 3 ,…,L i The maximum distance value is used to construct the noise threshold constraint in the following formula: K j =max(L 1 ,L 2 ,L 3 ,...,L i ) When an extreme point appears in the coordinate sequence of the joint point j predicted by the two-dimensional joint, if the distance between the extreme point and the coordinate points of the previous and next two frames is greater than K j , then the extreme point is regarded as a noise point; 2.2 Two-dimensional joint filtering: Assume X = {x 1 ,x 2 ,x 3 ,…,x i } represents a two-dimensional posture sequence with frame number i, x i is the 2D joint coordinates of the i-th frame, the matrix D is generated by subtracting the 2D posture of the previous frame from the 2D posture of the i-th frame, and the matrix R is generated by subtracting the 2D posture of the next frame from the 2D posture of the i-th frame. According to the noise threshold constraint K, the noise point matrix N can be generated. In order to update the maximum noise point and the minimum noise point, N is used to generate the corresponding extreme value matrix N ex And negate it to get N ex ′, multiply the elements of the corresponding positions of the mean matrix and the extreme noise point matrix and add them to the original matrix to complete the update, then a two-dimensional joint filter as shown below can be constructed to reduce the noise points in the output two-dimensional joint point sequence and increase the smoothness: Among them, X old is the original two-dimensional posture sequence, X new is the smoothed 2D pose sequence.
3. The method for estimating 3D human posture from monocular video according to claim 1, It is characterized in that The specific process of step 5 is as follows: for each layer of dilated convolution units, assuming that the input data dimension is (3R, 1024), the output data dimension after a certain layer of convolution is (R, 1024). In order to accumulate the original data into the output to form a residual connection, a slicing function is used to establish the association, and the area is set to a 1×3 matrix. The elements in the matrix are extracted at intervals of 3 to ensure that the output dimension matches while retaining the main features.
4. The method for estimating 3D human posture from monocular video according to claim 1, It is characterized in that The specific process of step 7 is as follows: during the reconstruction process, the three-dimensional joint coordinates of the pelvis output by the model are placed at the center position in the three-dimensional space, the three-dimensional coordinates of the remaining 16 joints are mapped to the three-dimensional space and the related joints are connected one by one, and finally a visualized three-dimensional human posture estimation result corresponding to the input human motion video is output.
