A 3D Human Pose Estimation Method for Virtual Fashion Shows

Abstract

This invention relates to a 3D human pose estimation method for virtual fashion shows. It employs an improved Kalman filter to preprocess the input image; designs a high-resolution 2D human pose estimation network based on HRNet-W32, trains this network using a standard 2D dataset, and uses it to estimate the 2D human pose corresponding to RGB images in a 3D dataset; constructs a graph convolutional 3D regression network based on residual connections and attention mechanisms, while incorporating semantic information. The human pose estimated by the 2D human pose estimation network is used as input, and the corresponding actual 3D human pose values ​​are used as labels to train the 3D regression network. The two trained networks are concatenated to obtain the final 3D human pose estimation model. Compared with other 3D human pose estimation methods, this invention achieves more accurate 3D human pose estimation on a standard 3D dataset.

Description

Technical Field

[0001] This invention relates to a three-dimensional human pose estimation technique, and more particularly to a three-dimensional human pose estimation method for virtual fashion shows. Background Technology

[0002] Fashion shows are increasingly moving towards virtualization. Traditional virtual fashion shows mainly fall into two categories: animation simulation and special effects production. Animation simulation requires professional artists to design the runway movements, and the animation production cycle is long and the effects are limited. Special effects production requires the participation of real models, making it difficult to achieve a truly virtual fashion show. At the same time, this method requires a significant investment of manpower and resources, as well as substantial financial expenditure.

[0003] As a crucial component of computer vision, 3D human pose estimation has been widely applied in various fields, such as augmented reality and human-computer interaction, thanks to the development of deep learning. If 3D human pose estimation technology could be applied to virtual fashion shows, using models walking the runway to drive virtual character models, it would save significant resources while simultaneously diversifying and virtualizing the movements of the fashion show.

[0004] Current image-based 3D human pose estimation methods are mainly divided into single-stage and two-stage approaches. The single-stage approach directly uses RGB images as input, extracting key joint information to regress the 3D human pose. However, this method lacks supervised training and is easily affected by background and lighting factors, resulting in poor accuracy and generalization ability. The two-stage approach first uses a 2D human pose estimation network to perform 2D human pose estimation on the input RGB image, obtaining the 2D coordinates of the key joints. Then, using a 2D skeleton as input, it performs 3D pose regression through a constructed neural network. Because the two-stage approach uses supervised training for the 2D human pose estimation network, the network's overall generalization ability is better. However, the accuracy of 3D regression often depends on the accuracy of the 2D human pose estimation. Considering the complexity of fashion show processes, existing two-stage 3D human pose estimation methods cannot effectively realize virtual fashion shows based on 3D human pose estimation. Summary of the Invention

[0005] To address the lack of accuracy in 3D human pose estimation in virtual fashion show scenarios, a new 3D human pose estimation method for virtual fashion show scenarios is proposed, which achieves accurate 3D human pose estimation and meets the needs of virtual fashion show scenarios.

[0006] The technical solution of this invention is: a three-dimensional human pose estimation method for virtual fashion shows, specifically including the following steps:

[0007] 1) Image data acquisition: Process the model catwalk video in a single scene into frame-by-frame images to acquire the image data of the input video;

[0008] 2) Image preprocessing: Improved Kalman filtering is used to optimize human motion state and reduce the prediction deviation of human joint points caused by clothing occlusion and self-occlusion in video images, thus achieving image preprocessing;

[0009] 3) Construct and train a two-dimensional human pose estimation network model, and feed the preprocessed image into the trained two-dimensional human pose estimation network model to obtain two-dimensional pose estimation.

[0010] The structure of the two-dimensional human pose estimation network model includes the first three layers of the high-resolution two-dimensional human pose estimation network of HRNet-W32. The convolutional kernels of the bottleneck module in the first layer and the basic block module in the second and third layers are replaced by the pyramid segmentation attention module to compensate for the accuracy loss caused by removing the fourth layer of the HRNet-W32 network.

[0011] The training of the two-dimensional human pose estimation network model: For this network model, the public dataset COCO2017 is used for training. Before training, the images in the COCO2017 dataset need to be preprocessed, the image size is fixed at 256×192, the learning rate is set to 0.001, the training period is set to 210, and the average accuracy index is used as the evaluation standard for two-dimensional pose estimation.

[0012] 4) Construct and train a 3D human pose regression network model. Input the 2D pose estimation into the trained 3D human pose regression network model to perform 3D pose regression.

[0013] The 3D human pose regression network model is a graph convolutional network based on residual connections and attention mechanisms. It uses graph convolution combined with semantic information to learn the channel weights of the implicit prior edges in the 2D human pose, and combines them with the kernel matrix to improve the power of graph convolution. The human skeleton is regarded as a graph structure data, and residual connections are used to eliminate the oversmoothing problem during the graph convolution stacking process. The attention mechanism is used to obtain local and global contextual information between different key points through global context, and at the same time solve the occlusion and depth blur problems in 3D human pose estimation.

[0014] The training of the 3D human pose regression network model involved using S1, S5, S6, S7, and S8 from the Human 3.6M dataset as the training set and S9 and S11 as the validation set to verify the effect. The Euclidean distance between the 3D joint coordinates predicted by the network and the actual labeled human joint coordinates was used as the evaluation criterion for the final 3D human pose estimation result to verify the effectiveness of the 3D human pose estimation.

[0015] Furthermore, the specific method for optimizing human motion state using improved Kalman filtering in step 2) is as follows: the motion of the human body on each axis in three-dimensional space is a Bezier curve, and the motion on each axis is approximately uniformly accelerated or decelerated. The prediction of the current position is obtained by combining the state of the previous three positions with the change in acceleration.

[0016] Kalman filter optimization: First, obtain the first three states x. k-1 x k-2 x k-3 The value of is used to predict the current state based on the previous three states: in To predict location, For the observation location, K k This is the Kalman gain.

[0017] Furthermore, in step 3), the pyramid segmentation attention module consists of an SPC module and an SE Weight module. The SPC module segments the attention channels and performs multi-scale feature extraction on the spatial information of each channel's feature image. The SE Weight module is used to extract the channel attention of feature images at different scales, thereby obtaining its channel attention vector at each scale. Then, Softmax is used to recalibrate the multi-scale attention vectors to obtain new multi-scale interactive attention weights. Then, the recalibrated attention weights and the corresponding feature maps are subjected to element-wise point-forming operations, and finally, a feature map with richer multi-scale information representation capability is output, realizing finer-grained extraction and fusion of multi-scale feature information and improving network accuracy.

[0018] Furthermore, in step 3), the last layer of the two-dimensional human pose estimation network outputs three feature maps of different sizes. An adaptive spatial feature fusion algorithm is used to fuse multi-scale features in the final stage of the network. Size and number of channels are selected as the feature fusion criteria for adaptive spatial feature fusion. The fused output is then subjected to a 1×1 convolution to obtain the final output.

[0019] Furthermore, in step 4), to avoid the loss of feature information in the human joint heatmap output by the 2D pose estimation network during 3D regression, which would degrade the network model performance, the joints of the 2D pose are estimated as integrals of all positions in the heatmap, and then normalized by probability-weighted summation. The specific calculation formula is as follows:

[0020]

[0021]

[0022] Where p is the position in the domain; q is the pixel related to the position; N k For the transformed joints; M k For heatmaps; This is the heatmap after regularization; Ω is M k The domain.

[0023] The beneficial effects of this invention are as follows: This invention provides a three-dimensional human pose estimation method for virtual fashion shows. Based on the original two-dimensional human pose estimation network HRNet-W32, it eliminates the fourth stage, which has serious information redundancy in the network. At the same time, it replaces the 3×3 convolution in the bottleneck and basicblock modules of the original network with a pyramid segmentation attention module, achieving more efficient multi-scale information extraction. Then, in the output stage, an adaptive spatial feature fusion strategy is used to fuse the features of each layer before outputting, so as to more fully obtain the semantic information of the high-level features. This compensates for the lack of feature extraction at low resolution caused by the elimination of the fourth stage, and finally achieves accurate calculation of two-dimensional human pose. When performing 3D regression from 2D pose estimation, this invention first performs integral regression on the joint heatmap output by the 2D pose estimation network to avoid feature information loss when using heatmaps for 3D regression. At the same time, it uses graph convolution combined with semantic information to overcome the problem that convolutional filters are limited to operating within the single-step neighborhood of each node, the receptive field of the convolutional kernel is always 1, and the network information exchange rate is seriously low. Finally, a non-local layer is introduced in the 3D regression part to capture the local and global relationships between nodes, thereby improving the performance of 3D human pose regression. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method for constructing a three-dimensional human pose estimation model for virtual fashion shows according to the present invention.

[0025] Figure 2 A schematic diagram of the pyramid segmentation attention module structure;

[0026] Figure 3a The architecture diagram after replacing the bottleneck module in HRNet-W32 with the pyramid segmentation attention module;

[0027] Figure 3b The architecture diagram after replacing the basicblock module in HRNet-W32 with the pyramid segmentation attention module;

[0028] Figure 4 This is a schematic diagram of the overall network structure of the 3D human pose estimation model for virtual fashion shows according to the present invention. Detailed Implementation

[0029] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0030] like Figure 1 The flowchart shown illustrates the construction process of a 3D human pose estimation model for a virtual fashion show, which includes the following steps:

[0031] 1. Image Data Acquisition: Process the model catwalk video in a single scene into frame-by-frame images to acquire the image data of the input video.

[0032] 2. Image Preprocessing: To address the issue that clothing occlusion and self-occlusion in video images can easily cause significant deviations in the prediction of human joint points, an improved Kalman filter is used to optimize the human motion state, reduce the deviation, and achieve image preprocessing.

[0033] Considering that the human body moves along each axis in three-dimensional space in the form of Bézier curves, the motion on each axis can be approximated as uniformly accelerated (decelerated) motion. Therefore, the prediction of the current position can be obtained by combining the state of the previous three positions with the change in acceleration.

[0034] Kalman filter optimization: First, obtain the first three states x. k-1 x k-2 x k-3 The value of is used to predict the current state based on the previous three states: in To predict location, For the observation location, K k This is the Kalman gain.

[0035] 3. Construction of a 2D Human Pose Estimation Network Model: A high-resolution 2D human pose estimation network based on HRNet-W32 was established. The standard HRNet-W32 network consists of four levels. The first level contains four residual units, each consisting of a 64-width bottleneck module followed by a 3x3 convolutional layer. The second, third, and fourth levels each contain several basicblock multi-resolution modules. Network Improvement: Through appropriate network pruning and analysis of ablation experiments on the original high-resolution network model, the first three stages of the network were retained. Simultaneously, a pyramid segmentation attention module was used to replace the original bottleneck and basicblock convolutional kernels, enhancing the receptive field of the image. Deep and shallow features were extracted to ensure multi-scale feature extraction, improving the accuracy of 2D joint prediction while reducing the number of system parameters and computational complexity.

[0036] 4. Training of the 2D human pose estimation network model: The network model is trained using the public dataset COCO2017. Before training, the images in the COCO2017 dataset need to be preprocessed, with the image size fixed at 256×192, the learning rate set to 0.001, and the training period set to 210. Mean average precision (mAP) and other metrics are used as evaluation criteria for 2D pose estimation.

[0037] 5. Construction of a 3D Human Pose Regression Network Model: A graph convolutional network based on residual connections and attention mechanisms is constructed, incorporating semantic information to achieve regression from 2D pose to 3D space. Based on the characteristics of skeleton-based human pose models, the human skeleton can be viewed as a graph structure data. Residual connections are used during graph convolution stacking to eliminate oversmoothing issues. To further explore the relationships between human keypoints and capture the hidden semantic information between different keypoints, an attention mechanism is used to obtain local and global contextual information between different keypoints through global context, while simultaneously addressing occlusion and depth blur issues in 3D human pose estimation.

[0038] The keypoint heatmap output by the 2D pose estimation network is used for integral regression to avoid feature information loss when using heatmaps for 3D regression. At the same time, graph convolution combined with semantic information is used to overcome the problem that the convolutional filter is limited to running in the single-step neighborhood of each node, the receptive field of the convolutional kernel is always 1, and the network information exchange rate is seriously low. Finally, a non-local layer is introduced in the 3D regression part to capture the local and global relationships between nodes, thereby improving the performance of 3D human pose regression.

[0039] 6. Training the 3D Human Pose Regression Network Model: The 3D human pose regression network was trained using sets S1, S5, S6, S7, and S8 from the Human 3.6M dataset, and sets S9 and S11 were used as validation sets to verify the performance. The Euclidean distance between the 3D joint coordinates predicted by the network and the true labeled human joint coordinates was used as the evaluation criterion for the final 3D human pose estimation result to verify the effectiveness of the 3D human pose estimation.

[0040] As shown in Tables 1 and 2, the fourth stage of the HRNet-W32 network increases the number of parameters and floating-point operations by 72.53% and 38.15% of the total number of parameters and floating-point operations, respectively. However, the human pose estimation accuracy on the MPII dataset only improves by 0.4%. The main reason for this is that as the number of network layers increases, the efficiency of feature information extraction decreases, and the receptive field scale during feature extraction exceeds the original image size, causing information redundancy. To reduce the floating-point operations and network parameters, this invention adjusts the feature receptive field of the HRNet-W32 network and removes the fourth stage, which has significant information redundancy.

[0041] Table 1

[0042]

[0043] Table 2

[0044]

[0045] like Figure 2 As shown, to compensate for the accuracy loss caused by removing the fourth stage of the HRNet-W32 network, this invention introduces a pyramid segmentation attention module to replace the 3×3 convolutions in the bottleneck and basic block modules of HRNet-W32. The two replaced modules are as follows: Figure 3a and 3b As shown.

[0046] The pyramid segmentation attention module mainly consists of an SPC module and an SE Weight module. The SPC module segments the attention channels and extracts multi-scale features from the spatial information of each channel's feature image. The SE Weight module extracts channel attention from feature images at different scales, thus obtaining channel attention vectors at each scale. Then, Softmax is used to recalibrate the multi-scale attention vectors to obtain new multi-scale interactive attention weights. Finally, the recalibrated attention weights and the corresponding feature maps are element-wise processed to output a feature map with richer multi-scale information representation capabilities. This enables finer-grained extraction and fusion of multi-scale feature information, improving network accuracy.

[0047] The mathematical expression is as follows:

[0048] For the input feature map X, it is divided into S parts, denoted as [X0, X1, ..., X...]. S-1 [Indicates the number of channels in each segment] C represents the total number of channels, and X represents the segmented feature map. i ∈R C′×W×H Let i = 0, 1, ..., S-1, where W represents the image width and H represents the image height. For each segmented channel feature map, multi-scale convolutional kernels are used for group convolution, extracting spatial information from feature maps of different scales while reducing the number of parameters. Feature maps F at different scales... i The specific calculation method is as follows: F i =Conv(K i ×K i G i (X) i ), i = 0, 1, ..., S-1

[0049] here,

[0050]

[0051] This yields the feature map after multi-scale fusion:

[0052] F = Cat([F0,F1,...,F) S-1 ])

[0053] Here F∈R C×W×H After extracting multi-scale feature maps, the feature maps F at different scales are processed. i Channel attention weights are extracted using the following formula:

[0054] Z i =SEWeight(F i ), i = 0, 1, ..., S-1

[0055] Here Z i ∈R C′×1×1 The entire multi-scale channel attention weight vector is:

[0056]

[0057] Then, Softmax is used to recalibrate the weights of channel attention to achieve information interaction between multi-scale channel attention.

[0058]

[0059] Finally, the feature map F at the corresponding scale is... iMultiply the attention vector with the weighted recalibrated vector at the channel-wise level, i.e.:

[0060] Y i =F i ⊙att i i = 1, 2, 3, ..., S-1

[0061] Finally, the multi-scale channel attention-weighted feature maps are concatenated to output a feature map with richer multi-scale information:

[0062] Out = Cat([Y0,Y1,...,Y) S-1 ]).

[0063] like Figure 4 As shown, to fully utilize the three different-sized feature maps in the last layer of the 2D human pose estimation network, this invention uses an adaptive spatial feature fusion algorithm (ASFF) to fuse multi-scale features in the final stage of the network, achieving more accurate keypoint detection. Considering that the sizes of the three feature maps are 1 / 4, 1 / 8, and 1 / 16 of the original image size, respectively, the size and number of channels of the feature map at 1 / 4 of the original image size are selected as the feature fusion standard. First, a 1×1 convolution is performed on the other two different-sized feature maps to ensure that their number of channels matches that of the feature map at 1 / 4 of the original image size. Second, the feature maps at 1 / 8 and 1 / 16 of the original image size are upsampled by 2x and 4x respectively to ensure that the sizes of the three feature maps are consistent. Finally, the three feature maps are X... 1 m,n X 2 m,n X 3 m,n Adaptive spatial feature fusion is performed, and the fused output is convolved with 1×1 to obtain the final output, so that the network always maintains high-resolution representation.

[0064] The three adjusted feature maps, each with the same size and number of channels, contain different local detail features. ASFF is used to fuse the three feature maps based on the weight parameters of each layer, defining a... m,n b m,n c m,n If the weights are the parameters, then the fusion strategy is as follows:

[0065] a m,n X 1 m.n +b m,n X 2 m,n +c m,n X 3 m,n =Y m,n

[0066] Where Y m,n For the fused feature map, a m,n b m,n c m,n ∈[0,1] and satisfy:

[0067] a m，n +b m，n +c m，n =1

[0068] And the weighting parameter a m,n b m,n c m,n This is by using X 1 m,n X 2 m,n X 3 m,n These three feature maps are obtained by performing a 1×1 convolution, and the parameter a m,n b m,n c m,n After contact, softmax is used to limit their range to [0,1] and their sum to 1. The specific calculation formula is as follows:

[0069]

[0070]

[0071]

[0072] Performing joint coordinate transformation on the heatmap output by a 2D human pose estimation network is significant for improving the network model's performance. This is because joint coordinate transformation eliminates the need for the heatmap to maintain high resolution at all times, fully leveraging the advantages of both heatmap representation and regression methods, and significantly reducing the computational complexity of subsequent network operations. This invention's integral regression of the 2D heatmap estimates joints as integrals across all locations in the heatmap, followed by probability-weighted summation and normalization. Since the integral is parameterless, integral regression has minimal impact on the network model's performance in terms of computation and storage. The specific integral formula is as follows:

[0073]

[0074]

[0075] Where p is the position in the domain; q is the pixel related to the position; M k For heatmaps; This is the heatmap after regularization; Ω is M k The domain.

[0076] The two-stage 3D human pose estimation first establishes a 2D pose estimation network to perform 2D human pose estimation on the input image or video, obtaining the coordinates of the 2D human joints, and then using these coordinates as input for 3D pose regression. Considering the interrelationships between human joints, the human pose constructed from the 2D human joints can be regarded as a graph structure, and graph convolution can be used to achieve 3D human pose regression. However, to solve the problem of multiple neighborhoods of graph nodes, graph convolution allows the convolutional filter to share the same weight matrix for all nodes; at the same time, it restricts the convolutional filter to operate within the single-step neighborhood of each node, making the receptive field of the convolutional kernel always 1, which seriously affects the information exchange efficiency when the network is deepened. To solve the above problems, this invention uses graph convolution combined with semantic information to learn the channel weights of the implicit prior edges in the 2D human pose, and combines them with the kernel matrix to improve the capability of graph convolution; to alleviate the oversmoothing problem caused by stacking of graph convolutional networks, a graph convolutional network model based on residual connections is constructed, and a non-local layer is introduced to capture the local and global relationships between nodes, improving the performance of 3D human pose regression.

[0077] Table 3 shows the experimental results of the first-stage 2D pose estimation network on the COCO 2017 validation set compared with other methods. The results indicate that the proposed method achieves better performance in 2D human pose estimation compared to other networks. Compared to the original network HRNet-W32, AP... 50 It increased by 3.1%, AP 75 The accuracy was improved by 2.7%, and the mean average precision (mAP) was improved by 2.2%. This demonstrates that the proposed method for 2D human pose estimation achieves higher accuracy than other networks.

[0078] Table 3

[0079]

[0080] As shown in Table 4, the accuracy of 2D keypoint detection was verified using 384×384 images on the COCO 2017 validation set. The proportion of correctly estimated keypoints (PCK) was calculated and compared with the proportions of correctly estimated keypoints of other network models. Here, `head` represents the average proportion of correctly estimated 5 keypoints related to the head; `shoulder` represents the average proportion of correctly estimated 2 keypoints related to the shoulder; `elbow` represents the average proportion of correctly estimated 2 keypoints related to the elbow; `wrist` represents the average proportion of correctly estimated 2 keypoints related to the chest; `buttocks` represents the average proportion of correctly estimated 2 keypoints related to the hip; `knee` represents the average proportion of correctly estimated 2 keypoints related to the knee; `ankle` represents the average proportion of correctly estimated 2 keypoints related to the ankle; and `average` represents the average proportion of correctly estimated keypoints across all joints. The comparison results in Table 4 show that the 2D pose estimation network proposed in the first stage of this invention improves the accuracy of keypoint estimation to a certain extent, and the average estimation accuracy also reaches a higher value.

[0081] Table 4

[0082]

[0083] Table 5 shows the experimental results of the proposed two-stage 3D human pose estimation network model on the Human3.6M dataset, compared with other methods. The results indicate that the proposed method can achieve a smaller Euclidean distance between the predicted 3D human joint coordinates and the actual labeled human joint coordinates in 3D human pose estimation, thus improving the accuracy of 3D human pose estimation.

[0084] Table 5

[0085]

[0086]

[0087] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for estimating three-dimensional human pose for virtual fashion shows, characterized in that, Specifically, the steps include the following: 1) Image data acquisition: Process the model catwalk video in a single scene into frame-by-frame images to acquire the image data of the input video; 2) Image preprocessing: Improved Kalman filtering is used to optimize human motion state and reduce the prediction deviation of human joint points caused by clothing occlusion and self-occlusion in video images, thus achieving image preprocessing; 3) Construct and train a two-dimensional human pose estimation network model, and feed the preprocessed image into the trained two-dimensional human pose estimation network model to obtain two-dimensional pose estimation. The structure of the two-dimensional human pose estimation network model includes the first three layers of the high-resolution two-dimensional human pose estimation network of HRNet-W32. The convolutional kernels of the bottleneck module in the first layer and the basic block module in the second and third layers are replaced by the pyramid segmentation attention module to compensate for the accuracy loss caused by removing the fourth layer of the HRNet-W32 network. The training of the two-dimensional human pose estimation network model: For this network model, the public dataset COCO2017 is used for training. Before training, the images in the COCO2017 dataset need to be preprocessed, the image size is fixed at 256×192, the learning rate is set to 0.001, the training period is set to 210, and the average accuracy index is used as the evaluation standard for two-dimensional pose estimation. 4) Construct and train a 3D human pose regression network model. Input the 2D pose estimation into the trained 3D human pose regression network model to perform 3D pose regression. The 3D human pose regression network model is a graph convolutional network based on residual connections and attention mechanisms. It uses graph convolution combined with semantic information to learn the channel weights of the implicit prior edges in the 2D human pose, and combines them with the kernel matrix to improve the ability of graph convolution. The human skeleton is regarded as a graph structure data, and residual connections are used to eliminate the oversmoothing problem during the graph convolution stacking process. The attention mechanism is used to obtain local and global contextual information between different key points through global context, while solving the problems of occlusion and depth blur in 3D human pose estimation. The training of the 3D human pose regression network model involved using S1, S5, S6, S7, and S8 from the Human 3.6M dataset as the training set and S9 and S11 as the validation set to verify the effect. The Euclidean distance between the 3D joint coordinates predicted by the network and the actual labeled human joint coordinates was used as the evaluation criterion for the final 3D human pose estimation result to verify the effectiveness of the 3D human pose estimation.

2. The three-dimensional human pose estimation method for virtual fashion shows according to claim 1, characterized in that, The specific method of improving human motion state by Kalman filtering in step 2) is as follows: the motion of the human body on each axis in three-dimensional space is a Bézier curve, and the motion on each axis is approximately uniformly accelerated or decelerated. The prediction of the current position is obtained by combining the state of the previous three positions with the change of acceleration. Kalman filter optimization: First, obtain the first three states x. k-1 x k-2 x k-3 The value of is used to predict the current state based on the previous three states: in To predict location, For the observation location, K k For Kalman gain.

3. The three-dimensional human pose estimation method for virtual fashion shows according to claim 1, characterized in that, In step 3), the pyramid segmentation attention module consists of an SPC module and an SE Weight module. The SPC module segments the attention channels and performs multi-scale feature extraction on the spatial information of each channel's feature image. The SE Weight module is used to extract the channel attention of feature images at different scales, thereby obtaining the channel attention vectors at each scale. Then, Softmax is used to recalibrate the multi-scale attention vectors to obtain new multi-scale interactive attention weights. Then, the recalibrated attention weights and the corresponding feature maps are subjected to element-wise point-forming operations, and finally, a feature map with richer multi-scale information representation capability is output, realizing finer-grained extraction and fusion of multi-scale feature information and improving network accuracy.

4. The three-dimensional human pose estimation method for virtual fashion shows according to claim 3, characterized in that, Step 3) The last layer of the two-dimensional human pose estimation network outputs three feature maps of different sizes. An adaptive spatial feature fusion algorithm is used to fuse multi-scale features in the final stage of the network. Size and number of channels are selected as the feature fusion criteria for adaptive spatial feature fusion. The fused output is then subjected to a 1×1 convolution to obtain the final output.

5. The three-dimensional human pose estimation method for virtual fashion shows according to claim 1, characterized in that, In step 4), to avoid the loss of feature information in the human joint heatmap output by the 2D pose estimation network during 3D regression, which would degrade the network model performance, the joints of the 2D pose are estimated as integrals of all positions in the heatmap, and then normalized by probability-weighted summation. The specific calculation formula is as follows: Where p is the position in the domain; q is the pixel related to the position; N k For the transformed joints; M k For heatmaps; This is the heatmap after regularization; Ω is M k The domain.

Images