A small-scale perceptually enhanced human pose estimation method

Abstract

The application discloses a small-scale perception enhanced human posture estimation method, which draws on the shortcomings of previous models, optimizes the performance of small-scale targets, selects a more accurate Top-down structure, discards the heat map representation method, and uses SimCC based on one-dimensional vector representation to more accurately locate the key points of small-scale targets, and a TAA module is further proposed based on the SimCC-baseline, and the effectiveness of the TAA module is verified through an ablation experiment, and finally good results are achieved in the overall AP. The performance of small-scale targets greatly limits the improvement of the overall accuracy of the model, and most of the previous networks ignore this point, so the SSA-Net has practical application value and scientific research value.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision and human pose estimation, and particularly relates to a small-scale perception enhanced human pose estimation method. BACKGROUND

[0002] Two-dimensional human pose estimation (HPE) is a very important branch in the field of computer vision, and has attracted the attention of many researchers. The specific method is to locate the human joint (knee, elbow, etc.) from the picture. HPE has a large number of applications in actual life, such as action recognition, animation production, augmented reality and other fields.

[0003] In the field of HPE, there are mainly two branches: one is a coordinate regression-based method, and the other is a heat map regression-based method.

[0004] Coordinate regression based method; the existing human key point dataset all annotates the joint points in the form of coordinates, so researchers are most likely to first think of making the network generate the same form of signal for supervised learning, so the coordinate regression based method is very popular in the early stage, the literature [Toshev, A., Szegedy, C.: Deep pose: Human pose estimation via deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1653-1660 (2014)] first converts the human pose estimation problem into the key point coordinate regression problem, after which many works are based on this, but as the research deepens, many problems are gradually exposed, one is that the numerical range of human key points is large and the distribution is scattered, which is not conducive to direct network learning; secondly, there is very rich constraint information between human key points and background, but the network output of the coordinate regression method is only x, y coordinates, and this information is lost, these shortcomings greatly limit the performance of the coordinate regression method, so that the method cannot surpass the heat map regression method for a long time. Until the successful proposal of the literature [Li, J., Bian, S., Zeng, A., Wang, C., Pang, B., Liu, W., Lu, C.: Human pose regression with residual log-likelihood estimation. In: Proceedings of the IEEE / CVF International Conference on Computer Vision. pp. 11025-11034 (ICCV 2021)], the coordinate regression method first surpasses the heat map based method to become the SOTA model, the core of the RLE work is to estimate the distribution probability density of the model output joint through the normalization flow, once the prior distribution function is estimated to be satisfactory, the loss function loss can be dynamically optimized, thereby promoting the regression training of the model.

[0005] Heatmap-based regression methods; Heatmap regression is a hot topic in the field of human pose estimation. Due to its excellent performance, it has been favored by many researchers, and even the literature [Li, Y., Zhang, S., Wang, Z., Yang, S., Yang, W., Xia, S. T., Zhou, E.: Tokenpose: Learning keypoint tokens for human pose estimation. arXiv preprint arXiv:2104.03516 (2021)] based on the Transformer work, ultimately cannot do without heatmap. The principle of heatmap regression is roughly to encode the label into a heatmap that conforms to the two-dimensional Gaussian distribution. The size of the heatmap is generally set to 1 / 4 of the input picture size. In the training process, the probability value in the Gaussian kernel is constantly adjusted. Finally, the index of the maximum probability point is extracted through the argmax function to calculate. The two-dimensional heatmap representation method has two outstanding advantages: first, because it uses a two-dimensional heatmap to represent key points, it can retain the spatial position information of key points; second, in many pictures, it is difficult to mark the joint node with a pixel point, because the points around it are also very similar to the joint node. If the surrounding points are set as negative labels, it is obviously unreasonable. The Gaussian kernel can well simulate the position of the joint node. As the earliest regression method using heatmap, the literature [Tompson, J., Jain, A., LeCun, Y., Bregler, C.: Joint training of a convolutional network and a graphical model for human pose estimation. preprint arXiv:1406.2984 (2014)] proposed to use the structural relationship between human key points and combine the idea of Markov random field to optimize the prediction results. The literature [Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: European conference on computer vision. pp. 483-499. Springer (2016)] proposed a conical hourglass network as a very effective way that has been used consistently to date.The document [Cheng, B., Xiao, B., Wang, J., Shi, H., Huang, T. S., Zhang, L.: Higher hrnet: Scaleaware representation learning for bottom-up human pose estimation. In: Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. pp. 5386-5395 (2020)] improves HRNet and proposes HigherHRNet, which uses a high-resolution feature pyramid for feature fusion. The purpose is to improve the performance of small and medium-sized figures to improve the overall accuracy, but the authors find that the final contribution is indeed the medium-sized figures, and the performance improvement for small-scale targets is not obvious. The document [Luo, Z., Wang, Z., Huang, Y., Tan, T., Zhou, E.: Rethinking the heatmap regression for bottom-up human pose estimation. arXiv preprint arXiv:2012.15175 (2020)] proposes a scale weight adaptive heatmap representation method to solve the scale problem of the Gaussian kernel. The results are surprisingly consistent with HigherHRNet. Tests show that the final contribution is mostly from medium-sized targets.

[0006] Heatmap regression limits small-scale targets; as shown by the work of Bowen Cheng and Zhengxiong Luo, although the authors are aware of the scale perception imbalance problem of mainstream models and have proposed corresponding solutions, they have not realized that the heatmap regression method is not suitable for small-scale targets, and even for small-scale target prediction, it is fatal.

[0007] First, when using heatmap regression, the label needs to be converted into a heatmap that conforms to the two-dimensional Gaussian distribution. The specific conversion method is as follows:

[0008]

[0009] Where (x, y) is the real key point coordinate, (i, j) is the coordinate on the heatmap, σ is the standard deviation, p represents the heatmap corresponding to the pth key point, and heatmap p (i, j) represents the probability value corresponding to the (i, j) point on the heatmap.

[0010] Assuming that the human body is labeled 17 key points, 17 heat maps will be generated, each heat map represents only one key point, and the real key point coordinates are (x, y). A Gaussian kernel with the point as the center will be generated on the heat map, and the closer the distance to the center, the greater the probability value. In the training process of the network, the probability value will be updated step by step. In the output stage, the most suitable point will be found in the two-dimensional heat map to calculate the loss by using argmax method. As can be seen from the above, the closer the distance to the target point, the greater the probability value of the point on the heat map. This is very obvious when the target scale is large and the points are relatively scattered. However, when the target scale is small and the key points are relatively dense, the Gaussian distribution cannot distinguish each key point well because the same standard deviation σ is used for different scales of targets in the same picture. Assuming that the nose and the right eye are the same, and the standard deviation σ is the same, it is easy to cause semantic confusion in the process of predicting small-scale targets.

[0011] As shown in Figure 1 , it can be more intuitively seen that when the target is small, the Gaussian kernel with the same standard deviation is not good for small targets; for small targets far away, the Gaussian kernels of the left eye, right eye, nose, and mouth are highly overlapped; when the target is large, the Gaussian kernel at the nose is just right, but when the target is small far away, the Gaussian kernel at the nose has covered the entire face, which is obviously unreasonable.

[0012] Secondly, the size of heat map is generally small, about 1 / 4 of the original size, assuming the real coordinates of the nose is (427, 427), the generated heat map coordinates is (427, 427) / / 4 = (106, 106), then even if the predicted value is exactly (106, 106), there will be a quantization error of 3 pixels 427-106*4 = 3, in the training process, as the target becomes smaller, the impact of this quantization error will be greater. Literature [Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: European conference on computer vision. pp. 483-499. Springer (2016)] and literature [Zigang Geng1,3*, Ke Sun1*, Bin Xiao3, Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression] try to add additional post-processing to reduce the quantization error, but this error is due to the characteristics of the heat map itself, so it can only be reduced but not eliminated. In view of this, we have reason to believe that if we start from the perspective of small scale target, the heat map based method may not be a good choice, which is due to the limitations of the heat map itself.

[0013] One-dimensional vector based regression method; In the field of facial landmark detection, the paper [Yin, S., Wang, S., Chen, X., Chen, E.: Attentive one-dimensional heatmap regression for facial landmark detection and tracking (2020)] introduced a co-attention mechanism to represent the edge distribution of x, y coordinates with two one-dimensional heatmaps. The paper [Xiong, Y., Zhou, Z., Dou, Y., Su, Z.: Gaussian vector: An efficient solution for facial landmark detection. In: Proceedings of the Asian Conference on Computer Vision (2020)] introduced a Band Pooling module to convert heatmaps into one-dimensional vectors for each pair of real coordinates. Back to the field of human pose estimation, the paper [Li, Y., Zhang, S., Wang, Z., Yang, S., Yang, W., Xia, S. T., Zhou, E.: Tokenpose: Learning keypoint tokens for human pose estimation. arXiv preprint arXiv:2104.03516 (2021)] proposed two token (one-dimensional vector) representation methods, first, the feature map is split by patch and then pulled into a one-dimensional feature vector, second, each real coordinate is also represented by a one-dimensional vector, and the two one-dimensional vectors are concatenated together and fed into the network for training. The paper [Yanjie Li1, Sen Yang2, Peidong Liu: SimCC: a Simple Coordinate Classification Perspective for Human Pose Estimation. In: ECCV2022] redefines the pose estimation task as a classification task of horizontal and vertical coordinates, and the proposed SimCC divides each pixel into multiple bins uniformly, achieving sub-pixel level positioning accuracy and low quantization error. SUMMARY

[0014] In view of the above, in the one-dimensional vector regression method, the SimCC sub-pixel level positioning is more friendly to the prediction of small figures, and therefore the application provides a small scale-aware enhanced human pose estimation method, which combines a TAA module, adopts a top-down method to construct an SSA-Net, and solves the scale-aware imbalance problem of mainstream networks, so as to achieve better results.

[0015] A small scale-aware enhanced human pose estimation method comprises the following steps:

[0016] (1) A large number of pictures containing figures are obtained, and the joint position of the figure in the picture is labeled;

[0017] (2) A model framework based on an SSA-Net (Small Scale-Aware Enhanced Network For Human Pose Estimation) is constructed, which comprises:

[0018] a BackBone module for extracting features of the input picture;

[0019] a TAA module for enhancing the extracted feature map;

[0020] a SimCC module for predicting the joint position according to the enhanced feature map;

[0021] (3) The above model framework is trained by using the pictures obtained in step (1) and the labeling information thereof;

[0022] (4) A picture containing a figure to be estimated is input into the trained model, and the position coordinates of each joint of the figure in the picture can be predicted.

[0023] Further, the BackBone module takes HRNet_W48 as a backbone network, retains a high-resolution branch, and performs feature fusion on the outputs of other resolution branches, and the performance is significantly higher than that of a ResNet, an Hourglass and the like.

[0024] Further, the input feature map of the TAA module is 1 / 4 of the original picture size, the module first uses a transpose convolution to change the size of the input feature map to 1 / 2 of the original picture size, then performs a 1x1 convolution operation on the feature map, expands the number of channels to twice the original number on the premise of keeping the size of the feature map unchanged, and then sends the feature map into a coordinate attention mechanism module for coding, and finally performs feature fusion on the input and output of the coordinate attention mechanism module through a residual mechanism to obtain the final output result of the TAA module.

[0025] Furthermore, the coordinate attention mechanism module first performs global average pooling on the input feature map in both the width and height directions to obtain feature maps in the width and height directions respectively. Then, the feature maps in these two directions are concatenated together and sequentially processed through a 1×1 convolution operation, batch normalization, and a sigmoid activation function to obtain feature map f. Finally, feature map f is convolved with 1×1 operations on its original height and width to obtain feature map F with the same number of channels as the input. h and F, F h After F is activated by the Sigmoid function, the attention weights g on the corresponding height and width are obtained. h and g w Finally, using g h and g w The input feature map is multiplied and weighted to obtain a feature map with attention weights in the height and width directions, which is the output of the coordinate attention mechanism module.

[0026] Further, the input of the SimCC module is the feature map H×W×C output from the TAA module, where H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels. H and W are respectively half the height and half the width of the original image. The SimCC module first flattens the input feature map of each channel into a one-dimensional vector with a length of HW. Then, it reduces the number of channels C of the feature map to N through a linear layer, where N is the number of keypoints. The one-dimensional vectors of the N channels are then passed through a horizontal axis coordinate classifier and a vertical axis coordinate classifier. The horizontal axis coordinate classifier reduces the length of the one-dimensional vector of the N channels to kW and extracts the maximum value from each channel vector, thus mapping the predicted x-coordinates of these maximum values. The vertical axis coordinate classifier reduces the length of the one-dimensional vector in the N channels to kH and extracts the maximum value in each channel vector, thereby mapping the predicted ordinate of the points corresponding to these maximum values. Finally, the predicted coordinates of each joint are output. i is a natural number and 1≤i≤N, k is a scaling factor greater than 1.

[0027] Furthermore, the specific implementation of step (3) is as follows:

[0028] 3.1 Initialize model parameters, including the bias vector and weight matrix of each layer, learning rate, and optimizer;

[0029] 3.2 Input the image containing people into the model, and the model outputs the corresponding prediction result, i.e., the coordinates of the key points, through forward propagation. Calculate the loss function between the prediction result and the annotation information.

[0030] 3.3 According to the loss function, the model parameters are continuously iteratively updated by the optimizer through gradient descent method until the loss function converges, and the training is completed.

[0031] Further, the loss function adopts the KL divergence between the prediction result and the labeled information.

[0032] The SSA-Net model designed in the application learns from the shortcomings of the previous models, optimizes the performance of small-scale targets, selects a more accurate Top-down structure, discards the heat map representation method, and uses SimCC based on one-dimensional vector representation to more accurately locate the key points of small-scale targets, and proposes a TAA module based on SimCC-baseline, which is also verified by ablation experiments. The effectiveness of the TAA module, and finally achieves good results in the overall AP.

[0033] The pose estimation of small-scale targets is a problem that cannot be ignored. From the perspective of practical application, the characters displayed in the picture in many scenarios are not as ideal as those in the MPII dataset. For example, in wide-angle video monitoring and remote shooting scenarios, the relationship between the characters themselves and the background is very complex, so the SSA-Net has great practical application value. From the perspective of scientific research, the performance of small-scale targets greatly limits the improvement of the overall precision of the model, and most of the previous networks ignore this point. Therefore, the SSA-Net model proposed in the application has both practical application value and scientific research value. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 It is a schematic diagram of the influence of the scale of the Gaussian kernel on the key parts of the small character.

[0035] Figure 2 It is a schematic diagram of the SSA-Net model framework of the application.

[0036] Figure 3 It is a schematic diagram of the structure of the TAA module in the model of the application.

[0037] Figure 4 It is a schematic diagram of the structure of the coordinate attention mechanism module in the model of the application.

[0038] Figure 5 It is a schematic diagram of the structure of the SimCC module in the model of the application. DETAILED DESCRIPTION

[0039] In order to more specifically describe the application, the technical solutions of the application will be described in detail below in combination with the drawings and specific embodiments.

[0040] The small-scale perception enhanced human pose estimation method of the application comprises the following steps:

[0041] (1) taking a two-dimensional picture containing a person as input, and taking labeled joint point coordinates as a supervision signal to perform supervised learning.

[0042] (2) constructing an SSA-Net model framework composed of a BackBone, a TAA module and a SimCC module, taking a two-dimensional picture as input, performing feature extraction through the BackBone, performing feature enhancement on the output feature map through the TAA module, and then performing key point prediction through the SimCC module.

[0043] In the one-dimensional vector regression method, the sub-pixel level positioning of SimCC is more friendly to the prediction of small figures, so the application decides to take SimCC as a Baseline, combine the TAA module, and adopt a top-down method to construct the SSA-Net, and the specific network structure is as shown in Figure 2 .

[0044] The BackBone in the model of the application selects HRNet_W48, HRNet retains a high-resolution branch, and the outputs of other resolution branches are fused, and the performance is significantly higher than that of ResNet, Hourglass and other backbone networks.

[0045] The TAA module in the model of the application can significantly improve the perception ability of the network for small-scale targets, wherein T represents a transpose convolution, the first A represents a coordinate attention mechanism, the second A represents a residual mechanism, and the structure of the TAA module is as shown in Figure 3 .

[0046] Firstly, the feature map output by the BackBone is about 1 / 4 of the size of the original picture, and we use a transpose convolution to change the size of the feature map to 1 / 2 of the size of the original picture, and the calculation formula is as follows:

[0047] H out =(H in -1)×stride[0]-2×padding[0]+kernel_size[0]

[0048] W out =(W in -1)×stride[1]-2×padding[1]+kernel_size[1]

[0049] wherein H represents the length of the feature map, W represents the width of the feature map, stride is the stride of the convolution kernel, kernel_size is the size of the convolution kernel, and padding is an important parameter used to calculate the padding of the feature map.

[0050] Then, the 1 / 2 size feature map is sent to a 1x1 convolution to expand the number of channels to twice the original number while keeping the feature map size unchanged.

[0051] Next, the feature map is sent to a coordinate attention mechanism module, as shown in Figure 4 . Specifically, the coordinate attention mechanism encodes precise positional information in both the height and width directions by first dividing the input feature map into two directions for global average pooling, obtaining feature maps in both width and height directions, as shown in the following formula:

[0052]

[0053]

[0054] where W is the width of the feature map and H is the height of the feature map.

[0055] Then, the width and height direction feature maps with global receptive field are concatenated and sent to a shared 1x1 convolution module to reduce the dimension to C / r of the original, and then the batch-normalized feature map F1 is sent to a Sigmoid activation function to obtain a feature map f of the form 1x(W+H)xC / r, as shown in the following formula:

[0056] f = δ (F1([z h ,z w ]))

[0057] The feature map f is convolved with a 1x1 convolution kernel in the original height and width to obtain feature maps F h and F with the same number of channels, and after Sigmoid activation, the attention weights g h and g w in the height and width directions are obtained, as shown in the following formula:

[0058] g h = σ (F h (f h ))

[0059] g w = σ (F w (f w ))

[0060] After the above calculation, the attention weights g h in the height direction and g wFinally, the final feature map with attention weight in width and height direction is obtained by multiplication weighting calculation on the original feature map, and the formula is as follows:

[0061]

[0062] In summary, the coordinate attention mechanism can be regarded as decomposing the channel attention into two 1D feature encoding processes of aggregating features along different directions, which has the advantages of capturing long-range dependencies along one spatial direction and preserving accurate position information along the other spatial direction. Then, the generated feature maps are respectively encoded to form a pair of direction-aware and position-sensitive feature maps, which are very helpful when the key points are dense for human pose estimation.

[0063] Finally, in order to make each module better play a role, a residual mechanism is added to fuse the output of the transpose convolution and the output of the coordinate attention mechanism.

[0064] The SimCC module in the model is a new coordinate representation method, which specifically comprises:

[0065] Coordinate encoding; in the method, the x and y coordinates of the key points are represented by two independent one-dimensional vectors, and the length of the one-dimensional vector obtained by a scaling factor k (>=1) will also be greater than or equal to the picture side length. For the pth key point, the encoded coordinates will be represented as:

[0066] p'=(x',y')=(round(x p ×k),round(y p ×k))

[0067] The scaling factor k uniformly divides each pixel into k bins, and its role is to enhance the positioning accuracy to a smaller level than a single pixel.

[0068] Coordinate decoding; assuming that the model outputs two one-dimensional vectors, it is natural that the coordinate calculation method of the predicted point is:

[0069]

[0070]

[0071] That is, the position of the maximum point on the one-dimensional vector is divided by the scaling factor to restore to the picture scale.

[0072] The overall network structure of SimCC is as shown in Figure 5 First, the output feature map of the TAA module is straightened into a one-dimensional vector of d dimensions, and then the coordinate classifier of the horizontal axis and the vertical axis is used to generate the predicted coordinates of the ith key point

[0073] SSA-Net learns the shortcomings of previous models, and optimizes the performance of small-scale targets. It selects a more accurate Top-down structure, discards the heat map representation method, and uses SimCC based on one-dimensional vector representation to more accurately locate the key points of small-scale targets. Meanwhile, the TAA module is proposed on the basis of SimCC-baseline, and the effectiveness of the TAA module is verified through ablation experiments. Finally, good results are achieved in the overall AP.

[0074] The pose estimation of small-scale targets is a problem that cannot be ignored. From the perspective of practical application, the characters shown in the picture in many scenarios are not as ideal as those in the MPII dataset. For example, in wide-angle video monitoring and remote shooting scenarios, the relationship between the characters and the background is very complex. Therefore, SSA-Net has great practical application value. From the perspective of scientific research, the performance of small-scale targets greatly limits the improvement of the overall precision of the model. Previous networks mostly ignore this point. Therefore, the proposal of SSA-Net has both practical application value and scientific research value.

[0075] (3) Train the model framework using the pictures and the supervision signal in step (1).

[0076] (4) Send the picture containing the character into the trained model framework, and the corresponding joint coordinates can be predicted.

[0077] The above description of the embodiments is to facilitate those skilled in the art to understand and apply the present application. Those skilled in the art can easily make various modifications to the above embodiments, and apply the general principles described herein to other embodiments without creative labor. Therefore, the present application is not limited to the above embodiments, and the improvements and modifications made by those skilled in the art based on the disclosure of the present application should be within the scope of protection of the present application.

Claims

1. A small-scale perceptual enhancement human pose estimation method, comprising the following steps: (1) Obtain a large number of pictures containing people, and label the joint positions of the people in the pictures; (2) Construct a model framework based on SSA-Net, which includes: BackBone module for feature extraction of the input picture; TAA module for feature enhancement of the extracted feature map; the input feature map of the TAA module is 1 / 4 of the original picture size, this module first uses transpose convolution to change the size of the input feature map to 1 / 2 of the original picture size, then the feature map is passed through a 1x1 convolution operation to expand the channel number to twice the original number under the premise of keeping the feature map size unchanged; then the feature map is sent to the coordinate attention mechanism module for encoding, and finally the input and output of the coordinate attention mechanism module are fused through the residual mechanism to obtain the final output result of the TAA module; The SimCC module predicts the position of the joint node according to the enhanced feature map; the input of the SimCC module is the feature map of HxWxC output by the TAA module, wherein H represents the height of the feature map, W represents the width of the feature map, C represents the number of channels, and H and W are 1 / 2 of the original picture height and width respectively; the SimCC module first flattens the input feature map of each channel into a one-dimensional vector form, and the vector length is HW, then reduces the number of channels C of the feature map to N through a linear layer, N is the number of joint nodes, and then the one-dimensional vectors of the N channels are classified through a horizontal axis coordinate classifier and a vertical axis coordinate classifier respectively; the horizontal axis coordinate classifier reduces the one-dimensional vector length of the N channels to kW and extracts the maximum value in each channel vector, and then maps the predicted point abscissa corresponding to the maximum values ; the vertical axis coordinate classifier reduces the one-dimensional vector length of the N channels to kH and extracts the maximum value in each channel vector, and then maps the predicted point ordinate corresponding to the maximum values ; finally, the predicted coordinates of each joint node are output , i is a natural number and 1≤i≤N, and k is a scaling factor greater than 1; (3) Train the above model framework using the pictures and their labeled information obtained in step (1); (4) Input the picture containing the person to be estimated into the trained model, and the position coordinates of each joint of the person in the picture can be predicted.

2. The human pose estimation method of claim 1, wherein: The BackBone module uses HRNetW48 as the backbone network, retains the high-resolution branch, and fuses the outputs of other resolution branches.

3. The human pose estimation method of claim 1, wherein: The coordinate attention mechanism module firstly performs global average pooling on the input feature map from the width and height directions respectively to obtain feature maps in the width and height directions, then concatenates the feature maps in the two directions, and sequentially passes through a 1*1 convolution operation, batch normalization processing and a Sigmoid activation function to obtain a feature map f, and then performs a 1*1 convolution operation on the feature map f according to the original height and width to obtain a feature map with the same number of channels as the input and , and respectively pass through a Sigmoid activation function to obtain attention weights in the corresponding height and width directions and , and finally use and to perform multiplication weighting calculation on the input feature map to obtain a feature map with attention weights in the height and width directions as the output result of the coordinate attention mechanism module.

4. The human pose estimation method of claim 1, wherein: The specific implementation of step (3) is as follows: 3.1 Initialize the model parameters, including the bias vector and weight matrix of each layer, learning rate and optimizer; 3.2 Input the picture containing the person into the model, and the corresponding prediction result, i.e. the joint position coordinates, is obtained by forward propagation of the model, and the loss function between the prediction result and the labeled information is calculated; 3.3 According to the loss function, the model parameters are iteratively updated by the optimizer through gradient descent method until the loss function converges, and the training is completed.

5. The human pose estimation method of claim 4, wherein: The loss function adopts the KL divergence between the prediction result and the labeled information.

Images