Lightweight two-dimensional human body posture estimation method suitable for low-computing-power platform

Abstract

The invention discloses a lightweight two-dimensional human body posture estimation method suitable for a low-computing-power platform. The method comprises the steps that real-time image data containing a human body are collected, and an image is preprocessed; building a lightweight feature extraction module, and extracting high-dimensional features in the image by using a convolutional neural network; building a feature fusion module based on a channel attention mechanism, and fusing a plurality of high-dimensional features; a guide training module is constructed, and the model is helped to improve the precision under the condition that the parameter quantity is not increased; and constructing a human body posture decoder based on sub-pixel classification, and predicting the accurate position of each articulation point of the human body in the image from the high-dimensional features. According to the method, a lightweight two-dimensional human body posture estimation technology is used, the problems that a two-dimensional human body posture estimation algorithm is poor in detection robustness and slow in reasoning time under the condition of low calculation power are solved, the model parameter quantity is reduced, meanwhile, the detection precision of the model is guaranteed, and the detection efficiency is improved. The robustness and the reasoning speed of two-dimensional human body posture estimation on a low-computing-power platform are effectively improved.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and computer vision, and in particular to a lightweight two-dimensional human posture estimation method suitable for a low computing power platform. Technical Background

[0002] As a key technology in the field of computer vision, human posture estimation plays an important role in many fields such as motion recognition, motion capture, motion analysis, and medical rehabilitation. With the continuous development of deep learning models, the accuracy of human posture estimation algorithms has been continuously improved, and the application prospects have become broader. However, in practical applications, due to the limitations of the computing power of terminal devices, the balance between algorithm accuracy and resource consumption is crucial.

[0003] In the experiment, we found that the richness of features has a significant impact on the prediction results of the model. Generally speaking, rich features provide more reference information, which helps to improve the accuracy and universality of the model. However, rich features may also contain redundant information. In order to help the model identify features that are useful for prediction and reduce the interference of redundant features, the present invention proposes a channel attention mechanism based on feature contribution values. This mechanism assigns a weight to each feature in the feature fusion stage and adjusts the weight according to the contribution of the feature to the prediction result. For features with greater contributions, higher weights are assigned; for features with smaller contributions or irrelevant features, lower weights are assigned to reduce their negative impact on subsequent predictions. This method enables the model to pay more attention to important features while suppressing the interference of irrelevant or harmful features.

[0004] In addition, the fineness of the features also has an important impact on the performance of the model. Fine-grained features can help the model capture more accurate and rich detail information, which is particularly important for prediction tasks that require high precision; while coarse-grained features focus on global and macro-level feature information, can provide a wider context, and help the model understand and learn features from a holistic perspective. Based on this, the present invention introduces a guided training module based on the backbone network, which receives the outputs of the feature extraction module and the feature fusion module respectively, and generates results similar to the output of the backbone network decoder by fusing these outputs. The core advantage of this design is that the coarse-grained features of the guided training module can help the model obtain a wider range of context information at the global level, so that the model will not lose global coordination in the learning of detail features, and avoid overfitting of detail features in the early stage of training. Since the output of the guided training module is the same as the output of the posture decoder of the backbone network, the two can cooperate with each other during the training process. The backbone network can use the coarse-grained features provided by the guided training module to improve the stability of training and the final detection accuracy during fine-tuning. At the same time, since the backbone network participates alone during reasoning, the model can maintain efficient and accurate prediction capabilities, avoiding redundant calculations of the guided training module during reasoning. This multi-level feature learning strategy not only improves the accuracy of the model through the effective combination of fine-grained and coarse-grained features, but also enhances the model's adaptability to different data distributions.

[0005] Finally, we note that the traditional two-dimensional human posture estimation model usually performs multiple upsampling on the extracted features before predicting the joint position. However, a large amount of upsampling will not only consume more computing resources, but may also introduce certain computing errors in the upsampling process. This may cause the features used for joint position prediction to contain large cumulative errors, thereby affecting the prediction accuracy. These errors may accumulate in the feature map, thereby affecting the final prediction accuracy of the joint position. In order to avoid the negative impact of these cumulative errors, the present invention abandons the use of upsampling layers in the model design, but adopts a method based on the SIMCC algorithm in the posture decoder to directly predict the joint position from the high-dimensional feature map. The advantage of this method is that the SIMCC algorithm can effectively extract accurate joint position information from the high-dimensional feature map without going through multiple upsampling steps, thereby avoiding the accumulation of errors and the waste of computing resources. This design not only improves the running speed of the present invention on a low-computing power platform, but also ensures the prediction accuracy of the model. Summary of the invention

[0006] This invention adopts advanced lightweight two-dimensional human pose estimation technology, which effectively solves the challenges of insufficient detection robustness and slow reasoning speed faced by traditional two-dimensional human pose estimation algorithms in computing power-constrained environments. By optimizing the algorithm, the present invention can still maintain high-precision two-dimensional human pose estimation under low computing power conditions, greatly improving the practicality of the algorithm. This innovation provides a practical solution for human pose estimation and action recognition in low-computing power application scenarios such as monitoring, smart phones, and smart door locks.

[0007] To achieve the above object, the technical solution of the present invention is as follows: A lightweight two-dimensional human posture estimation method suitable for a low computing power platform comprises the following steps:

[0008] S1, collect real-time RGB images of any angle and shape of the human body;

[0009] S2, preprocessing the collected RGB image;

[0010] S3. Build a lightweight feature extraction module based on group convolution, input the preprocessed image data into the feature extraction module, and use convolutional neural network to extract high-dimensional features of three different dimensions;

[0011] S4, build a feature fusion module, which takes the high-dimensional features of three different dimensions extracted by the feature extraction module as input, and uses the channel attention mechanism based on the feature contribution value to fuse the high-dimensional features of three different dimensions into one high-dimensional fine-grained feature. The channel attention mechanism based on the feature contribution value can help the model automatically identify and strengthen the features with large contribution values ​​to posture prediction, and reduce the impact of redundant features on the prediction results; S5, build a guided training module, which takes the outputs of S3 and S4 as two inputs respectively, and uses the channel attention mechanism based on the feature contribution value to fuse multiple input features of different dimensions again, generate high-dimensional coarse-grained features and predict the position of the joints based on the features. This module only participates in the calculation during training, providing more comprehensive global context information for the backbone network, helping the algorithm to learn effective feature representations faster in the early stages of training and converge faster in the later stages. Since this module does not participate in the calculation during the final prediction, the model detection accuracy is improved without increasing the amount of calculation;

[0012] S6. Build a human posture decoder based on sub-pixel classification. Based on the fine-grained high-dimensional features output by S4, use regression analysis to predict the horizontal and vertical coordinate values ​​of each joint of the human body in the image from the fine-grained high-dimensional features. In the model training stage, the output of S6 and the output of S5 together constitute the loss function.

[0013] Furthermore, in S1, RGB images of the human body at any angle and in any form are collected, including single-person scenes, multi-person scenes, motion, stillness, or interactive behaviors.

[0014] Furthermore, in S2, the target detection algorithm is used on the collected RGB image to obtain the area where each person in the image is located, and the content of the area is resized and the image data is normalized.

[0015] Furthermore, in S3, a lightweight feature extraction module based on grouped convolution is built. The module is composed of ShuffleNetV2, which is a lightweight convolutional neural network architecture designed for efficient operation in environments with limited computing resources such as mobile devices and embedded systems. It is mainly used in the field of target detection. By introducing strategies such as deep separable convolution, channel segmentation and channel exchange, the computational overhead and memory requirements of the model are significantly reduced. Deep separable convolution splits traditional convolution into deep convolution and point-by-point convolution, which can significantly reduce the amount of convolution calculation. Assuming the input feature map size is H×W×C in , where H represents the height of the feature map, W represents the width of the feature map, and C in Represents the number of channels of the feature map. Assuming the convolution kernel size is k, the number of channels of the output feature map is C out , then when using traditional convolution to process the feature map, the amount of calculation is as follows:

[0016] H×W×C in ×C out ×k 2

[0017] When using depth-wise separable convolution to process the feature map, the depth-wise convolution applies a convolution kernel to each input channel separately, instead of using one convolution kernel to process all channels like traditional convolution. Therefore, the computational cost of the depth-wise convolution is:

[0018] H×W×C in ×k 2

[0019] Point-by-point convolution uses a 1×1 convolution kernel to linearly combine all input channels in the spatial dimension, thereby reducing the number of channels of the input feature map from C in Mapping to C out , its computational complexity is:

[0020] H×W×C in ×C out

[0021] According to the above formula, the computational complexity of separable convolution can be obtained as:

[0022] H×W×C in×k 2 +H×W×C in ×C out

[0023] Compared with traditional convolution, its computational complexity is greatly reduced, especially when the convolution kernel is large; in addition, the channel splitting operation divides the input feature channel into two parts, one part of the channel is not calculated and directly jump-connected, and the other part is processed through a lightweight convolution layer, which reduces the number of channels that need to be convolved and reduces the amount of computation for each convolution layer; in order to help different feature channels to fully exchange information, the channel exchange mechanism can be used to shuffle the feature channels and rearrange the data from different channels after each channel merge, which can effectively avoid the problem of feature information isolation caused by channel splitting and effectively improve the model's ability to express complex features.

[0024] Furthermore, a feature fusion module is built in S4. In the CNN-based target detection model, the feature extraction module usually outputs features of different dimensions. The feature pyramid is used to realize the information interaction between features of different dimensions through different feature fusion strategies and output fused features with the same shape as the input features. These features are used for tasks such as bounding box prediction, category recognition, and confidence reasoning. However, in the application scenario of the present invention, since there is only one task of predicting the coordinates of human joints, it is only necessary to fuse multiple features of different dimensions into a high-dimensional feature. In order to meet this demand, the present invention innovatively proposes a feature integration module based on the channel attention mechanism, which helps the model achieve more efficient feature fusion on the basis of adding a small number of parameters. Assuming that the input feature F i Shape H f ×W f ×C f , where H f , W f , C f Representing its height, width and number of channels respectively, the input features are first reduced and increased in dimension through a set of "hourglass" models composed of 1×1 convolutions. In the process of dimensionality reduction, the model learns the contribution of each value in the feature to subsequent processing, and then restores the original shape through dimensionality increase. The features after dimensionality increase will pass through the Sigmoid activation function to generate a weight tensor W i , whose shape is similar to the input feature F i The same, and the size of each element is in the interval [-1,1], and each value in the tensor represents the importance of different channels; as shown below:

[0025] F o =F i +W i ⊙F i

[0026] The channel attention mechanism adjusts the output features F of each channel by multiplying each value in the input feature with the value of the corresponding position in the weight tensor and then adding them together. o , where ⊙ represents W i With F i Multiply the values ​​of the corresponding dimensions in W; i The values ​​in all belong to the interval [-1,1], so F i The values ​​that contribute more to posture estimation will be amplified by 1 to 2 times after the channel attention mechanism. Similarly, the values ​​that contribute less or are irrelevant to subsequent processing will be reduced after the channel attention mechanism, and the minimum value is 0. This operation can help the model pay more attention to these important features in subsequent predictions, while weakening or ignoring features that are irrelevant or even harmful to the prediction, thereby reducing the negative impact of their interference. Through this optimization strategy, the model shows higher accuracy in joint point positioning, effectively improving the overall human posture estimation performance.

[0027] Furthermore, in S5, a training guidance module is built. This module takes the output of the feature extraction module and the feature fusion module as input, uses the channel attention mechanism to fuse multiple input features, generates coarse-grained high-dimensional features, and predicts the position of the joint point based on the features; in the early stage of training, the training guidance module not only updates its own parameters, but also provides additional gradient feedback to the feature extraction module and the feature fusion module to help the backbone network learn quickly and accelerate convergence in the early stage; in the middle stage of training, since the backbone network has learned a better feature expression method, the training guidance module no longer returns the gradient to the backbone network, but only updates its own parameters; In the later stage of training, in order to help the backbone network approach the global optimal solution faster, the training guidance module starts to return the gradient to the backbone network again; during the entire training process, the training guidance module continues to participate in the calculation of the loss function, which can ensure that the overall performance of the model remains stable; through this design, although the overall number of model parameters increases during training, the model training time does not change significantly, and the accuracy is also improved to a certain extent. Since the training guidance module only participates in reasoning during model training and only the backbone network participates in prediction, by adding this module, the model accuracy can be improved without increasing the number of model parameters in actual application;

[0028] Furthermore, in S6, a human posture decoder based on sub-pixel classification is built. Previous two-dimensional human posture estimation tasks usually adopt the method of directly regressing joint point coordinates from feature maps or using heat maps to regress joint point positions. However, these methods often rely on more complex model structures and have a large amount of calculation; in addition, since these methods directly regress joint point coordinates from high-dimensional features, they are usually accompanied by large cumulative errors. Especially when processing complex scenes or large-scale data, errors are easily accumulated in multiple steps, affecting accuracy and efficiency; in order to solve these problems, the present invention uses the method of converting the two-dimensional human posture estimation task into the classification of the horizontal and vertical coordinate joint points belonging to the human body from the perspective of sub-pixel accuracy; since the pooling layer that may cause the loss of spatial features in the traditional method is discarded, the model structure is simplified and the cumulative error is effectively avoided. This method not only reduces the amount of calculation, but also significantly improves the prediction accuracy and stability. In the data processing stage, the present invention adopts a Gaussian smoothing label strategy instead of the traditional single-point label. This Gaussian smoothing strategy introduces spatial context information into the label, so that the model can better handle subtle changes in the joint points and incorporates the inductive bias into the model training process, thereby effectively improving the accuracy of joint point prediction. Due to the use of the Gaussian smoothing label strategy, the true label of the joint point position is expressed in the form of probability distribution. In order to unify the expression of the predicted value and the label and avoid numerical overflow, this paper uses log-softmax to calculate the predicted value of each joint point. The following formula expresses the prediction value processing process related to all x-coordinates, where i = 1, 2, ..., D, D represents the dimension of each predicted value,

[0029]

[0030] In order to measure the difference between the predicted probability distribution and the true probability distribution, this paper uses the KL divergence (Kullback-Leibler Divergence) loss function to calculate the loss value of the model. The calculation method is as follows, where G i represents the true label of the joint point, P i Represents the probability distribution of the predicted value of the joint point:

[0031]

[0032] For joint point j, the loss of calculating x-coordinate and y-coordinate is:

[0033]

[0034] Where B is the batch size, w b,jis the predefined weight of each joint point. At the same time, in order to ensure the consistency of the model throughout the training process, the prediction results of the guided training module are also added to the calculation of the loss function. The final loss function is:

[0035]

[0036] N represents the number of human joints, loss p,j and loss t,j represents the loss value of the prediction result of the backbone network and the loss value of the prediction result of the guided training module; w p and w t They respectively represent the weight of the loss value of the backbone network prediction result and the weight corresponding to the loss value of the guided training module; in the design of the present invention, two 1×1 convolutional layers are used to directly predict the horizontal and vertical coordinates of the human body joints from the output of the feature fusion module. This method predicts the coordinates of the joints by direct classification, which reduces the accumulation of errors compared to traditional regression methods and makes the model more efficient.

[0037] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the lightweight two-dimensional human body posture estimation method applicable to a low computing power platform is implemented.

[0038] A computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the lightweight two-dimensional human body posture estimation method suitable for a low-computing-power platform.

[0039] Compared with the prior art, the advantages of the present invention are as follows:

[0040] The present invention proposes a novel lightweight two-dimensional human posture estimation method based on the target detection model structure. The method uses group convolution to replace the traditional convolution in the model to reduce model parameters, and uses a channel exchange mechanism to enhance feature interaction to make up for the shortcomings that may be caused by group convolution. In addition, the present invention innovatively introduces a self-learning channel attention mechanism, which enables the model to automatically identify and enhance features that contribute more to posture prediction, effectively improving the accuracy of low-parameter models. At the same time, in order to improve training efficiency and reduce training overhead, a guided training module is also integrated into the algorithm. This module only participates in calculations during training, and by providing more coarse-grained features containing global feature information to the backbone network, it helps the algorithm learn effective feature representations faster in the early stages of training and converge faster in the later stages. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1It is a flowchart of a lightweight two-dimensional human body posture estimation method suitable for a low computing power platform in an embodiment of the present invention;

[0042] Figure 2 This is a two-dimensional human body posture estimation network model diagram of a lightweight two-dimensional human body posture estimation method suitable for a low computing power platform in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application.

[0045] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0046] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0047] Example 1

[0048] like Figure 1 , Figure 2 As shown, a lightweight two-dimensional human posture estimation method suitable for low computing power platforms includes the following steps:

[0049] S1, collect real-time RGB images of any angle and shape of the human body;

[0050] In S1, RGB images of the human body at any angle and shape are collected, including single-person scenes, multi-person scenes, motion, stillness or interactive behaviors;

[0051] S2, preprocessing the collected RGB image;

[0052] In S2, the target detection algorithm is used to obtain the area where each person in the image is located on the collected RGB image, and the content of the area is resized and the image data is normalized;

[0053] S3. Build a lightweight feature extraction module based on group convolution, input the preprocessed image data into the feature extraction module, and use convolutional neural network to extract high-dimensional features in the image;

[0054] In S3, a lightweight feature extraction module based on grouped convolution is built. The module is composed of ShuffleNet V2, which is a lightweight convolutional neural network architecture designed for efficient operation in environments with limited computing resources such as mobile devices and embedded systems. It is mainly used in the field of target detection. By introducing strategies such as deep separable convolution, channel segmentation and channel exchange, the computational overhead and memory requirements of the model are significantly reduced. Deep separable convolution splits traditional convolution into deep convolution and point-by-point convolution, which can significantly reduce the amount of convolution calculation. Assuming the input feature map size is H×W×C in , where H represents the height of the feature map, W represents the width of the feature map, and C in Represents the number of channels of the feature map. Assuming the convolution kernel size is k, the number of channels of the output feature map is C out , then when using traditional convolution to process the feature map, the amount of calculation is as follows:

[0055] H×W×C in ×C out ×k 2

[0056] When using depth-wise separable convolution to process the feature map, the depth-wise convolution applies a convolution kernel to each input channel separately, instead of using one convolution kernel to process all channels like traditional convolution. Therefore, the computational cost of the depth-wise convolution is:

[0057] H×W×C in ×k 2

[0058] Point-by-point convolution uses a 1×1 convolution kernel to linearly combine all input channels in the spatial dimension, thereby reducing the number of channels of the input feature map from C in Mapping to C out , its computational complexity is:

[0059] H×W×C in ×C out

[0060] According to the above formula, the computational complexity of separable convolution can be obtained as:

[0061] H×W×C in×k 2 +H×W×C in ×C out

[0062] Compared with traditional convolution, its computational complexity is greatly reduced, especially when the convolution kernel is large; in addition, the channel segmentation operation divides the input feature channel into two parts, one part of the channel is not calculated and directly jumps to the other part, and the other part is processed by a lightweight convolution layer, which reduces the number of channels that need to be convolved and reduces the amount of computation for each convolution layer; in order to help different feature channels to fully interact with each other, the channel exchange mechanism can be used to disrupt the feature channels and rearrange the data from different channels after each channel merge, which can effectively avoid the problem of feature information isolation caused by channel segmentation and effectively improve the model's ability to express complex features;

[0063] S4. Build a feature fusion module based on the channel attention mechanism. This module takes the three different dimensional features extracted by the feature extraction module as input. Based on the channel attention mechanism, the model can automatically identify and enhance the features with greater contribution to posture prediction, and fuse the three different dimensional features into one high-dimensional feature.

[0064] In S4, a feature fusion module based on the channel attention mechanism is built. In the target detection model based on CNN, the feature extraction module usually outputs features of different dimensions. The feature pyramid is used to realize the information interaction between features of different dimensions through different feature fusion strategies and output the fused features with the same shape as the input features. These features are used for tasks such as bounding box prediction, category recognition, and confidence reasoning. However, in the application scenario of the present invention, since there is only one task of predicting the coordinates of human joints, it is only necessary to fuse multiple features of different dimensions into a high-dimensional feature. In order to meet this demand, the present invention innovatively proposes a feature integration module based on the channel attention mechanism, which helps the model achieve more efficient feature fusion on the basis of adding a small number of parameters. Assuming that the input feature F i Shape H f ×W f ×C f , where H f , W f , C f Representing its height, width and number of channels respectively, the input features are first reduced and increased in dimension through a set of "hourglass" models composed of 1×1 convolutions. In the process of dimensionality reduction, the model learns the contribution of each value in the feature to subsequent processing, and then restores the original shape through dimensionality increase. The features after dimensionality increase will pass through the Sigmoid activation function to generate a weight tensor W i , whose shape is similar to the input feature F iThe same, and the size of each element is in the interval [-1,1], and each value in the tensor represents the importance of different channels; as shown below:

[0065] F o =F i +W i ⊙F i

[0066] The channel attention mechanism adjusts the output features F of each channel by multiplying each value in the input feature with the value of the corresponding position in the weight tensor and then adding them together. o , where ⊙ represents W i With F i Multiply the values ​​of the corresponding dimensions in W; i The values ​​in all belong to the interval [-1,1], so F i The values ​​that contribute more to posture estimation will be amplified by 1 to 2 times after the channel attention mechanism. Similarly, the values ​​that contribute less or are irrelevant to subsequent processing will be reduced after the channel attention mechanism, and the minimum value is 0. This operation can help the model pay more attention to these important features in subsequent predictions, while weakening or ignoring features that are irrelevant or even harmful to the prediction, thereby reducing the negative impact of their interference. Through this optimization strategy, the model shows higher accuracy in joint point positioning, effectively improving the overall human posture estimation performance.

[0067] S5. Construct a guided training module. This module uses the output of the feature extraction module and the feature fusion module as input, fuses multiple input features using the channel attention mechanism, generates high-dimensional features, and predicts the position of the joint point based on the features. This module only participates in the calculation during training, helping the algorithm to learn effective feature representations faster in the early stages of training and converge faster in the later stages. Since this module does not participate in the calculation during prediction, it improves the model detection accuracy without increasing the amount of calculation.

[0068] In S5, a training guidance module is built. This module takes the output of the feature extraction module and the feature fusion module as input, uses the channel attention mechanism to fuse multiple input features, generates coarse-grained high-dimensional features, and predicts the position of the joint point based on the features; in the early stage of training, the training guidance module not only updates its own parameters, but also provides additional gradient feedback to the feature extraction module and the feature fusion module to help the backbone network learn quickly and accelerate convergence in the early stage; in the middle stage of training, since the backbone network has learned a better feature expression method, the training guidance module no longer returns the gradient to the backbone network, but only updates its own parameters. number; in the later stage of training, in order to help the backbone network approach the global optimal solution faster, the training guidance module starts to return the gradient to the backbone network again; during the entire training process, the training guidance module continues to participate in the calculation of the loss function, which can ensure that the overall performance of the model remains stable; through this design, although the overall number of model parameters increases during training, the model training time does not change significantly, and the accuracy is also improved to a certain extent. Since the training guidance module only participates in reasoning during model training and only the backbone network participates in prediction, by adding this module, the model accuracy can be improved without increasing the number of model parameters in actual application;

[0069] In S6, a human posture decoder based on sub-pixel classification is built. Previous two-dimensional human posture estimation tasks usually adopt the method of directly regressing joint point coordinates from feature maps or using heat maps to regress joint point positions. However, these methods often rely on more complex model structures and have a large amount of calculation; in addition, since these methods directly regress joint point coordinates from high-dimensional features, they are usually accompanied by large cumulative errors. Especially when processing complex scenes or large-scale data, errors are easily accumulated in multiple steps, affecting accuracy and efficiency; in order to solve these problems, the present invention uses the method of converting the two-dimensional human posture estimation task into classifying the horizontal and vertical coordinate joint points belonging to the human body from the perspective of sub-pixel accuracy; since the pooling layer that may cause the loss of spatial features in the traditional method is discarded, the model structure is simplified and the cumulative error is effectively avoided. This method not only reduces the amount of calculation, but also significantly improves the prediction accuracy and stability. In the data processing stage, the present invention adopts a Gaussian smoothing label strategy instead of the traditional single-point label. This Gaussian smoothing strategy introduces spatial context information into the label, so that the model can better handle subtle changes in the joint points and incorporates the inductive bias into the model training process, thereby effectively improving the accuracy of joint point prediction. Due to the use of the Gaussian smoothing label strategy, the true label of the joint point position is expressed in the form of probability distribution. In order to unify the expression of the predicted value and the label and avoid numerical overflow, this paper uses log-softmax to calculate the predicted value of each joint point. The following formula expresses the prediction value processing process related to all x-coordinates, where i = 1, 2, ..., D, D represents the dimension of each predicted value,

[0070]

[0071] In order to measure the difference between the predicted probability distribution and the true probability distribution, this paper uses the KL divergence (Kullback-Leibler Divergence) loss function to calculate the loss value of the model. The calculation method is as follows, where G i represents the true label of the joint point, P i Represents the probability distribution of the predicted value of the joint point:

[0072]

[0073] For joint point j, the loss of calculating x-coordinate and y-coordinate is:

[0074]

[0075] Where B is the batch size, w b,j is the predefined weight of each joint point. At the same time, in order to ensure the consistency of the model throughout the training process, the prediction results of the guided training module are also added to the calculation of the loss function. The final loss function is:

[0076]

[0077] N represents the number of human joints, loss p,j and loss t,j represents the loss value of the prediction result of the backbone network and the loss value of the prediction result of the guided training module; w p and w t They respectively represent the weight of the loss value of the backbone network prediction result and the weight corresponding to the loss value of the guided training module; in the design of the present invention, two 1×1 convolutional layers are used to directly predict the horizontal and vertical coordinates of the human body joints from the output of the feature fusion module. This method predicts the coordinates of the joints by direct classification, which reduces the accumulation of errors compared to traditional regression methods and makes the model more efficient.

[0078] The method proposed in this paper contains only about 3.82 million parameters. It is evaluated on the COCO 2017val dataset using the average precision (AP50-90) indicator and achieves an accuracy of 69.1%. Compared with methods of the same parameter level, this algorithm has achieved a significant improvement in accuracy. This innovation provides a practical solution for human posture estimation and action recognition in low-computing application scenarios such as surveillance, smartphones, and smart door locks.

[0079] The comparative data is shown in Table 1:

[0080] Table 1:

[0081]

[0082] Embodiment 2:

[0083] This embodiment provides a lightweight two-dimensional human posture estimation method and device suitable for low computing power platforms, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method according to Example 1.

[0084] Embodiment 3:

[0085] This embodiment provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in Embodiment 1. These computer program instructions may also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions executed on the computer or other programmable device for implementing the process described in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0086] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above technical features.

Claims

1. A lightweight two-dimensional human posture estimation method suitable for low computing power platforms, characterized in that: The steps include: S1, collect real-time RGB images of any angle and shape of the human body; S2, preprocessing the collected RGB image; S3. Build a lightweight feature extraction module based on group convolution, input the preprocessed image data into the feature extraction module, and use convolutional neural network to extract high-dimensional features of three different dimensions; S4. Build a feature fusion module. This module takes the high-dimensional features of three different dimensions extracted by the feature extraction module as input, and uses the channel attention mechanism based on feature contribution value to fuse the high-dimensional features of three different dimensions into one fine-grained high-dimensional feature. S5, build a guided training module, which takes the outputs of S3 and S4 as two inputs respectively, uses the channel attention mechanism based on feature contribution value to fuse multiple input features of different dimensions, generates high-dimensional coarse-grained features and predicts the position of joints based on the features. S6. Build a human posture decoder based on sub-pixel classification. Based on the fine-grained high-dimensional features output by S4, use regression analysis to predict the horizontal and vertical coordinate values ​​of each joint of the human body in the image from the fine-grained high-dimensional features. In the model training stage, the output of S6 and the output of S5 together constitute the loss function.

2. A lightweight two-dimensional human posture estimation method suitable for a low computing power platform according to claim 1, characterized in that: In S1, RGB images of the human body at any angle and shape are collected, including single-person scenes, multi-person scenes, motion, stillness or interactive behaviors.

3. The lightweight two-dimensional human posture estimation method suitable for a low computing power platform according to claim 1, characterized in that: In S2, the target detection algorithm is used to obtain the area where each person in the image is located on the collected RGB image, and the content of the area is resized and the image data is normalized.

4. The lightweight two-dimensional human posture estimation method suitable for a low computing power platform according to claim 1, characterized in that: In S3, a lightweight feature extraction module based on group convolution is built. The module is composed of ShuffleNet V2. Assume that the input feature map size is H×W×C in , where H represents the height of the feature map, W represents the width of the feature map, and C in Represents the number of channels of the feature map. Assuming the convolution kernel size is k, the number of channels of the output feature map is C out , when using traditional convolution to process the feature map, the amount of calculation is as follows, H×W×C in ×C out ×k 2 When using depth-wise separable convolution to process the feature map, the depth-wise convolution applies a convolution kernel to each input channel separately, and the computational cost of the depth-wise convolution is: H×W×C in ×k 2 Point-by-point convolution uses a 1×1 convolution kernel to linearly combine all input channels in the spatial dimension, thereby reducing the number of channels of the input feature map from C in Mapping to C out , its computational complexity is: H×W×C in ×C out According to the above formula, the computational complexity of separable convolution is: H×W×C in ×k 2 +H×W×C in ×C out 。 5. The lightweight two-dimensional human posture estimation method suitable for a low computing power platform according to claim 1, characterized in that: In S4, a feature fusion module is built. In the CNN-based object detection model, the feature extraction module usually outputs features of different dimensions. The feature pyramid is used to achieve information interaction between features of different dimensions through different feature fusion strategies and output fused features with the same shape as the input features. These features are used for bounding box prediction, category recognition, and confidence reasoning tasks respectively. Assume that the input feature F i Shape H f ×W f ×C f , where H f , W f , C f Representing its height, width and number of channels respectively, the input features are first reduced and increased in dimension through a set of "hourglass" models composed of 1×1 convolutions. In the process of dimensionality reduction, the model learns the contribution of each value in the feature to subsequent processing, and then restores the original shape through dimensionality increase. The features after dimensionality increase will pass through the Sigmoid activation function to generate a weight tensor W i , whose shape is similar to the input feature F i The same, and the size of each element is in the interval [-1,1], and each value in the tensor represents the importance of different channels; as shown below: F o =F i +W i ⊙F i The channel attention mechanism adjusts the output features F of each channel by multiplying each value in the input feature with the value of the corresponding position in the weight tensor and then adding them together. o , where ⊙ represents W i With F i Multiply the values ​​of the corresponding dimensions in W; i The values ​​in all belong to the interval [-1,1], so F i The values ​​that contribute more to posture estimation will be amplified by 1 to 2 times after the channel attention mechanism.

6. The lightweight two-dimensional human posture estimation method suitable for a low computing power platform according to claim 1, characterized in that: In S5, a training guidance module is built. The module takes the output of the feature extraction module and the feature fusion module as input, and uses the channel attention mechanism to fuse multiple input features to generate coarse-grained high-dimensional features, and predicts the position of the joint points based on the features. In the early stage of training, the training guidance module not only updates its own parameters, but also provides additional gradient feedback to the feature extraction module and the feature fusion module. By providing more global feature information, it helps the backbone network to learn quickly and accelerate convergence in the early stage. In the middle stage of training, since the backbone network has learned a better feature expression method, the training guidance module no longer returns the gradient to the backbone network, but only updates its own parameters. In the late stage of training, the training module is guided to start returning the gradient to the backbone network again. During the entire training process, the training module is guided to continuously participate in the calculation of the loss function.

7. The lightweight two-dimensional human posture estimation method suitable for a low computing power platform according to claim 1, characterized in that: In S6, a human posture decoder based on sub-pixel classification is built, and the predicted value of each joint point is calculated using log-softmax. The following formula expresses the processing process of the predicted values ​​related to all x coordinates, P x,i represents the predicted value of the x-coordinate of the i-th dimension, where i = 1, 2, ..., D, and D represents the dimension of each predicted value. The KL divergence (Kullback-Leibler Divergence) loss function is used to calculate the loss value of the model. The calculation method is as follows, where G i represents the true label of the joint point, P i Represents the probability distribution of the predicted value of the joint point: For joint point j, the loss of calculating x-coordinate and y-coordinate is: Where B is the batch size, w b,j is the predefined weight of each joint point. At the same time, in order to ensure the consistency of the model throughout the training process, the prediction results of the guided training module are also added to the calculation of the loss function. The final loss function is: N represents the number of human joints, loss p,j and loss t,j They represent the loss value of the prediction result of the backbone network at the jth joint point and the loss value of the prediction result of the guided training module respectively; w p and w t They respectively represent the weight of the loss value of the backbone network prediction result and the weight corresponding to the loss value of the guided training module; in the design, two 1×1 convolutional layers are used to directly predict the horizontal and vertical coordinates of the human joint points from the output of the feature fusion module.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the lightweight two-dimensional human body posture estimation method suitable for low computing power platforms as described in any one of claims 1 to 7 above is implemented.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instruction is executed by the processor, a lightweight two-dimensional human body posture estimation algorithm suitable for a low computing power platform is implemented as described in any one of claims 1 to 7.

Images