Human body posture estimation method and device, readable storage medium and program product
By using a pre-trained teacher model to guide the initial student model through linear decreasing and self-guided loss optimization, the problems of background noise and model redundancy in human pose estimation are solved, achieving more efficient and accurate pose estimation.
Patent Information
- Application Number
- CN202511236293.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-01
- Publication Date
- 2025-11-21
AI Technical Summary
Existing human pose estimation techniques suffer from problems such as significant background noise and redundant model parameters, as well as insufficient computational efficiency and prediction accuracy.
A pre-trained teacher model is used to guide the initial student model in linearly decreasing parameter optimization. A stepwise self-guided strategy and multiple intermediate models are combined to optimize the self-guided loss and form the target model, thereby improving the computational efficiency and prediction accuracy of human pose estimation.
By optimizing linearly decreasing and self-guided loss, the computational efficiency and prediction accuracy of human pose estimation are significantly improved, and the inference speed is increased.
Smart Images
Figure CN120997908A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer vision technology, and in particular to a human pose estimation method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology
[0002] With the development of computer vision technology, human pose estimation technology has emerged. Human pose estimation refers to the task of annotating key points of the torso, hands, face, and feet of a target person in an image. It is a crucial part of the field of computer vision, and the accurate localization of key points provides indispensable support for applications such as 3D human reconstruction, face recognition, and motion capture. However, current human pose estimation techniques can be divided into two types according to model structure: top-down and bottom-up. Bottom-up methods directly feed the original 2D image into a pre-designed model to obtain the result, but this method is greatly affected by background noise. On the other hand, top-down methods first perform coarse localization of all people in the image and fine localization of different parts, using different networks to address the differences in different body parts. However, this method has the limitation of generating a large amount of redundant model parameters and is also affected by data limitations. Summary of the Invention
[0003] Based on this, this application provides a human pose estimation method, apparatus, computer device, computer-readable storage medium, and computer program product, which can improve the computational efficiency and prediction accuracy of human pose estimation, and can significantly improve the inference speed in practical applications.
[0004] On the one hand, this application provides a human pose estimation method, including:
[0005] Obtain a set of human body images;
[0006] Human pose estimation is performed on the human image set based on the pre-trained teacher model and the initial student model to obtain teacher pose estimation data and student pose estimation data.
[0007] Based on the teacher posture estimation data, the student posture estimation data, and the preset linear decreasing strategy, the initial student model is optimized by linear decreasing parameters to obtain a candidate student model.
[0008] The target model is obtained by optimizing the self-guided loss based on the preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models; wherein, the intermediate model is an untrained model with the same student parameters as the initial student model.
[0009] The pose estimation results are obtained by performing pose estimation on the human image set based on the target model.
[0010] In one embodiment, the teacher pose estimation data includes: teacher feature vector data and teacher predicted values; the student pose estimation data includes: student feature vector data and student predicted values; the step of performing human pose estimation on the human image set based on the pre-trained teacher model and the initial student model to obtain teacher pose estimation data and student pose estimation data includes:
[0011] Based on the teacher model and the initial student model, feature extraction is performed on the human image set to obtain the teacher feature vector data and the student feature vector data;
[0012] Based on the teacher model, pose prediction without activation function is performed on the teacher feature vector data to obtain the teacher prediction value;
[0013] The student's predicted value is obtained by performing pose prediction on the student's feature vector data without activation function based on the student model.
[0014] In one embodiment, the step of optimizing the initial student model by linearly decreasing parameters based on the teacher posture estimation data, the student posture estimation data, and a preset linear decreasing strategy to obtain a candidate student model includes:
[0015] Loss calculations are performed on the teacher posture estimation data and the student posture estimation data to obtain posture estimation loss data;
[0016] A loss weight model is constructed based on the preset linear decreasing strategy to obtain a loss weight decreasing model; wherein, the loss weight decreasing model is a model that decreases linearly as the number of times the linear decreasing parameter is optimized increases;
[0017] A first-stage distillation loss model is constructed based on the loss weight decreasing model and the attitude estimation loss data.
[0018] The parameters of the initial student model are optimized based on the distillation loss in the first stage to obtain the candidate student model.
[0019] In one embodiment, the teacher pose estimation data includes: teacher feature vector data and teacher predicted values; the student pose estimation data includes: student feature vector data and student predicted values; the step of performing loss calculation on the teacher pose estimation data and the student pose estimation data to obtain pose estimation loss data includes:
[0020] Loss calculation is performed on the teacher feature vector data and the student feature vector data to obtain the feature loss data of the pose estimation loss data;
[0021] Loss calculations are performed on the teacher's predicted values and the student's predicted values to obtain the predicted loss data of the attitude estimation loss data.
[0022] In one embodiment, the step of optimizing the self-guided loss based on a preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models to obtain the target model includes:
[0023] Based on the candidate student model and the pre-trained initialization parameters, a teacher model is constructed to obtain a candidate teacher model;
[0024] A self-guided distillation process is performed based on the candidate teacher model and multiple intermediate models to obtain an updated student model.
[0025] The candidate teacher model is updated to the updated student model, and the self-guided distillation step is repeated until the updated student model meets the preset conditions, and the updated student model is used as the target model; wherein, the preset conditions are determined by the preset stepwise self-guided strategy and the predicted value distillation loss of the self-guided distillation step.
[0026] In one embodiment, determining the preset conditions based on the preset step-by-step self-guided strategy and the predicted distillation loss of the self-guided distillation step includes:
[0027] In the self-guided distillation step, the distillation loss of the predicted values of the candidate teacher model and the intermediate model for the human image set is calculated;
[0028] A self-guiding control factor is determined based on the stepwise self-guiding strategy; wherein the self-guiding control factor is used to gradually reduce the influence of the candidate teacher model on the intermediate model as the number of self-guiding distillation steps increases;
[0029] A self-guided distillation loss model is constructed based on the self-guided control factor and the predicted distillation loss.
[0030] The preset conditions are determined based on the self-guided distillation loss model.
[0031] In one embodiment, after performing pose estimation on the human image set according to the target model to obtain the human pose estimation result, the method further includes:
[0032] Obtain the pose application data of the human image set;
[0033] Based on the human pose estimation results, pose data fusion is performed on the pose application data to obtain application image data;
[0034] The human pose estimation result is verified based on the applied image data to obtain the pose verification result.
[0035] On the one hand, this application also provides a human posture estimation device, including:
[0036] The acquisition module is used to acquire a set of human body images;
[0037] The first pose estimation module is used to perform human pose estimation on the human image set based on the pre-trained teacher model and the initial student model, so as to obtain teacher pose estimation data and student pose estimation data.
[0038] The linear decreasing module is used to optimize the initial student model by linearly decreasing parameters based on the teacher posture estimation data, the student posture estimation data and the preset linear decreasing strategy, so as to obtain a candidate student model.
[0039] The self-guiding module is used to optimize the self-guiding loss according to a preset stepwise self-guiding strategy, the candidate student model, and multiple intermediate models to obtain the target model; wherein, the intermediate model is an untrained model with the same student parameters as the initial student model;
[0040] The second pose estimation module is used to perform pose estimation on the human body image set based on the target model to obtain the human body pose estimation result.
[0041] On the one hand, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0042] Obtain a set of human body images;
[0043] Human pose estimation is performed on the human image set based on the pre-trained teacher model and the initial student model to obtain teacher pose estimation data and student pose estimation data.
[0044] Based on the teacher posture estimation data, the student posture estimation data, and the preset linear decreasing strategy, the initial student model is optimized by linear decreasing parameters to obtain a candidate student model.
[0045] The target model is obtained by optimizing the self-guided loss based on the preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models; wherein, the intermediate model is an untrained model with the same student parameters as the initial student model.
[0046] The pose estimation results are obtained by performing pose estimation on the human image set based on the target model.
[0047] On the one hand, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0048] Obtain a set of human body images;
[0049] Human pose estimation is performed on the human image set based on the pre-trained teacher model and the initial student model to obtain teacher pose estimation data and student pose estimation data.
[0050] Based on the teacher posture estimation data, the student posture estimation data, and the preset linear decreasing strategy, the initial student model is optimized by linear decreasing parameters to obtain a candidate student model.
[0051] The target model is obtained by optimizing the self-guided loss based on the preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models; wherein, the intermediate model is an untrained model with the same student parameters as the initial student model.
[0052] The pose estimation results are obtained by performing pose estimation on the human image set based on the target model.
[0053] On the one hand, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:
[0054] Obtain a set of human body images;
[0055] Human pose estimation is performed on the human image set based on the pre-trained teacher model and the initial student model to obtain teacher pose estimation data and student pose estimation data.
[0056] Based on the teacher posture estimation data, the student posture estimation data, and the preset linear decreasing strategy, the initial student model is optimized by linear decreasing parameters to obtain a candidate student model.
[0057] The target model is obtained by optimizing the self-guided loss based on the preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models; wherein, the intermediate model is an untrained model with the same student parameters as the initial student model.
[0058] The pose estimation results are obtained by performing pose estimation on the human image set based on the target model.
[0059] The aforementioned human pose estimation method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire a set of human images; perform human pose estimation on the human image set based on a pre-trained teacher model and an initial student model to obtain teacher pose estimation data and student pose estimation data; optimize the initial student model by linearly decreasing parameters based on the teacher pose estimation data, student pose estimation data, and a preset linear decreasing strategy to obtain a candidate student model; optimize the self-guided loss based on a preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models to obtain a target model; wherein, the intermediate models are untrained models with the same student parameters as the initial student model; and perform pose estimation on the human image set based on the target model to obtain the human pose estimation result. Therefore, by optimizing the initial student model with linearly decreasing parameters based on teacher pose estimation data, student pose estimation data, and a preset linear decreasing strategy, i.e., using a pre-trained teacher model to guide the initial student model to learn from scratch to obtain candidate student models, and then optimizing the self-guided loss based on a preset stepwise self-guided strategy, candidate student models, and multiple intermediate models to obtain the target model, the performance of human pose estimation is further improved by performing stepwise self-guided optimization on the trained candidate student model and multiple intermediate models. This improves the computational efficiency and prediction accuracy of human pose estimation and significantly increases inference speed in practical applications. Attached Figure Description
[0060] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0061] Figure 1 This is a diagram illustrating the application environment of a human pose estimation method in one embodiment.
[0062] Figure 2 This is a flowchart illustrating a human pose estimation method in one embodiment;
[0063] Figure 3 This is a flowchart illustrating a human pose estimation method in one embodiment;
[0064] Figure 4 This is a flowchart illustrating a human pose estimation method in another embodiment;
[0065] Figure 5 This is a flowchart illustrating a human pose estimation method in another embodiment;
[0066] Figure 6This is a structural block diagram of a human pose estimation device in one embodiment;
[0067] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0068] To make the objectives, technical solutions, and beneficial effects of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0069] The human pose estimation method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 acquires a set of human images; performs human pose estimation on the human images based on a pre-trained teacher model and an initial student model, obtaining teacher pose estimation data and student pose estimation data; optimizes the initial student model's parameters using a linear decreasing strategy based on the teacher pose estimation data, student pose estimation data, and a preset linear decreasing strategy, obtaining a candidate student model; optimizes the self-guided loss using a preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models, obtaining a target model; wherein, the intermediate models are untrained models with the same student parameters as the initial student model; and performs pose estimation on the human images based on the target model, obtaining the human pose estimation result.
[0070] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.
[0071] In one exemplary embodiment, such as Figure 2 As shown, a human pose estimation method is provided, which is then applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 202 to 210. Wherein:
[0072] Step 202: Obtain the human body image set.
[0073] Among them, the human body image set is a collection of images containing the target human body.
[0074] In some embodiments, the human image set can be obtained directly from a human image database, or by acquiring video footage in real time and converting video frames into human images, and is not limited to these methods.
[0075] Step 204: Perform human pose estimation on the human image set based on the pre-trained teacher model and the initial student model to obtain teacher pose estimation data and student pose estimation data.
[0076] The pre-trained teacher model is used for human pose estimation. This teacher model is a fixed-parameter teacher model obtained by optimizing parameters and training fully on a human image training set. The initial student model is used for human pose estimation. This initial student model is obtained by constructing the same network structure as the teacher model, thus obtaining an unoptimized initial student model.
[0077] In some embodiments, teacher pose estimation data includes: teacher feature vector data and teacher predicted values; student pose estimation data includes: student feature vector data and student predicted values; performing human pose estimation on a human image set based on a pre-trained teacher model and an initial student model to obtain teacher pose estimation data and student pose estimation data includes: extracting features from the human image set based on the teacher model and the initial student model to obtain teacher feature vector data and student feature vector data; performing pose prediction on the teacher feature vector data without activation function based on the teacher model to obtain teacher predicted values; and performing pose prediction on the student feature vector data without activation function based on the student model to obtain student predicted values.
[0078] In some embodiments, a set of human images containing the target human body is input into a pre-trained teacher model for human pose estimation to obtain teacher pose estimation data. The teacher model then extracts features from the human image set to obtain teacher feature vector data. Then, based on the teacher model, pose prediction is performed on the teacher feature vector data without activation function to obtain the teacher prediction value without activation function. .
[0079] In some embodiments, a set of human images containing the target human body is input into an initial student model for human pose estimation to obtain student pose estimation data. The initial student model then extracts features from the human image set to obtain student feature vector data. Then, based on the initial student model, pose prediction is performed on the student feature vector data without activation function to obtain the student prediction value without activation function. .
[0080] Step 206: Based on the teacher posture estimation data, student posture estimation data and the preset linear decreasing strategy, optimize the initial student model by linearly decreasing parameters to obtain the candidate student model.
[0081] Among them, the preset linear decreasing strategy is a knowledge distillation strategy. This strategy gradually reduces the weight of distillation loss by using a linear decreasing method. The optimization of the linear decreasing parameter is a knowledge distillation process.
[0082] In some embodiments, the initial student model is optimized by linearly decreasing parameters based on teacher posture estimation data, student posture estimation data, and a preset linear decreasing strategy to obtain a candidate student model. This includes: calculating the loss of the teacher posture estimation data and student posture estimation data to obtain posture estimation loss data; constructing a loss weight model based on the preset linear decreasing strategy to obtain a loss weight decreasing model; wherein the loss weight decreasing model is a model that decreases linearly with the number of linear decreasing parameter optimizations; constructing a first-stage distillation loss model based on the loss weight decreasing model and posture estimation loss data; and optimizing the parameters of the initial student model based on the first-stage distillation loss to obtain a candidate student model.
[0083] In some embodiments, teacher pose estimation data includes: teacher feature vector data and teacher predicted values; student pose estimation data includes: student feature vector data and student predicted values; loss calculation is performed on the teacher pose estimation data and student pose estimation data to obtain pose estimation loss data, including: loss calculation is performed on the teacher feature vector data and student feature vector data to obtain feature loss data of pose estimation loss data; loss calculation is performed on the teacher predicted values and student predicted values to obtain predicted value loss data of pose estimation loss data.
[0084] In some embodiments, loss calculation is performed on teacher feature vector data and student feature vector data, that is, the distillation loss between the output of the teacher model and the initial student model is calculated from the perspective of features. Attention-aware feature distillation loss is used to measure the loss between teacher feature vector data and student feature vector data, resulting in feature loss data of pose estimation loss data. The specific calculation formula is shown in formula (1) below:
[0085] (1)
[0086] Where C, H, and W represent the number of channels, height, and width of the input image, respectively. It is a single-layer convolutional network used to process student feature vector data. Teacher feature vector data Alignment Attention heatmap explicitly generated from the teacher model.
[0087] In some embodiments, loss calculation is performed on the teacher's and student's predicted values. Specifically, the distillation loss between the teacher model and the initial student model outputs is calculated from the perspective of the predicted values. The continuous coordinates of the x and y axes are discretized into L integer labels. This process follows the SimCC algorithm, and KL divergence is used to measure the loss between the integer labels corresponding to the teacher's and student's predicted values, thus obtaining the predicted value loss data for the pose estimation loss data. The specific calculation formula is shown in formula (2) below:
[0088] (2)
[0089] Where N represents the number of human samples in a batch of images, K represents the number of keypoints, and L is the number of integer labels.
[0090] In some embodiments, a loss weight model is constructed according to a preset linear decreasing strategy to obtain a loss weight decreasing model; wherein, the loss weight decreasing model is a model that decreases linearly with the number of optimizations of the linear decreasing parameters, that is, the weight of distillation loss is gradually reduced using a linear decreasing strategy, and the loss weight decreasing model is defined based on this strategy. As shown in formula (3) below:
[0091] (3)
[0092] Where t represents the current iteration number, This represents the total number of iterations required in the entire linearly decreasing parameter optimization phase. It is a function that changes dynamically with training stability. The specific function formula is shown in formula (4) below:
[0093] (4)
[0094] in, This represents the variance of the loss from step tk to step t. This strategy allows for a slower rate of decay when there are large fluctuations in the early stages of training.
[0095] In some embodiments, a first-stage distillation loss model is constructed based on the loss weight reduction model and attitude estimation loss data, combined with the loss weight reduction model. and attitude estimation loss data and The first-stage distillation loss model is obtained as shown in the following formula (5). :
[0096] (5)
[0097] Among them, the hyperparameters and are predefined to measure the contributions of the feature vector-based distillation loss and the prediction-based distillation loss to the optimization parameters, respectively.
[0098] In some embodiments, the parameters of the initial student model are optimized based on the first-stage distillation loss. During the parameter optimization process, when the first-stage distillation loss is the minimum in all iterations, or when the change in the first-stage distillation loss error is less than a preset condition, the initial student model trained at this time is used as a candidate student model.
[0099] Step 208: Optimize the self-guiding loss based on the preset stepwise self-guiding strategy, candidate student models, and multiple intermediate models to obtain the target model.
[0100] The intermediate model is an untrained model with the same student parameters as the initial student model. The preset stepwise self-guided strategy is a self-guided knowledge distillation strategy that gradually reduces the influence of the teacher model on the student model by introducing a control factor.
[0101] In some embodiments, the self-guided loss optimization is performed based on a preset stepwise self-guided strategy, a candidate student model, and multiple intermediate models to obtain a target model, including: constructing a teacher model based on the candidate student model and pre-trained initialization parameters to obtain a candidate teacher model; performing self-guided distillation based on the candidate teacher model and multiple intermediate models to obtain an updated student model; updating the candidate teacher model to the updated student model, and repeating the self-guided distillation step until the updated student model meets a preset condition, and using the updated student model as the target model; wherein, the preset condition is determined by the preset stepwise self-guided strategy and the predicted value distillation loss of the self-guided distillation step.
[0102] Among them, the pre-trained initialization parameters are used as the initialization parameters of the backbone network and head network of the candidate teacher model, and self-guided distillation is a parameter optimization method that performs stepwise self-guided knowledge distillation on the pre-trained candidate student model.
[0103] In some embodiments, the candidate student model is used as the candidate teacher model. The backbone network and head network of the candidate teacher model are initialized with pre-trained parameters. In addition, multiple student models with the same parameters as the initial student model are constructed, and their head networks are replaced with untrained head networks to obtain multiple intermediate models. Self-guided distillation is performed based on the candidate teacher model and multiple intermediate models to obtain the updated student model.
[0104] In some embodiments, the candidate teacher model is updated to the updated student model, and the self-guided distillation step is repeated until the updated student model meets the preset conditions, that is, the updated student model in the current iteration has met the model convergence condition, and then the updated student model is used as the target model; wherein, the preset conditions are determined by the preset stepwise self-guided strategy and the predicted value distillation loss of the self-guided distillation step.
[0105] In some embodiments, determining preset conditions based on a preset stepwise self-guiding strategy and the predicted value distillation loss of the self-guiding distillation steps includes: in the self-guiding distillation step, calculating the predicted value distillation loss of the candidate teacher model and the intermediate model for the human image set; determining a self-guiding control factor based on the stepwise self-guiding strategy; wherein the self-guiding control factor is used to gradually reduce the influence of the candidate teacher model on the intermediate model as the number of self-guiding distillation steps increases; constructing a self-guiding distillation loss model based on the self-guiding control factor and the predicted value distillation loss; and determining preset conditions based on the self-guiding distillation loss model.
[0106] In some embodiments, the self-guided distillation step is performed using formula (2) from the above embodiments. The distillation loss is calculated by using the method of candidate teacher model and intermediate model to predict the human image set. Among them, the predicted distillation loss It will change with the number of self-guided distillation cycles t.
[0107] In some embodiments, a self-guiding control factor is determined according to a stepwise self-guiding strategy; wherein, the self-guiding control factor is used to gradually reduce the influence of the candidate teacher model on the intermediate model as the number of self-guiding distillation steps increases, specifically, the self-guiding control factor in the t-th round. It can be expressed as the following formula (6):
[0108] (6)
[0109] Where T represents the total number of training rounds required for the entire training phase. This represents the control factor for the last training round.
[0110] First, a self-guided loss function is constructed based on the self-guided control factor and the predicted distillation loss. It can be expressed as the following formula (7):
[0111] (7)
[0112] Then based on the self-guided loss function and predicted distillation loss They are added together to the self-guided distillation loss model, and the preset conditions are determined based on the self-guided distillation loss model.
[0113] Step 210: Perform pose estimation on the human body image set based on the target model to obtain the human body pose estimation result.
[0114] Pose estimation is used to accurately estimate the pose information of the target human body in the current human body image set.
[0115] In some embodiments, pose estimation of a human image set is performed based on a target model. By leveraging the high computational efficiency and high prediction accuracy of the target model, inference speed in pose estimation can be significantly improved.
[0116] In the above-mentioned human pose estimation method, a human image set is acquired; human pose estimation is performed on the human image set based on a pre-trained teacher model and an initial student model to obtain teacher pose estimation data and student pose estimation data; the initial student model is optimized by linearly decreasing parameters based on the teacher pose estimation data, student pose estimation data, and a preset linear decreasing strategy to obtain a candidate student model; the self-guided loss is optimized based on a preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models to obtain the target model; wherein, the intermediate model is an untrained model with the same student parameters as the initial student model; the human pose is estimated based on the target model to obtain the human pose estimation result. Therefore, by optimizing the initial student model with linearly decreasing parameters based on teacher pose estimation data, student pose estimation data, and a preset linear decreasing strategy, i.e., using a pre-trained teacher model to guide the initial student model to learn from scratch to obtain candidate student models, and then optimizing the self-guided loss based on a preset stepwise self-guided strategy, candidate student models, and multiple intermediate models to obtain the target model, the performance of human pose estimation is further improved by performing stepwise self-guided optimization on the trained candidate student model and multiple intermediate models. This improves the computational efficiency and prediction accuracy of human pose estimation and significantly increases inference speed in practical applications.
[0117] In one exemplary embodiment, such as Figure 3 As shown, after estimating the pose of the human image set based on the target model and obtaining the human pose estimation result, the human pose estimation method further includes steps 302 to 306. Wherein:
[0118] Step 302: Obtain pose application data from the human image set.
[0119] Among them, posture application data refers to application data used in a specific field to combine the human posture estimation results corresponding to the human image set to achieve the specific application requirements in the corresponding field. Posture application data can be application data in fields such as human 3D reconstruction, face recognition, and motion capture.
[0120] In some embodiments, posture application data of a human image set can be obtained through direct user input, or posture application data of a human image set can be generated based on user input application requirements, and this is not limited to these methods.
[0121] Step 304: Based on the human pose estimation results, perform pose data fusion on the pose application data to obtain the application image data.
[0122] Among them, posture data fusion is the process of fusing posture application data with human posture estimation results to form application result data for the corresponding application domain.
[0123] In some embodiments, posture data fusion is performed on posture application data based on human posture estimation results to obtain application image data of the target human posture in the posture application data domain.
[0124] Step 306: Perform pose verification on the human pose estimation results based on the applied image data to obtain the pose verification results.
[0125] Among them, pose verification is used to verify the accuracy of human pose estimation results.
[0126] In some embodiments, the human pose estimation results are evaluated by applying the pose effect of the image in the image data, and the evaluation results are mathematically calculated to obtain a comprehensive evaluation result. Finally, the human pose estimation results are verified by the comprehensive evaluation result to obtain the pose verification result.
[0127] In this embodiment, by acquiring posture application data from a human image set, posture data fusion is performed on the posture application data based on the human posture estimation results to obtain application image data. The human posture estimation results are then verified based on the application image data to obtain posture verification results. This allows for detailed posture verification of the posture application performance of the human posture estimation results in various application fields, thereby obtaining accurate posture verification results. Users can further fine-tune and train the target model based on the posture verification results to further improve the accuracy of the target model.
[0128] To gain a clearer understanding of this application, this document combines... Figure 4 and 5 The explanation is as follows:
[0129] First, such as Figure 4 As shown, Figure 4 This paper demonstrates the process of linearly decreasing parameter optimization. A set of human images is acquired and used as input to the teacher model and the initial student model. Human pose estimation is then performed using the backbone and head networks of the teacher and initial student models, respectively, yielding teacher and student pose estimation data. During pose estimation, feature vector structure distillation and predicted value logical distillation are performed on the teacher and student pose estimation data according to the iteration number t and a preset linear decreasing strategy. Knowledge distillation is also performed on the unfixed backbone and head networks in the student model to achieve linearly decreasing parameter optimization, thus obtaining candidate student models.
[0130] Secondly, such as Figure 5 As shown, Figure 5 This paper demonstrates the process of optimizing self-guided loss. A teacher model is constructed based on the candidate student model and pre-trained initial parameters to obtain the candidate teacher model. Human images are used as input, and human pose estimation is performed through the backbone network and head network of the candidate teacher model and intermediate models, respectively. Then, according to the preset stepwise self-guided strategy, the candidate student model and multiple intermediate models are used to perform logical distillation of the predicted values to achieve self-guided loss optimization. The parameters of the head network in the intermediate models are then optimized to obtain the target model. The intermediate models are untrained models with the same student parameters as the initial student model.
[0131] Finally, the pose estimation of the human image set can be performed based on the target model to obtain the human pose estimation result, which can be used to verify the pose estimation accuracy of the target model. The target model can also be further fine-tuned based on the human pose estimation result.
[0132] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0133] Based on the same inventive concept, this application also provides a human pose estimation device for implementing the human pose estimation method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more human pose estimation device embodiments provided below can be found in the limitations of the human pose estimation method described above, and will not be repeated here.
[0134] In one exemplary embodiment, such as Figure 6 As shown, a human pose estimation device is provided, including: an acquisition module 602, a first pose estimation module 604, a linear decreasing module 606, a self-guiding module 608, and a second pose estimation module 610, wherein:
[0135] Module 602 is used to acquire a set of human body images;
[0136] The first pose estimation module 604 is used to perform human pose estimation on the human image set based on the pre-trained teacher model and the initial student model, so as to obtain teacher pose estimation data and student pose estimation data.
[0137] The linear decreasing module 606 is used to optimize the initial student model by linear decreasing parameters based on teacher posture estimation data, student posture estimation data and preset linear decreasing strategy to obtain candidate student models.
[0138] The self-guiding module 608 is used to optimize the self-guiding loss according to the preset stepwise self-guiding strategy, candidate student models and multiple intermediate models to obtain the target model; wherein, the intermediate models are untrained models with the same student parameters as the initial student model.
[0139] The second pose estimation module 610 is used to estimate the pose of the human body image set based on the target model and obtain the human body pose estimation result.
[0140] In some embodiments, the teacher pose estimation data includes: teacher feature vector data and teacher predicted values; the student pose estimation data includes: student feature vector data and student predicted values; the first pose estimation module 604 is further configured to extract features from the human image set according to the teacher model and the initial student model to obtain teacher feature vector data and student feature vector data; perform pose prediction on the teacher feature vector data without activation function according to the teacher model to obtain teacher predicted values; and perform pose prediction on the student feature vector data without activation function according to the student model to obtain student predicted values.
[0141] In some embodiments, the linear decreasing module 606 is further configured to perform loss calculation on teacher posture estimation data and student posture estimation data to obtain posture estimation loss data; construct a loss weight model according to a preset linear decreasing strategy to obtain a loss weight decreasing model; wherein, the loss weight decreasing model is a model that decreases linearly with the number of times the linear decreasing parameters are optimized; construct a first-stage distillation loss model based on the loss weight decreasing model and posture estimation loss data; and optimize the parameters of the initial student model based on the first-stage distillation loss to obtain a candidate student model.
[0142] In some embodiments, the teacher pose estimation data includes: teacher feature vector data and teacher predicted values; the student pose estimation data includes: student feature vector data and student predicted values; the linear decreasing module 606 is further configured to perform loss calculation on the teacher feature vector data and student feature vector data to obtain feature loss data of pose estimation loss data; and to perform loss calculation on the teacher predicted values and student predicted values to obtain predicted value loss data of pose estimation loss data.
[0143] In some embodiments, the self-guiding module 608 is further configured to construct a teacher model based on the candidate student model and pre-trained initialization parameters to obtain a candidate teacher model; perform self-guiding distillation based on the candidate teacher model and multiple intermediate models to obtain an updated student model; update the candidate teacher model to the updated student model, and repeat the self-guiding distillation step until the updated student model meets a preset condition, and use the updated student model as the target model; wherein, the preset condition is determined by a preset stepwise self-guiding strategy and the predicted value distillation loss of the self-guiding distillation step.
[0144] In some embodiments, the self-guiding module 608 is further configured to, in the self-guiding distillation step, calculate the distillation loss of the predicted values of the candidate teacher model and the intermediate model for the human image set; determine a self-guiding control factor according to a stepwise self-guiding strategy; wherein the self-guiding control factor is used to gradually reduce the influence of the candidate teacher model on the intermediate model as the number of self-guiding distillation steps increases; construct a self-guiding distillation loss model based on the self-guiding control factor and the predicted value distillation loss; and determine preset conditions based on the self-guiding distillation loss model.
[0145] In some embodiments, after performing pose estimation on the human image set according to the target model to obtain the human pose estimation result, the device further includes: a verification module, configured to acquire pose application data of the human image set; perform pose data fusion on the pose application data according to the human pose estimation result to obtain application image data; and perform pose verification on the human pose estimation result according to the application image data to obtain a pose verification result.
[0146] In the aforementioned human pose estimation device, a human image set is acquired; human pose estimation is performed on the human image set based on a pre-trained teacher model and an initial student model to obtain teacher pose estimation data and student pose estimation data; the initial student model is optimized by linearly decreasing parameters based on the teacher pose estimation data, student pose estimation data, and a preset linear decreasing strategy to obtain a candidate student model; a self-guided loss optimization is performed based on a preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models to obtain a target model; wherein, the intermediate model is an untrained model with the same student parameters as the initial student model; and the human pose is estimated based on the target model to obtain the human pose estimation result. Therefore, by optimizing the initial student model with linearly decreasing parameters based on teacher pose estimation data, student pose estimation data, and a preset linear decreasing strategy, i.e., using a pre-trained teacher model to guide the initial student model to learn from scratch to obtain candidate student models, and then optimizing the self-guided loss based on a preset stepwise self-guided strategy, candidate student models, and multiple intermediate models to obtain the target model, the performance of human pose estimation is further improved by performing stepwise self-guided optimization on the trained candidate student model and multiple intermediate models. This improves the computational efficiency and prediction accuracy of human pose estimation and significantly increases inference speed in practical applications.
[0147] The modules in the aforementioned human pose estimation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.
[0148] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores a set of human images. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a human pose estimation method.
[0149] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0150] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.
[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.
[0152] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0154] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0155] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0156] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for estimating human pose, characterized in that, The method includes: Obtain a set of human body images; Human pose estimation is performed on the human image set based on the pre-trained teacher model and the initial student model to obtain teacher pose estimation data and student pose estimation data. Based on the teacher posture estimation data, the student posture estimation data, and the preset linear decreasing strategy, the initial student model is optimized by linear decreasing parameters to obtain a candidate student model. The target model is obtained by optimizing the self-guided loss based on the preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models; wherein, the intermediate model is an untrained model with the same student parameters as the initial student model. The pose estimation results are obtained by performing pose estimation on the human image set based on the target model.
2. The method according to claim 1, characterized in that, The teacher pose estimation data includes: teacher feature vector data and teacher predicted values; the student pose estimation data includes: student feature vector data and student predicted values; the process of performing human pose estimation on the human image set based on the pre-trained teacher model and the initial student model to obtain teacher pose estimation data and student pose estimation data includes: Based on the teacher model and the initial student model, feature extraction is performed on the human image set to obtain the teacher feature vector data and the student feature vector data; Based on the teacher model, pose prediction without activation function is performed on the teacher feature vector data to obtain the teacher prediction value; The student's predicted value is obtained by performing pose prediction on the student's feature vector data without activation function based on the student model.
3. The method according to claim 1, characterized in that, The step of optimizing the initial student model using linearly decreasing parameters based on the teacher posture estimation data, the student posture estimation data, and a preset linear decreasing strategy to obtain a candidate student model includes: Loss calculations are performed on the teacher posture estimation data and the student posture estimation data to obtain posture estimation loss data; A loss weight model is constructed based on the preset linear decreasing strategy to obtain a loss weight decreasing model; wherein, the loss weight decreasing model is a model that decreases linearly as the number of times the linear decreasing parameter is optimized increases; A first-stage distillation loss model is constructed based on the loss weight decreasing model and the attitude estimation loss data. The parameters of the initial student model are optimized based on the distillation loss in the first stage to obtain the candidate student model.
4. The method according to claim 3, characterized in that, The teacher posture estimation data includes: teacher feature vector data and teacher predicted values; the student posture estimation data includes: student feature vector data and student predicted values; the loss calculation of the teacher posture estimation data and the student posture estimation data to obtain posture estimation loss data includes: Loss calculation is performed on the teacher feature vector data and the student feature vector data to obtain the feature loss data of the pose estimation loss data; Loss calculations are performed on the teacher's predicted values and the student's predicted values to obtain the predicted loss data of the attitude estimation loss data.
5. The method according to claim 1, characterized in that, The step of optimizing the self-guided loss based on a preset stepwise self-guided strategy, the candidate student model, and multiple intermediate models to obtain the target model includes: Based on the candidate student model and the pre-trained initialization parameters, a teacher model is constructed to obtain a candidate teacher model; A self-guided distillation process is performed based on the candidate teacher model and multiple intermediate models to obtain an updated student model. The candidate teacher model is updated to the updated student model, and the self-guided distillation step is repeated until the updated student model meets the preset conditions, and the updated student model is used as the target model; wherein, the preset conditions are determined by the preset stepwise self-guided strategy and the predicted value distillation loss of the self-guided distillation step.
6. The method according to claim 5, characterized in that, The step of determining the preset conditions based on the preset step-by-step self-guided strategy and the predicted distillation loss of the self-guided distillation step includes: In the self-guided distillation step, the distillation loss of the predicted values of the candidate teacher model and the intermediate model for the human image set is calculated; A self-guiding control factor is determined based on the stepwise self-guiding strategy; wherein the self-guiding control factor is used to gradually reduce the influence of the candidate teacher model on the intermediate model as the number of self-guiding distillation steps increases; A self-guided distillation loss model is constructed based on the self-guided control factor and the predicted distillation loss. The preset conditions are determined based on the self-guided distillation loss model.
7. The method according to claim 1, characterized in that, After performing pose estimation on the human image set based on the target model to obtain the human pose estimation result, the method further includes: Obtain the pose application data of the human image set; Based on the human pose estimation results, pose data fusion is performed on the pose application data to obtain application image data; The human pose estimation result is verified based on the applied image data to obtain the pose verification result.
8. A human posture estimation device, characterized in that, The device includes: The acquisition module is used to acquire a set of human body images; The first pose estimation module is used to perform human pose estimation on the human image set based on the pre-trained teacher model and the initial student model, so as to obtain teacher pose estimation data and student pose estimation data. The linear decreasing module is used to optimize the initial student model by linearly decreasing parameters based on the teacher posture estimation data, the student posture estimation data and the preset linear decreasing strategy, so as to obtain a candidate student model. The self-guiding module is used to optimize the self-guiding loss according to a preset stepwise self-guiding strategy, the candidate student model, and multiple intermediate models to obtain the target model; wherein, the intermediate model is an untrained model with the same student parameters as the initial student model; The second pose estimation module is used to perform pose estimation on the human body image set based on the target model to obtain the human body pose estimation result.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.