Multi-level key point visibility classification algorithm for human body posture estimation
By introducing a multi-class visibility prediction head and an adaptive BCE loss function into the RTMpose model, the problem of insufficient accuracy of traditional pose estimation methods in complex occlusion scenarios is solved, and more accurate keypoint visibility judgment and pose estimation are achieved.
Patent Information
- Application Number
- CN202511313625.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-15
- Publication Date
- 2026-02-24
AI Technical Summary
Traditional pose estimation methods struggle to accurately determine the visibility of key points in complex occlusion and partial visibility scenarios, leading to decreased pose estimation accuracy, especially in multi-target occlusion and complex scenes.
The RTMpose model is used as the base model, and a multi-class visibility prediction head (VH module) is introduced. Combined with the multi-class BCE loss function, the visibility of key points is divided into three categories, 0, 0.5 and 1, by using the Sigmoid activation function to optimize the visibility classification of key points and improve the robustness of the model in complex scenes.
It improves the accuracy of keypoint visibility assessment and the robustness of pose estimation, reduces the interference of occluded data on model training, and enhances the overall accuracy of pose estimation.
Smart Images
Figure CN121564754A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of posture estimation technology, specifically to a multi-level keypoint visibility classification algorithm for human posture estimation. Background Technology
[0002] Human pose estimation is one of the core tasks in the field of computer vision. It aims to locate the positions of key joints of the human body (such as head, shoulders, elbows, wrists, hips, knees, ankles, etc.) and infer the overall pose of the human body through image or video data.
[0003] Traditional pose estimation techniques typically rely on predicting the coordinates and corresponding confidence scores of each keypoint to indirectly determine its visibility. Most methods use a binary classification approach (visible / invisible) to handle the visibility problem, that is, using a binary label to represent whether a keypoint is visible. However, this method has obvious shortcomings, especially when dealing with complex scenes (such as occlusion, self-occlusion, or partial occlusion), where it performs poorly. Existing technologies mainly rely on the confidence scores of coordinate predictions to indirectly determine visibility, without directly modeling the visibility type of keypoints, such as fully visible, partially visible, and invisible. In cases of multiple targets or severe occlusion, the model often cannot accurately determine the actual visibility of keypoints, especially in the case of partial occlusion, where partially visible keypoints are easily misclassified as invisible or fully visible, thus affecting the final pose estimation results.
[0004] To address the aforementioned technical challenges, we have designed a multi-level keypoint visibility classification algorithm for human pose estimation. Summary of the Invention
[0005] The purpose of this invention is to provide a multi-level keypoint visibility classification algorithm for human pose estimation. This algorithm can more accurately determine the actual visibility of keypoints, thereby improving the accuracy and robustness of pose estimation. It solves the problem that although traditional methods can predict the location of keypoints, they perform poorly in complex occlusion and partial visibility situations. This can cause the model to be easily interfered with by occlusion data during training, affecting the final accuracy. In particular, when facing multi-target occlusion and complex scenes, traditional methods may not be able to handle occlusion relationships, leading to a decrease in pose estimation accuracy.
[0006] To achieve the above objectives, the present invention provides the following technical solution: a multi-level keypoint visibility classification algorithm for human pose estimation, comprising the following steps:
[0007] Step 1: Prepare a human pose estimation image dataset, which may contain multiple people or a single person, each with corresponding key point annotations;
[0008] Step 2: The RTMpose model is used as the basic model for pose estimation, and a multi-class visibility prediction head (VH module) is introduced on this basis to extract the pose information of the target person from the image;
[0009] Step 3: The multi-class visibility prediction head is achieved by fusing information from the pose feature layer output of RTMpose. Then, the prediction head uses the Sigmoid activation function to classify the visibility of each keypoint into three categories: 0, 0.5, and 1.
[0010] Step 4: Modify the traditional BCE loss function to be suitable for multi-class classification problems. The visibility labels 0, 1 and 2 of each key point correspond to the new target values 0, 0.5 and 1, respectively, and are used to calculate the visibility classification loss of each key point.
[0011] Step 5: Train the model to minimize the overall loss by combining the loss function described above;
[0012] Step 6: After inference and visibility prediction, output the key point coordinates and their visibility labels for each image.
[0013] Preferably, in step 1, the dataset for each image contains the following: each target person has a set of annotations, including the two-dimensional coordinates of its key points, and the visibility label (0, 1, 2) for each key point.
[0014] Preferably, in step 3, 0 represents invisible, 0.5 represents partially visible, and 1 represents fully visible.
[0015] Preferably, in step 3, the prediction head classifies the visibility value of each key point according to its relationship with the refraction interval, outputting category values of 0, 0.5 and 1, and predicts them together with their two-dimensional coordinates.
[0016] Preferably, in step 4, the loss calculation is obtained by converting between three categories: 0, 1, and 2. The categories are achieved by setting the Sigmoid refraction interval [0.3, 0.7].
[0017] Preferably, in step 4, the loss calculation is obtained by converting between three categories: 0, 1, and 2. The categories are achieved by setting the Sigmoid refraction interval [0.3, 0.7].
[0018] Preferably, in step 4, when the label is 1, the specific formula is: .
[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0020] This invention employs adaptive multi-class BCE loss to simultaneously optimize the two-dimensional coordinates and visibility classification of key points. This loss function can handle more detailed visibility classifications (0, 1, 2), thereby better reflecting the actual visibility state of key points in complex scenes during pose estimation. Attached Figure Description
[0021] Figure 1 A flowchart of a multi-level keypoint visibility classification algorithm for human pose estimation;
[0022] Figure 2 This is a diagram illustrating the training method of the present invention;
[0023] Figure 3 This is a diagram of the traditional BCE training method. Detailed Implementation
[0024] Please see Figures 1-3 A multi-level keypoint visibility classification algorithm for human pose estimation includes the following steps:
[0025] Step 1: Prepare a human pose estimation image dataset, which may contain multiple people or a single person, each with corresponding key point annotations;
[0026] Step 2: The RTMpose model is adopted as the basic model for pose estimation. The RTMpose model is an efficient basic model for human pose estimation, which performs well in balancing real-time performance and accuracy. It is suitable as the basic framework of this algorithm. Through the backbone network and feature pyramid structure, it can capture pose details at different resolutions and adapt to the situation of human body size changes in the image. On this basis, a multi-class visibility prediction head (VH module) is introduced. By setting the multi-class visibility prediction head (VH module), the visibility classification precision can be improved, feature utilization can be optimized, and the robustness of the model can be enhanced. In the case of multi-target occlusion or complex background, the visibility of key points can be judged more accurately, the interference of occlusion data on training can be reduced, and the robustness of the model in difficult scenarios such as partial occlusion and severe occlusion can be improved, thereby improving the overall pose estimation accuracy and extracting the pose information of the target person from the image.
[0027] Step 3: The multi-class visibility prediction head is achieved by fusing information from the pose feature layer output of RTMpose. Through RTMpose, multi-scale and multi-level pose features can be extracted, providing a rich feature base for subsequent visibility judgment and coordinate prediction. Then, the prediction head uses the Sigmoid activation function to classify the visibility of each key point into three categories, namely 0, 0.5 and 1.
[0028] Step 4: Modify the traditional BCE loss function to be suitable for multi-class classification problems. The visibility labels 0, 1 and 2 of each key point correspond to the new target values 0, 0.5 and 1, respectively, and are used to calculate the visibility classification loss of each key point.
[0029] Step 5: Train the model to minimize the overall loss by combining the loss function described above;
[0030] Step 6: After inference and visibility prediction, output the key point coordinates and their visibility labels for each image.
[0031] In step 1, the dataset for each image contains the following: each target person has a set of annotations, including the two-dimensional coordinates of their keypoints, and the visibility label (0, 1, 2) for each keypoint.
[0032] In step 3, 0 represents invisible, 0.5 represents partially visible, and 1 represents fully visible.
[0033] In step 3, the prediction head classifies the visibility value of each key point according to its relationship with the refraction interval, outputting category values of 0, 0.5 and 1, and predicts them together with their two-dimensional coordinates.
[0034] In step 4, the loss calculation is obtained through three categories: 0, 1, and 2. The categories are achieved by setting the Sigmoid refraction interval [0.3, 0.7].
[0035] In step 4, when the label is 0 or 2, the specific formula is: Where N is the number of key points. The actual label values are 0 and 1. This represents the probability predicted by the model.
[0036] In step 4, when the label is 1, the specific formula is:
[0037] ,
[0038] Where N is the number of key points. The probability predicted by the model is such that the closer it is to 0.5, the closer the loss is to 0. This is the penalty coefficient for probabilities exceeding 0.7. For safety values.
[0039] Example 1:
[0040] A multi-level keypoint visibility classification algorithm for human pose estimation includes the following steps:
[0041] Step 1: Prepare a human pose estimation image dataset, which may contain multiple people or a single person, each with corresponding key point annotations;
[0042] Step 2: The RTMpose model is adopted as the basic model for pose estimation. The RTMpose model is an efficient basic model for human pose estimation, which performs well in balancing real-time performance and accuracy. It is suitable as the basic framework of this algorithm. Through the backbone network and feature pyramid structure, it can capture pose details at different resolutions and adapt to the situation of human body size changes in the image. On this basis, a multi-class visibility prediction head (VH module) is introduced. By setting the multi-class visibility prediction head (VH module), the visibility classification precision can be improved, feature utilization can be optimized, and the robustness of the model can be enhanced. In the case of multi-target occlusion or complex background, the visibility of key points can be judged more accurately, the interference of occlusion data on training can be reduced, and the robustness of the model in difficult scenarios such as partial occlusion and severe occlusion can be improved, thereby improving the overall pose estimation accuracy and extracting the pose information of the target person from the image.
[0043] Step 3: The multi-class visibility prediction head is achieved by fusing information from the pose feature layer output of RTMpose. Through RTMpose, multi-scale and multi-level pose features can be extracted, providing a rich feature base for subsequent visibility judgment and coordinate prediction. Then, the prediction head uses the Sigmoid activation function to classify the visibility of each key point into three categories, namely 0, 0.5 and 1.
[0044] Step 4: Modify the traditional BCE loss function to be suitable for multi-class classification problems. The visibility labels 0, 1 and 2 of each key point correspond to the new target values 0, 0.5 and 1, respectively, and are used to calculate the visibility classification loss of each key point.
[0045] Step 5: Train the model to minimize the overall loss by combining the loss function described above;
[0046] Step 6: After inference and visibility prediction, output the key point coordinates and their visibility labels for each image.
[0047] In step 4, the loss calculation is obtained through three categories: 0, 1, and 2. The categories are achieved by setting the Sigmoid refraction interval [0.3, 0.7].
[0048] In step 4, when the label is 0 or 2, the specific formula is:
[0049] ,
[0050] Where N is the number of key points. The actual label values are 0 and 1. This represents the probability predicted by the model.
[0051] In step 4, when the label is 1, the specific formula is:
[0052] ,
[0053] Where N is the number of key points. The probability predicted by the model is such that the closer it is to 0.5, the closer the loss is to 0. This is the penalty coefficient for probabilities exceeding 0.7. For safety values.
[0054] Example 2:
[0055] A multi-level keypoint visibility classification algorithm for human pose estimation includes the following steps:
[0056] Step 1: Prepare a human pose estimation image dataset, which may contain multiple people or a single person, each with corresponding key point annotations;
[0057] Step 2: The RTMpose model is adopted as the basic model for pose estimation. The RTMpose model is an efficient basic model for human pose estimation, which performs well in balancing real-time performance and accuracy. It is suitable as the basic framework of this algorithm. Through the backbone network and feature pyramid structure, it can capture pose details at different resolutions and adapt to the situation of human body size changes in the image. On this basis, a multi-class visibility prediction head (VH module) is introduced. By setting the multi-class visibility prediction head (VH module), the visibility classification precision can be improved, feature utilization can be optimized, and the robustness of the model can be enhanced. In the case of multi-target occlusion or complex background, the visibility of key points can be judged more accurately, the interference of occlusion data on training can be reduced, and the robustness of the model in difficult scenarios such as partial occlusion and severe occlusion can be improved, thereby improving the overall pose estimation accuracy and extracting the pose information of the target person from the image.
[0058] Step 3: The multi-class visibility prediction head is achieved by fusing information from the pose feature layer output of RTMpose. Through RTMpose, multi-scale and multi-level pose features can be extracted, providing a rich feature base for subsequent visibility judgment and coordinate prediction. Then, the prediction head uses the Sigmoid activation function to classify the visibility of each key point into three categories, namely 0, 0.5 and 1.
[0059] Step 4: Modify the traditional BCE loss function to be suitable for multi-class classification problems. The visibility labels 0, 1 and 2 of each key point correspond to the new target values 0, 0.5 and 1, respectively, and are used to calculate the visibility classification loss of each key point.
[0060] Step 5: Train the model to minimize the overall loss by combining the loss function described above;
[0061] Step 6: After inference and visibility prediction, output the key point coordinates and their visibility labels for each image.
[0062] In step 1, the dataset for each image contains the following: each target person has a set of annotations, including the two-dimensional coordinates of their keypoints, and the visibility label (0, 1, 2) for each keypoint.
[0063] In step 3, 0 represents invisible, 0.5 represents partially visible, and 1 represents fully visible.
[0064] In step 3, the prediction head classifies the visibility value of each key point according to its relationship with the refraction interval, outputting category values of 0, 0.5 and 1, and predicts them together with their two-dimensional coordinates.
[0065] In summary, this multi-level keypoint visibility classification algorithm for human pose estimation solves the problem that while traditional methods can predict the location of keypoints, they perform poorly in complex occlusion and partial visibility situations. This makes the model susceptible to interference from occlusion data during training, affecting the final accuracy. In particular, when faced with multi-target occlusion and complex scenes, traditional methods may be unable to handle occlusion relationships, leading to a decrease in pose estimation accuracy.
Claims
1. A multi-level keypoint visibility classification algorithm for human pose estimation, characterized in that... Includes the following steps: Step 1: Prepare a human pose estimation image dataset, which may contain multiple people or a single person, each with corresponding key point annotations; Step 2: The RTMpose model is used as the basic model for pose estimation, and a multi-class visibility prediction head (VH module) is introduced on this basis to extract the pose information of the target person from the image; Step 3: The multi-class visibility prediction head is achieved by fusing information from the pose feature layer output of RTMpose. Then, the prediction head uses the Sigmoid activation function to classify the visibility of each keypoint into three categories: 0, 0.5, and 1. Step 4: Modify the traditional BCE loss function to be suitable for multi-class classification problems. The visibility labels 0, 1 and 2 of each key point correspond to the new target values 0, 0.5 and 1, respectively, and are used to calculate the visibility classification loss of each key point. Step 5: Train the model to minimize the overall loss by combining the loss function described above; Step 6: After inference and visibility prediction, output the key point coordinates and their visibility labels for each image.
2. The multi-level keypoint visibility classification algorithm for human pose estimation according to claim 1, characterized in that: In step 1, the dataset for each image contains the following: each target person has a set of annotations, including the two-dimensional coordinates of their key points, and the visibility label (0, 1, 2) for each key point.
3. The multi-level keypoint visibility classification algorithm for human pose estimation according to claim 1, characterized in that: In step 3, 0 represents invisible, 0.5 represents partially visible, and 1 represents fully visible.
4. The multi-level keypoint visibility classification algorithm for human pose estimation according to claim 1, characterized in that: In step 3, the prediction head classifies the visibility value of each key point according to its relationship with the refraction interval, outputting category values of 0, 0.5 and 1, and predicts them together with their two-dimensional coordinates.
5. The multi-level keypoint visibility classification algorithm for human pose estimation according to claim 1, characterized in that: In step 4, the loss calculation is obtained through three categories: 0, 1, and 2. The categories are achieved by setting the Sigmoid refraction interval [0.3, 0.7].
6. The multi-level keypoint visibility classification algorithm for human pose estimation according to claim 1, characterized in that: In step 4, when the label is 0 or 2, the specific formula is as follows: 。 7. The multi-level keypoint visibility classification algorithm for human pose estimation according to claim 1, characterized in that: In step 4, when the label is 1, the specific formula is: 。