Pose Estimation Training With Anatomy Parsing for Reliable Keypoints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing semi-supervised human pose estimation methods rely on noisy pseudo-labels generated by fixed teacher networks, leading to incorrect predictions and overfitting due to sparse and unreliable supervision, which is costly and labor-intensive.
Innovation Solution
Introduce regional guidance through an additional human parsing branch that segments the human body's semantics, combining it with pseudo heatmaps to produce mixed heatmaps that selectively regenerate Gaussian pseudo heatmaps based on dynamic categorical thresholds, thereby providing accurate keypoint predictions aligned with human body topology.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If pseudo-labeling is used to generate training labels from unlabeled images, then the quantity of training data is improved, but the reliability of labels deteriorates due to noise and incorrect predictions
Solution Approach 1:
The patent introduces an anatomy parsing network as an intermediary that generates body part segmentation masks to guide the pose estimation network. This mediator provides reliable anatomical structure information that constrains and guides keypoint prediction, reducing the impact of noisy pseudo-labels while still utilizing abundant unlabeled images for training.
Solution Approach 2:
The patent implements a feedback mechanism where the anatomy parsing results are used to generate guidance signals that are fed back to the pose estimation network during training. This feedback loop allows the network to continuously refine its keypoint predictions based on anatomical consistency, improving reliability while processing large quantities of unlabeled data.
2Device complexity
If a fixed teacher network is used for pseudo-label generation, then the device complexity is reduced, but the manufacturing precision of pseudo-labels deteriorates due to limited initial labeled data
Solution Approach 1:
The patent segments the training task into two separate networks: an anatomy parsing network that handles body part segmentation and a pose estimation network that handles keypoint prediction. This segmentation allows each network to specialize in its respective function, improving the precision of pseudo-label generation without requiring a complex unified architecture.
Solution Approach 2:
The patent performs preliminary anatomy parsing to generate body part masks before conducting pose estimation. This preliminary action provides accurate anatomical boundaries and structure information that guides subsequent keypoint prediction, improving pseudo-label precision while maintaining relatively simple network architectures through staged processing.
3Device complexity
If sparse pseudo-labels are used for supervision, then the loss function computation is simplified, but the measurement precision of keypoint locations deteriorates due to lack of regional guidance
Solution Approach 1:
The patent applies local quality by generating anatomy-specific guidance signals for different body regions. The anatomy parsing network produces region-specific segmentation masks that provide localized constraints and guidance for keypoint prediction in each body part, improving measurement precision without requiring complex global loss functions.
Solution Approach 2:
The patent adds another dimension to the supervision signal by incorporating anatomy parsing results (spatial region information) alongside traditional keypoint coordinates. This dimensional enrichment transforms sparse point-based supervision into dense region-aware supervision, improving keypoint precision while maintaining computational efficiency through additive rather than multiplicative complexity.
Data Source
AI summary
Provided are a computer program product, system, and method for training a pose estimation model to determine anatomy keypoints in images. A teacher network, implementing machine learning, processes images representing anatomies to produce heatmaps representing keypoints of the anatomies. An anatomy parsing network, implementing machine learning, processes the images to produce segmentation representations labeling anatomies represented in the images. The segmentation representations from the anatomy parsing network and the heatmaps from the teacher network are concatenated to produce mixed heatmaps. A pose estimation model, implementing machine learning, is trained to process the images to output predicted heatmaps to minimize a loss function of the output predicted heatmaps from the pose estimation model and the mixed heatmaps.