Pose Estimation Heatmaps With Bipartite Matching for Real-Time Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing pose estimation systems face challenges in efficiently associating keypoints with individuals in digital images due to ambiguity in channel ordering, leading to unstable training and high computational costs, especially in real-time applications.
Innovation Solution
A bipartite matching algorithm is used during model training to directly supervise joint person heatmaps, allowing the model to learn its own ordering scheme and combine joint type and person heatmaps, eliminating the need for computationally intensive post-processing steps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional bottom-up or top-down detection systems are used for pose estimation, then pose detection capability is achieved, but computational cost increases significantly
Solution Approach 1:
The patent segments the pose estimation task into two independent heatmap prediction branches: joint type heatmaps (predicting locations of specific joint types across all persons) and joint person heatmaps (predicting locations of all joints for each person). This segmentation allows each branch to be optimized independently and avoids the computationally intensive keypoint association step in traditional systems.
Solution Approach 2:
The patent introduces a new dimension by predicting not just joint locations but also person identities through the joint person heatmaps. This additional dimensional information enables direct association of keypoints with persons without requiring post-processing matching algorithms, reducing computational overhead while maintaining accuracy.
2Measurement precision
If traditional detection systems perform keypoint association to link keypoints with persons, then pose estimation is achieved, but processing time increases
Solution Approach 1:
The patent performs preliminary action by directly predicting joint person heatmaps that encode both joint location and person identity information during the forward propagation step. This eliminates the need for subsequent post-processing association steps, as the model directly outputs person-specific joint locations without requiring separate matching algorithms.
3Stability of the object's composition
If fixed channel ordering is imposed on heatmaps during training, then training stability is improved, but adaptability to different ordering schemes is reduced
Solution Approach 1:
The patent implements self-service by enabling the model to learn its own channel ordering scheme automatically during training through the joint person heatmap branch. The model adapts to the specific ordering convention used in the dataset without requiring manual configuration or fixed ordering constraints, achieving both stability through consistent learning and adaptability through automatic ordering discovery.
Data Source
AI summary
Introduced here is an approach to allowing a computer-implemented model to learn its own ordering scheme by applying an appropriate loss function during training. More generally, the present disclosure pertains to computer programs and associated computer-implemented techniques for estimating pose of a living body through simultaneous analysis of multiple visualizations. For example, joint type heatmaps—where a given heatmap includes every visible joint of the corresponding type across all visible persons—can be combined with joint person heatmaps—where a given heatmap includes every visible joint of the corresponding person—to better understand the relationship between joints visible in a digital image and people included in the digital image.