Multi-Task Pose Estimation for Real-Time Multi-Person Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current pose estimation systems for multiple people in images face challenges in efficiently assigning body joints to person instances, with bottom-up methods being faster but less accurate and top-down methods being slower and more costly, while also lacking in detail and accuracy in multi-person pose estimation.
Innovation Solution
A multi-task learning model with a shared backbone for feature extraction, utilizing residual networks and feature pyramid networks to jointly handle person detection, keypoint detection, and pose estimation, and implementing a Pose Residual Network (PRN) for assigning keypoints to person instances, enabling simultaneous consideration of all joints for grouping and handling overlapping detections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If top-down methods are used for multi-person pose estimation, then accuracy is improved, but processing speed deteriorates
Solution Approach 1:
The patent merges the advantages of top-down and bottom-up approaches by integrating person detection results with keypoint detection in a unified framework. The system first detects people using top-down method, then detects keypoints for each person, and finally groups keypoints to person instances using learned associations, combining the accuracy benefits of top-down with the efficiency of bottom-up grouping
Solution Approach 2:
The patent segments the pose estimation process into distinct stages: person detection, keypoint detection, and keypoint-to-person assignment. By separating these tasks and processing them in sequence with specialized networks for each stage, the system achieves both accuracy through focused detection and efficiency through streamlined processing
2Productivity
If bottom-up methods are used for multi-person pose estimation, then processing speed is improved, but accuracy deteriorates
Solution Approach 1:
The patent implements feedback mechanisms where detected person instances provide contextual information that guides keypoint detection and grouping. The system uses detected person locations and bounding boxes as feedback to refine keypoint detection accuracy and improve the reliability of keypoint-to-person assignments, thereby enhancing overall accuracy while maintaining speed
3Measurement precision
If single-person pose estimation is executed for each person detection, then accuracy is improved, but computational cost increases
Solution Approach 1:
The patent develops a universal keypoint detection network that can handle multiple persons simultaneously rather than executing separate single-person estimators for each detection. This multi-functional network processes all persons in parallel, reducing computational overhead while maintaining accuracy through shared feature extraction and coordinated detection
4Productivity
If keypoint grouping is performed without person detection, then processing speed is improved, but keypoint assignment accuracy deteriorates
Solution Approach 1:
The patent performs preliminary person detection and obtains person instance information before conducting keypoint grouping and assignment. By having person detection results available in advance, the system can use this information to guide and constrain the keypoint grouping process, improving assignment accuracy while maintaining efficient processing through pre-computed person contexts
Data Source
AI summary
A system for estimating a pose of one or more persons in a scene includes a camera configured to capture an image of the scene; and a data processor configured to execute computer executable instructions for: (i) receiving the image of the scene from the camera; (ii) extracting features from the image of the scene for providing inputs to a keypoint subnet and a person detection subnet; (iii) generating one or more keypoints using the keypoint subnet; (iv) generating one or more person instances using the person detection subnet; (v) assigning the one or more keypoints to the one or more person instances by learning pose structures from the image data; and (vi) determining one or more poses of the one or more persons in the scene using the assignment of the one or more keypoints to the one or more person instances.