Multi-Person Pose Estimation via Keypoint Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current pose estimation systems for multiple people in images face challenges in efficiently assigning body joints to person instances, with top-down methods being slow and bottom-up methods lacking accuracy, while also requiring improvements in processing speed and extendibility to related image processing tasks.
Innovation Solution
A multi-task learning model with a Pose Residual Network (PRN) that jointly handles person detection, keypoint detection, and pose estimation, using a shared backbone for feature extraction and implementing a Pose Residual Network to assign keypoints to person instances, incorporating depth values to improve grouping accuracy and speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If top-down methods are used for multi-person pose estimation, then accuracy is improved, but processing speed deteriorates
Solution Approach 1:
The system segments the pose estimation problem into two independent stages: first detecting person instances (bounding boxes) and then estimating poses for each detected person. This segmentation allows the system to leverage accurate single-person pose estimation methods while avoiding the computational burden of jointly estimating poses for all persons simultaneously, thus maintaining high accuracy while improving processing speed.
Solution Approach 2:
The system performs preliminary person detection before pose estimation. By first identifying and localizing person instances in the image, the system prepares the data structure and reduces the search space for subsequent pose estimation. This preliminary action enables the pose estimation algorithm to focus only on detected person regions, significantly reducing computational complexity while maintaining accuracy.
2Productivity
If bottom-up methods are used for multi-person pose estimation, then processing speed is improved, but accuracy deteriorates
Solution Approach 1:
The system merges the advantages of both top-down and bottom-up approaches by combining person detection results with pose estimation in a unified framework. The system integrates the person instance detection module with the pose estimation module, allowing information from both stages to mutually reinforce each other, thereby achieving both high speed and high accuracy.
Solution Approach 2:
The system introduces person detection bounding boxes as an intermediary between raw image data and final pose estimation. These bounding boxes serve as a bridge that guides the pose estimation process, providing spatial constraints and reducing ambiguity. This intermediary structure enables the system to achieve accurate pose estimation without the computational overhead of traditional top-down methods.
3Measurement precision
If single-person pose estimation is executed for each person detection, then accuracy is improved, but device complexity increases
Solution Approach 1:
The system employs a universal pose estimation model that can handle both single-person and multi-person scenarios. The same pose estimation network is applied to each detected person instance, eliminating the need for separate specialized models. This multi-functional approach maintains accuracy while reducing system complexity by reusing the same computational framework across different person instances.
Data Source
AI summary
A system for estimating a pose of one or more persons in a scene includes a camera configured to capture one or more images of the scene; and a data processor configured to execute computer executable instructions for: (i) receiving the one or more images of the scene from the camera; (ii) extracting features from the one or more images of the scene for providing inputs to a keypoint subnet and a person detection subnet; (iii) generating one or more keypoints using the keypoint subnet; (iv) generating one or more person instances using the person detection subnet; (v) assigning the one or more keypoints to the one or more person instances by learning pose structures from image data; and (vi) determining one or more poses of the one or more persons in the scene using the assignment of the one or more keypoints to the one or more person instances.