Multi-Person Pose Estimation via ML Model Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional pose estimation techniques struggle to accurately determine the pose of multiple people in a 2D image, especially when individuals are obstructed or blocking each other, as they lack depth information and fail to distinguish between multiple people or recover obstructed joints.
Innovation Solution
The use of multiple machine learning models to detect and refine joint locations in images, including the detection of missing joints through feature extraction and fusion, allowing for accurate pose estimation and 3D modeling of individuals in complex scenes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional pose estimation techniques are used on 2D images, then the process is simple, but the accuracy of joint location detection deteriorates when multiple people are present or when joints are obstructed
Solution Approach 1:
The system segments the pose estimation task into multiple specialized components: a first ML model for initial joint detection, a second ML model for refinement, and a third ML model for recovering obstructed joints. Each model focuses on a specific aspect of the problem, improving overall accuracy while maintaining manageable complexity through functional decomposition.
Solution Approach 2:
The patent introduces intermediate processing steps between image input and final pose output, including feature extraction, joint candidate generation, and refinement stages. These intermediary processes act as mediators that progressively transform the input image into accurate joint locations, handling the complexity incrementally rather than in a single step.
2Reliability
If a single ML model is used for joint detection, then the system is simpler, but the ability to recover obstructed or missing joints deteriorates
Solution Approach 1:
The detection system is segmented into three specialized ML models: the first model detects visible joints, the second model refines detected joint locations, and the third model recovers obstructed joints using anatomical constraints. This segmentation allows each model to excel at its specific function, improving reliability while distributing complexity across multiple specialized components.
Solution Approach 2:
The system changes parameters between models by adjusting the level of processing: the first model operates on raw image features, the second model operates on detected joint candidates with refinement, and the third model operates with anatomical constraint parameters to recover missing joints. This parameter transformation allows each model to work in its optimal regime.
3Loss of information
If 2D images are used for pose estimation, then the imaging process is simpler, but the ability to distinguish multiple people and recover obstructed joints deteriorates due to lack of depth information
Solution Approach 1:
The system compensates for 2D image limitations by changing the parameter space through feature extraction that captures contextual relationships between joints and body parts. The ML models transform 2D spatial parameters into probabilistic joint location estimates, effectively recovering depth information through statistical reasoning rather than direct measurement.
Solution Approach 2:
The patent replaces the mechanical/physical depth sensing system with an information-processing system. Instead of using depth cameras or other 3D sensing hardware, the system uses ML models to infer depth and spatial relationships from 2D image data, substituting computational inference for physical depth measurement.
Data Source
AI summary
Disclosed herein are systems, methods and instrumentalities associated with multi-person joint location and pose estimation based on an image that depicts multiple people in a scene, where at least some of the joint locations of a person may be blocked or obstructed by other people or objects in the scene. The estimation may be performed by detecting and grouping joint locations in the image using a bottom-up approach, and refining each group of detected joint locations by recovering obstructed joint location(s) that may be missing from the group. The detection, grouping, and/or refinement may be accomplished based on one or more machine learning (ML) models that may be implemented using artificial neural networks such as convolutional neural networks.