User Pose Estimation from Face-Body Transforms Under Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional systems for determining user pose in autonomous mobile devices (AMDs) face challenges due to occlusion by obstacles and partial body occlusion outside the sensor's field of view, leading to inaccurate or unreliable user pose data.
Innovation Solution
The system processes sensor data, such as image and LIDAR data, to determine face and body pose data, using transforms to infer user pose even when the body is partially occluded, and employs Kalman filters and neural networks for accurate pose estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional sensor systems are used to determine user pose, then the system structure is simple, but the reliability of user pose data deteriorates due to occlusion by obstacles and partial body occlusion outside field of view
Solution Approach 1:
The system segments the user pose determination into two independent components: face pose detection and body pose detection. Each component processes sensor data separately and can operate independently, allowing the system to maintain reliability even when one component is occluded. The face pose module and body pose module are separate processing streams that can be selectively combined based on availability.
Solution Approach 2:
The system introduces an intermediary transform model that describes the spatial relationship between face pose and body pose. This transform acts as a mediator that can infer body pose from face pose when body data is occluded, or vice versa. The transform model bridges the gap between the two pose detection systems and enables reliable pose estimation under occlusion conditions.
2Measurement precision
If multiple sensors and processing algorithms are used to improve pose estimation accuracy, then measurement precision improves, but compute resources and processing time increase
Solution Approach 1:
The system implements partial action by selectively activating pose estimation components based on occlusion detection. When the body is fully visible, only body pose detection is needed. When occlusion is detected, the system partially activates the face pose detection and transform inference pipeline. This avoids the constant computational overhead of running all components at full capacity while maintaining accuracy when needed.
Solution Approach 2:
The system performs preliminary occlusion detection by analyzing sensor data to determine whether the user body or face is occluded before committing to a pose estimation strategy. This preliminary action allows the system to pre-select the appropriate processing pipeline (body-only, face-only, or combined with transform inference), avoiding unnecessary computational work and reducing energy consumption.
3Measurement precision
If the system waits for complete body visibility to determine accurate pose, then measurement precision improves, but response time and productivity deteriorate
Solution Approach 1:
The system performs preliminary assessment of data quality by detecting occlusion conditions in real-time. Based on this preliminary assessment, it immediately selects the appropriate pose estimation strategy without waiting for ideal conditions. This allows continuous pose determination at full speed while maintaining accuracy through adaptive methodology selection.
Solution Approach 2:
The system dynamically adapts its pose estimation approach based on real-time occlusion conditions. When the body is fully visible, it uses direct body pose detection for maximum accuracy. When occlusion is detected, it dynamically switches to face-based inference or alternative strategies. This dynamic adaptation maintains both speed and accuracy across varying conditions without requiring complete body visibility at all times.
Data Source
AI summary
An autonomous mobile device (AMD) that interacts with a user operates more effectively with reliable information about that user's pose. Occlusion due to obstacles or sensor field-of-view limits may prevent a sensor from “seeing” a user's torso or lower body while the face may remain visible. Face orientation alone produces unreliable results because a user frequently moves their head. Sensor data is acquired over time and used to determine face and body poses, with each pose representing a location and orientation in physical space of that portion of the user. A transform is determined that represents the relative arrangement of a face pose and a body pose at a particular time. If available, the body pose may be used as the user's pose. If the body pose is unavailable or unreliable at a given time, it may be inferred by applying the previously determined transform to the current face pose.