Egocentric Joint Pose Prediction Under Partial Visibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to accurately predict the pose of a person from egocentric images due to partial visibility and self-occlusion, with existing approaches being inaccurate, computationally expensive, or requiring extensive tuning.
Innovation Solution
A neural network is trained to directly predict a distribution over joint rotations and/or three-dimensional locations using a loss function, enabling accurate and compact pose prediction with uncertainty estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing methods are used to predict pose from egocentric images, then computational resources and time are consumed, but accuracy is insufficient due to partial visibility and self-occlusion
Solution Approach 1:
The neural network is divided into separate components: a backbone network for feature extraction and a prediction network for pose estimation. This segmentation allows each component to be optimized independently, improving both accuracy and computational efficiency by processing only relevant features at each stage.
Solution Approach 2:
The backbone network performs preliminary feature extraction from the egocentric image before the prediction network processes the pose estimation. This preliminary action prepares the data in advance, reducing the computational burden during the actual pose prediction and improving overall efficiency.
2Reliability
If complex methods are used to handle partial visibility, then more complete pose information can be obtained, but device complexity increases
Solution Approach 1:
The backbone network acts as an intermediary between the raw egocentric image and the prediction network. It processes the partially visible information and transforms it into meaningful features that the prediction network can use, maintaining reliability while keeping the overall system complexity manageable.
Solution Approach 2:
The neural network learns to infer complete pose information from partial observations by training on synthetic data that includes various occlusion scenarios. This creates a virtual copy of complete pose data that compensates for the partial visibility in real images, improving reliability without increasing physical device complexity.
3Manufacturing precision
If existing approaches are used, then implementation is straightforward, but manufacturing precision and accuracy are insufficient
Solution Approach 1:
The model is pre-trained on a large dataset of synthetic egocentric images with known ground truth poses before being applied to real images. This preliminary training action allows the model to learn accurate pose relationships in advance, achieving high manufacturing precision while simplifying the actual implementation process.
Solution Approach 2:
Synthetic training data is created as a copy of real-world scenarios with perfect ground truth labels. This virtual copy allows the model to learn precise pose predictions without the complexity of manually annotating real images, improving accuracy while maintaining ease of implementation.
Data Source
AI summary
To predict the pose of a person using images from egocentric cameras, at least one image from at least one egocentric camera is received, wherein the image depicts only a portion of the person. The method described herein uses a trained neural network to directly predict a distribution over rotation of a joint of the person or three-dimensional, 3D, location of a joint of the person.