Monocular 3D Pose Estimation for Articulated and Target Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for estimating hand and object poses in images rely on expensive multi-camera systems and active depth sensors, which are power-consuming and struggle with occlusions, lacking environmental understanding and semantic action recognition.
Innovation Solution
A data processing system using a trained neural network to jointly estimate three-dimensional articulated object and target object poses, recognizing action classes, utilizing a combination of a fully convolutional neural network and an interaction recurrent neural network to process monocular color or depth images, enabling efficient and accurate pose estimation and action recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-camera systems and active depth sensors are used for pose estimation, then measurement precision is improved, but device complexity and energy consumption increase
Solution Approach 1:
The patent combines multiple pose estimation tasks (articulated object pose and target object pose) into a single neural network model that processes monocular images. This merging of functions allows the system to achieve comprehensive pose estimation accuracy while using a simpler single-camera setup instead of complex multi-camera systems.
Solution Approach 2:
The patent uses a neural network model that learns to infer depth and three-dimensional pose information from two-dimensional monocular images. This creates a computational copy of depth sensing capabilities without requiring physical depth sensors, thereby reducing device complexity while maintaining pose estimation precision.
2Measurement precision
If multi-camera systems and active depth sensors are used for pose estimation, then measurement precision is improved, but use of energy increases
Solution Approach 1:
The patent replaces physical depth sensing mechanisms (active depth sensors and multi-camera systems) with a computational approach using a neural network that processes standard monocular images. This substitution eliminates the need for energy-intensive hardware while maintaining pose estimation accuracy through learned features from training data.
3Productivity
If traditional pose estimation methods are used, then processing speed is improved, but loss of information increases due to inability to recognize semantic actions
Solution Approach 1:
The neural network model is designed to perform multiple functions simultaneously: estimating articulated object pose, estimating target object pose, and recognizing semantic actions. This multi-functionality ensures comprehensive information extraction from monocular images without sacrificing processing efficiency, as all tasks are performed in an integrated framework.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A data processing system is provided that includes a processor having associated memory, the processor being configured to execute instructions using portions of the memory to cause the processor to, at classification time, receive an input image frame from an image source. The input image frame includes an articulated object (42) and a target object (44). The processor is further caused to process the input image frame using a trained neural network configured to, for each input cell of a plurality of input cells in the input image frame predict a three-dimensional articulated object pose of the articulated object (42) and a three-dimensional target object pose of the target object (44) relative to the input cell. The processor is further caused to output the three-dimensional articulated object pose and the three- dimensional target object pose from the neural network.