Single-Camera 3D Orientation from Key-Point Reprojection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video monitoring systems with a single camera struggle to accurately determine where a human is focusing their attention, as traditional techniques like skeleton tracking are inadequate, especially when only a single camera is used, limiting applications such as improved product placement and danger detection.
Innovation Solution
The system segments and analyzes a 2D image to identify key points, generates a 3D virtual object, rotates it in 3D space, and reprojects it into 2D space to determine the best matching orientation, allowing for precise determination of the object's area of focus without requiring additional computational resources like depth detection systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional skeleton tracking techniques are used to monitor human operations, then human movements can be tracked, but the system cannot determine where the human is focusing their attention
Solution Approach 1:
The patent transforms the 2D image data into 3D spatial information by generating a 3D virtual object and rotating it through various orientations. This dimensional transformation allows the system to determine the area of focus in 3D space and then map it back to the 2D image plane, enabling attention monitoring without additional depth detection hardware
Solution Approach 2:
The patent creates a 3D virtual object that corresponds to the detected 2D object. This virtual copy is then rotated and reprojected to find the orientation that best matches the 2D image data, allowing the system to infer the original object's orientation and determine the area of focus without requiring complex additional sensors
2Device complexity
If a single camera is used for monitoring, then device complexity is reduced, but the ability to determine depth and 3D information is limited
Solution Approach 1:
The patent compensates for the single camera's limitation by introducing 3D virtual objects and rotating them through space. This allows the system to synthesize depth information computationally from 2D image data, achieving 3D orientation determination without requiring multiple cameras or depth detection hardware
Solution Approach 2:
The patent changes the orientation parameters of the 3D virtual object by rotating it through various angles and positions. By comparing the reprojected 2D projections of these rotated objects with the actual 2D image data, the system can determine the correct orientation and infer depth information that would normally require additional sensors
3Measurement precision
If calibration processes are performed to map 2D image plane to 3D space, then depth information can be obtained, but the system still cannot accurately determine human area of focus
Solution Approach 1:
The patent segments the 2D image to identify key points of the object, then uses these key points to generate and orient the 3D virtual object. This segmentation approach allows the system to focus on specific features (like the head or gaze direction) to determine the area of focus, rather than treating the entire image uniformly
Solution Approach 2:
The patent enhances the calibrated 3D mapping by introducing rotational transformations of the 3D virtual object. This allows the system to test multiple orientations and determine which one best explains the observed 2D key points, thereby inferring the object's true orientation and the direction of attention in 3D space
Data Source
AI summary
Improved techniques for determining an object's 3D orientation. An image is analyzed to identify a 2D object and a first set of key points. The first set defines a first polygon. A 3D virtual object is generated. This 3D virtual object has a second set of key points defining a second polygon representing an orientation of the 3D virtual object. The second polygon is rotated a selected number of times. For each rotation, each rotated polygon is reprojected into 2D space, and a matching score is determined between each reprojected polygon and the first polygon. A specific reprojected polygon is selected whose corresponding matching score is lowest. The orientation of the 3D virtual object is set to an orientation corresponding to the specific reprojected polygon. Based on the orientation of the 3D virtual object, an area of focus of the 2D object is determined.