Bounding Box Projection for Object Depth Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision systems for augmented reality (AR) face challenges in accurately estimating the depth of objects, especially when the size of the object is unknown, leading to rough estimates and increased processing time and power consumption.
Innovation Solution
A technique that uses a multi-camera AR device to estimate the depth of objects by applying a bounding box detection algorithm only once to a first image, generating 2-D positional data for landmarks, and then using a 2-D to 3-D lifter network to derive 3-D positional data, which is then used for triangulation to achieve accurate depth estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a bounding box detection algorithm is applied to both images to identify objects and their positions, then the accuracy of object localization is improved, but the processing time and power consumption increase significantly
Solution Approach 1:
The patent creates a projected bounding box in the second image based on the first bounding box and 3-D positional data, copying the region of interest without re-detecting it. This allows the system to focus processing only on the relevant area in the second image, significantly reducing processing time while maintaining localization accuracy.
Solution Approach 2:
The patent performs bounding box detection on only the first image before proceeding to process the second image. By preliminarily identifying the object position in one image and projecting it to the other, the system avoids the need to perform computationally expensive bounding box detection on both images, thereby reducing overall processing time.
2Reliability
If a bounding box detection algorithm is applied to both images to identify objects, then the reliability of depth estimation is improved, but the power consumption increases
Solution Approach 1:
The patent copies the bounding box information from the first image to the second image through projection using 3-D positional data. This copying approach maintains the reliability of object identification and depth estimation by ensuring consistent object tracking across images, while avoiding the redundant power consumption of running detection algorithms on both images.
Solution Approach 2:
The patent performs the energy-intensive bounding box detection operation only once on the first image as a preliminary step. The results are then reused for the second image through projection, significantly reducing power consumption while maintaining the reliability needed for accurate depth estimation.
3Measurement precision
If the landmark search area is not limited, then the accuracy of 3-D positional data derivation is improved, but the processing time increases
Solution Approach 1:
The patent applies local quality by limiting the landmark search to specific regions defined by bounding boxes in each image, rather than searching the entire image. This focused approach maintains high accuracy in 3-D positional data derivation by concentrating computational resources on relevant areas, thereby improving processing speed without sacrificing precision.
4Stability of the object's composition
If bounding box detection is performed on both images, then the consistency of object tracking across images is improved, but the overall system complexity increases
Solution Approach 1:
The patent copies bounding box information from the first image to the second image through geometric projection, ensuring consistent object tracking across images. This approach maintains tracking stability while reducing system complexity by eliminating the need to run independent detection algorithms on both images, thereby simplifying the overall processing pipeline.
Data Source
AI summary
A device includes a processor, image sensors, and memory storing instructions to obtain images from the sensors and process a first image to identify coordinates of a bounding box around an object. The device processes the area within the first box to determine 2-D positions of landmarks associated with the object, derives first 3-D positions of the landmarks, and determines coordinates of a second box bounding the object in the second image using the 3-D landmark positions. The device processes the area within the second box to determine 2-D positions of landmarks and uses triangulation to derive second 3-D positions of the landmarks. Overall, the device obtains images, detects objects and landmarks, determines 2-D and 3-D positions of landmarks, and triangulates 3-D positions.