3D Object Localization Using Admissible Depth Intervals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic depth estimation algorithms in multi-camera systems produce error-prone depth maps, leading to artefacts in virtual view synthesis, and existing methods for manual correction are cumbersome or indirect, lacking clear methods to compute admissible depth intervals and handle occluders.
Innovation Solution
Derive admissible depth ranges from user-provided 3D geometry approximations, using inclusive and exclusive volumes to constrain depth values, reducing the search space for correspondence determination and minimizing computational effort.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automatic depth estimation algorithms are used in multi-camera systems, then depth maps can be generated efficiently, but the depth maps are error-prone and produce artefacts in virtual view synthesis
Solution Approach 1:
The patent introduces an intermediary refinement process that takes the initial automatic depth map and a coarse 3D mesh as inputs, then produces a refined depth map. This intermediary step mediates between the efficient but error-prone automatic estimation and the accurate but computationally intensive manual correction, combining both approaches to achieve high accuracy with reasonable efficiency.
Solution Approach 2:
The patent employs feedback mechanisms where the refined depth map is validated against the 3D mesh constraints and multi-camera geometry. Errors in the initial depth map are detected and corrected through iterative refinement processes, with feedback loops that continuously improve depth map accuracy by comparing against known geometric constraints and adjusting accordingly.
2Reliability
If manual correction methods are used to fix depth map errors, then depth map quality can be improved, but the correction process is cumbersome and indirect
Solution Approach 1:
The system performs self-service by automatically refining the depth map using the provided 3D mesh constraints and multi-camera geometry information. The refinement process operates autonomously without requiring manual intervention, with the system itself identifying and correcting errors based on geometric consistency checks and constraint satisfaction.
Solution Approach 2:
The patent applies preliminary action by pre-processing the depth map with constraint-based refinement before final rendering or synthesis. The 3D mesh and geometric constraints are prepared in advance to guide the refinement process, establishing admissible depth intervals beforehand to narrow down the search space and prevent erroneous depth values from propagating through the pipeline.
3Measurement precision
If the search space for depth values is not constrained, then all possible depth candidates can be evaluated, but computational effort increases significantly
Solution Approach 1:
The patent segments the continuous depth search space into discrete admissible depth intervals based on 3D mesh constraints and camera geometry. By dividing the search space into manageable segments defined by mesh surfaces and geometric boundaries, the system evaluates only relevant depth candidates within each interval rather than searching the entire continuous range, significantly reducing computational effort while maintaining precision.
Solution Approach 2:
The patent applies local quality by constraining the depth search space differently in different regions of the image based on local 3D mesh geometry and object boundaries. Admissible depth intervals are computed locally for each pixel or pixel region based on the corresponding 3D mesh surface properties, allowing precise depth estimation where needed while reducing the search space in regions with clear geometric constraints.
4Productivity
If 3D geometry approximations are used to constrain depth values, then the search space can be reduced, but inaccuracies in the 3D geometry can lead to wrong depth map constraints
Solution Approach 1:
The patent applies beforehand cushioning by computing admissible depth intervals that intentionally include a margin of error around the 3D mesh-based depth estimates. Instead of using tight constraints that might exclude correct depths due to mesh inaccuracies, the system expands the admissible intervals to cushion against potential errors in the 3D geometry approximation, ensuring that correct depth values remain within the constrained search space even when the mesh is imperfect.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
There is disclosed a method (30, 350) for localizing, in a space containing at least one determined object (91, 101, 111, 141), an object element (93, 143, X) associated to a particular 2D representation element (XL) in a determined 2D image of the space, the method comprising: deriving (351) a range or interval of candidate spatial positions (95, 105, 114b, 115, 135, 145, 361') for the imaged object element (93, 143, X) on the basis of predefined positional relationships (381a', 381b'); restricting (35, 36, 352) the range or interval of candidate spatial positions to at least one restricted range or interval of admissible candidate spatial positions (93a, 93a', 103a, 112', 137, 147, 362'), wherein restricting includes at least one of: limiting the range or interval of candidate spatial positions using at least one inclusive volume (96, 86, 106, 376, 426a, 426b) surrounding at least one determined object (91, 101); and limiting the range or interval of candidate spatial positions using at least one exclusive volume (279, 299a-299e, 375) surrounding non-admissible candidate spatial positions; and retrieving (37, 353), among the admissible candidate spatial positions of the restricted range or interval, a most appropriate candidate spatial position (363') on the basis of similarity metrics.