Camera-Based Distance Acquisition Using 4D Cost Volume
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods using LIDAR and RADAR sensors for distance information in autonomous vehicles suffer from low resolution outputs and high costs, while camera-based methods face inaccuracies in determining distances due to reference point limitations and noise recognition.
Innovation Solution
A method employing multiple cameras to generate sub and main cost volumes through image processing, using sweep networks and cost volume computation networks with 3D convolution layers to accurately determine distances by projecting pixels onto virtual geometries and converting pixel coordinates, thereby improving segmentation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LIDAR sensors or RADAR sensors are used to obtain distance information, then the distance information can be acquired, but the output resolution is low and the cost is high with large power consumption
Solution Approach 1:
The patent segments the distance measurement task into multiple camera views, each capturing different spatial regions. By dividing the overall scene into multiple fields of view and processing each separately, the system achieves higher effective resolution than a single sensor could provide, while avoiding the use of expensive LIDAR or RADAR sensors
Solution Approach 2:
The patent transitions from traditional 2D image processing to 4D cost volume processing by adding depth dimension and time dimension. This dimensional expansion allows the system to extract precise distance information along the optical axis, compensating for the inherent limitation of camera-based depth estimation and achieving high measurement precision without relying on low-resolution LIDAR/RADAR outputs
2Ease of operation
If a conventional reference point method is used to obtain distance information from multiple cameras, then the processing is simplified, but the distance information becomes inaccurate when objects are occluded or located in directions not visible from the reference point
Solution Approach 1:
The patent creates a unified 4D cost volume that serves multiple cameras simultaneously, allowing distance information to be obtained for objects in any direction regardless of which camera views them. This universal approach replaces the single reference point method, ensuring accurate distance measurement for all objects including those occluded from the original reference point
Solution Approach 2:
The patent introduces a 4D cost volume as an intermediary data structure that integrates information from multiple camera views. This intermediary allows the system to reconcile distance measurements from different cameras and determine accurate distances even when objects are not visible from any single reference point, resolving the limitations of conventional methods
3Adaptability or versatility
If multiple cameras are used to cover all directions of the moving body, then the field of view is expanded, but the device complexity increases
Solution Approach 1:
The patent merges data from multiple cameras into a single unified 4D cost volume structure. By combining the information processing rather than treating each camera independently, the system achieves comprehensive directional coverage while simplifying the overall processing architecture, reducing the complexity that would otherwise result from handling multiple separate camera systems
Data Source
AI summary
A method for acquiring a distance from a moving body to an object located in any direction of the moving body includes steps of: an image processing device (a) instructing a sweep network to project pixels of images, generated by cameras covering all directions of the moving body, onto main virtual geometries and apply 3D concatenation operation thereon to generate an initial 4D cost volume, (b) generating a final main 3D cost volume therefrom through a cost volume computation network, and (c) generating sub inverse distance indices corresponding to inverse values of sub separation distances between a sub reference point and sub virtual geometries, and main inverse distance indices corresponding to inverse values of main separation distances between a main reference point and the main virtual geometries, by using a sub cost volume and the final main 3D cost volume, to thereby acquire the distance to the object.