Bird's-Eye Composite Imaging With Ray-Based Hole Filling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing driving assistance systems face challenges in generating accurate composite images with holes due to missing height and motion data, which can hinder path planning and obstacle avoidance.
Innovation Solution
The system uses multiple cameras to generate a composite image and fills holes using rays computed from the cameras to estimate missing height and motion data, enhancing the accuracy of the composite image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If multiple cameras are used to generate a composite image, then the coverage area and information completeness are improved, but the complexity of image processing and hole detection increases
Solution Approach 1:
The patent segments the composite image into multiple regions corresponding to different camera fields of view. Each region is processed independently to detect holes, and then the results are merged. This segmentation approach reduces the overall processing complexity by breaking down the large-scale image processing task into smaller, manageable sub-tasks while maintaining complete coverage from multiple cameras.
Solution Approach 2:
The patent introduces an intermediary processing layer that manages the integration of data from multiple cameras. This intermediary system coordinates the alignment, merging, and hole detection processes across different camera inputs, reducing the complexity burden on individual processing components while ensuring comprehensive information completeness in the final composite image.
2Measurement precision
If inverse projection mapping is used to determine object heights, then the height information accuracy is improved, but holes are generated in the composite image
Solution Approach 1:
The patent extracts height information specifically through inverse projection mapping from selected camera views where the mapping is most effective. By taking out only the height data extraction function and applying it selectively, the system achieves accurate height measurement while being aware of where holes will occur, allowing for subsequent compensation strategies.
Solution Approach 2:
The patent changes parameters such as camera selection, projection angles, and mapping configurations to optimize height measurement accuracy. By adjusting these parameters, the system maximizes the effectiveness of inverse projection mapping for height determination while minimizing the creation of holes in the composite image.
3Loss of information
If rays are computed to fill holes in the composite image, then the data completeness is improved, but the processing time and computational load increase
Solution Approach 1:
The patent performs preliminary actions by pre-computing ray paths and identifying potential hole locations before the actual hole filling process. This preliminary preparation allows the system to quickly fill holes during runtime without excessive computational overhead, as the complex ray-tracing calculations have already been partially performed in advance.
Solution Approach 2:
The patent applies partial action by filling only the necessary holes that affect critical path planning and obstacle avoidance, rather than attempting to fill every possible hole in the composite image. This selective approach maintains data completeness where it matters most while reducing unnecessary processing time and computational load.
Data Source
AI summary
This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving first image data from a first image sensor of a first camera, receiving second image data from a second image sensor of a second camera, wherein a first field of view of the second image data overlaps at least a portion of a second field of view of the first image data, generating, based on the first image data and the second image data, a bird's eye view composite image, detecting one or more holes in the bird's eye view composite image, and filling the one or more holes in the bird's eye view composite image using one or more rays of the first camera. Other aspects and features are also claimed and described.