Vehicular Vision System 3D Point Registration Depth Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle vision systems using fish-eye lenses face challenges in accurate distance estimation due to variations in feature size and orientation, leading to unreliable depth calculations, especially in wide field of view optics, limiting their effectiveness beyond 3 meters and in central image areas.
Innovation Solution
A 3D point registration process is implemented, utilizing a priori knowledge of camera optics and extrinsic orientation to refine depth estimation through triangulation, weighted averaging, and iterative refinement of feature correspondences, enhancing the reliability and accuracy of distance calculations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If structure from motion (SfM) is used for depth estimation in wide field of view optics, then the system can operate with camera-only sensors, but the reliable estimation range is limited to about 3 meters and central image areas
Solution Approach 1:
The patent divides the image into multiple regions (central region and peripheral regions) and applies different processing strategies to each region. For the central region where SfM is limited, the system uses alternative methods such as object recognition databases and size-based distance estimation, while peripheral regions can still utilize SfM techniques. This segmentation allows the system to overcome the 3-meter range limitation in central areas while maintaining overall system functionality across the entire field of view.
Solution Approach 2:
The patent introduces an intermediary object recognition database that stores known object sizes and characteristics. This database acts as a mediator between the image data and distance estimation, providing reference information that enables accurate depth calculation in regions where traditional SfM fails. By comparing detected object features against the database, the system can estimate distances beyond the 3-meter SfM limitation.
2Area of stationary object
If fish-eye lenses are used to achieve wide field of view, then the camera can capture more environmental data, but feature size and orientation variations cause unreliable depth calculations
Solution Approach 1:
The patent applies different quality processing standards to different regions of the wide field of view image. Central regions with severe distortion use object recognition and database matching with relaxed precision requirements, while peripheral regions with less distortion can maintain higher precision SfM-based measurements. This local quality approach allows the system to utilize the entire wide field of view while maintaining reliable depth calculations in each specific region according to its characteristics.
Solution Approach 2:
The patent dynamically adjusts processing parameters based on the specific image region being analyzed. For central regions with high distortion, the system changes parameters such as feature matching thresholds, distance estimation methods, and reliance on object databases. For peripheral regions with lower distortion, standard SfM parameters are maintained. This parameter adaptation allows reliable operation across the entire fish-eye lens field of view despite varying distortion levels.
3Productivity
If traditional SfM is applied to moving vehicles with static targets, then depth can be estimated from frame-to-frame changes, but variations in motion along different optical axes reduce reliability
Solution Approach 1:
The patent performs preliminary classification of detected objects and image regions before applying depth estimation methods. By pre-identifying objects in the database and categorizing regions by their motion characteristics, the system can select the most appropriate estimation method in advance. This preliminary action allows the high-speed SfM method to be applied only where reliable, while switching to database matching for regions where motion variations would reduce reliability, thus maintaining both speed and consistency.
Data Source
AI summary
A vehicular vision system includes a camera disposed at an in-cabin side of a windshield of a vehicle. Responsive to image processing at an ECU of captured frames of image data, the vehicular vision system detects an object present in a field of view of the camera. As the vehicle moves relative to the detected object, captured frames of image data are processed to determine a point of interest, present in multiple captured frames of image data, on the detected object. The vehicular vision system, via processing of captured frames of image data as the vehicle moves relative to the detected object, and based on the determined point of interest present in multiple captured frames of image data, estimates a location in three dimensional space of the determined point of interest and determines distance to the determined point of interest on the detected object.