Vehicle Surround View 3D Reconstruction for Low-Light Parking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing surround view systems in vehicles struggle with detecting the environment around the vehicle, especially in poor lighting conditions and congested spaces, due to limitations in ultrasonic sensor performance and deep learning model reliability at night.
Innovation Solution
A method involving the capture of multiple wide-angle images around the vehicle using fisheye lens cameras, followed by the creation of a surround view image and the calculation of pose and depth estimates using neural networks, to generate a three-dimensional reconstruction of the environment and accurately map objects around the vehicle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If ultrasonic sensors are used for surround view detection, then the system can detect obstructions, but the sensing performance deteriorates in poor lighting conditions and parking lots
Solution Approach 1:
The patent combines multiple camera systems (front, rear, side cameras) to create a comprehensive surround view system that compensates for the limitations of individual sensors in poor lighting conditions. By merging data from multiple imaging devices with different fields of view, the system maintains reliable detection capability across all environmental conditions.
Solution Approach 2:
The patent transitions from two-dimensional image processing to three-dimensional reconstruction by calculating depth information and creating 3D models of the surround environment. This dimensional enhancement allows the system to accurately detect objects and distances even in challenging lighting conditions by adding spatial context to the visual data.
2Measurement precision
If deep learning models are used for vehicle light recognition, then the system can identify lights, but the model fails to generate reliable TTC values on pitch dark images
Solution Approach 1:
The patent changes the operational parameters of the deep learning model by training it on diverse lighting conditions including dark environments, and by adjusting inference parameters to optimize performance in low-light scenarios. The system also incorporates temporal information from multiple frames to compensate for poor illumination, effectively changing how the model processes input data in dark conditions.
Solution Approach 2:
The patent performs preliminary actions by pre-training the deep learning model on extensive datasets that include various lighting conditions, particularly dark environments. The model is prepared in advance with learned features that enable it to generate reliable TTC values even when encountering pitch dark images during actual operation.
3Area of stationary object
If multiple fisheye lens cameras are used to capture wide-angle images, then the field of view is improved, but the image distortion and processing complexity increase
Solution Approach 1:
The patent replaces complex manual calibration and geometric correction methods with neural network-based automated processing. The neural networks automatically handle the undistortion and stitching of fisheye images, substituting sophisticated mathematical computations with learned transformations that reduce processing complexity while maintaining accuracy.
Solution Approach 2:
The patent implements self-service by using the fisheye cameras and neural networks to automatically calibrate and correct their own distortion without requiring external calibration equipment or manual intervention. The system performs self-calibration by analyzing feature points across multiple images and automatically adjusting parameters to minimize distortion effects.
4Measurement precision
If disparity calculation is performed to detect object distance, then depth information is obtained, but the calculation accuracy deteriorates in overlapping field of view areas
Solution Approach 1:
The patent introduces neural networks as an intermediary between the raw disparity calculations and the final depth estimation. The neural network processes the disparity information from overlapping fields of view, learning to correctly interpret ambiguous regions and provide accurate depth estimates even where traditional disparity methods struggle due to image overlap and occlusion.
Data Source
AI summary
A computer implemented method for generating a three-dimensional reconstruction of an environment around a vehicle is provided. The method may include capturing multiple wide-angle images around the vehicle using multiple fisheye lens cameras mounted on the vehicle, creating a surround view image around the vehicle from the multiple captured images by generating one or more feature maps, and calculating a pose and depth estimate from the generated feature maps using at least one convolutional neural network. The method may also include detecting one or more objects in the multiple captured images, mapping one or more objects detected around the model of the vehicle to the created surround view image using the calculated pose and depth estimate; and constructing the three-dimensional reconstruction of the environment, using the surround view image and the mapped objects.