Surround View Blind Spot Filling via Image Patch Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Surround view camera systems for vehicles often suffer from obstructed views due to intrinsic and extrinsic objects such as mirrors and other protuberances, leading to blind spots that hinder the driver's ability to see a complete image of peripheral areas.
Innovation Solution
An imaging system that uses intrinsic and extrinsic blind spot data, along with vehicle movement data, to predict and fill in blind spot regions by merging image patches from multiple camera perspectives, ensuring an unobstructed bird's eye view of the vehicle's surroundings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If cameras are mounted at the corners of the vehicle to provide surround view, then the field of view coverage is improved, but the view is blocked by mirrors or other vehicle parts creating blind spots
Solution Approach 1:
The system creates a virtual copy of the blocked region by projecting image data from adjacent unblocked areas into the blind spot region. This virtual copying allows the system to reconstruct missing visual information without requiring additional physical cameras in positions that would be blocked by mirrors or vehicle parts.
Solution Approach 2:
The system uses image processing algorithms as an intermediary to transfer visual information from visible regions to blind spot regions. By using software-based image projection and merging, the system bridges the gap between what the cameras can directly capture and the complete surround view that is needed.
2Loss of information
If multiple cameras are used to cover blind spots, then the completeness of surround view is improved, but the device complexity increases
Solution Approach 1:
The system makes existing cameras multi-functional by enabling them to contribute to both their direct field of view and the reconstruction of blind spot regions. Each camera serves its primary function of capturing its designated area while also providing data that can be projected and merged to fill in blind spots, eliminating the need for additional dedicated cameras for each blind spot region.
Solution Approach 2:
The system merges image data from multiple sources through a unified processing pipeline that combines direct camera feeds with projected blind spot regions. This merging approach creates a seamless composite surround view that integrates information from all cameras while automatically handling the complexity of coordination and alignment.
3Loss of information
If cameras are positioned to avoid obstruction, then blind spots are reduced, but the camera mounting positions become more limited
Solution Approach 1:
The system transitions from a spatial solution (positioning cameras to avoid obstruction) to a computational solution (processing images in the digital domain). By moving the problem-solving from the physical dimension of camera placement to the digital dimension of image processing, the system maintains flexibility in mounting positions while achieving complete coverage through software-based blind spot filling.
Data Source
AI summary
An imaging system, method, and computer readable medium filling in blind spot regions in images of peripheral areas of a vehicle. Intrinsic or extrinsic blind spot data is used together with vehicle movement data including vehicle speed and steering angle information to determine one or more portions of a series of images of the peripheral areas that include or will include one or more blind spot obstructions in the images. Portions of the images predicted to be obstructed at a future time, portions of overlapping images obtained concurrently from plural sources, or both, are obtained and used as an image patch. A blind spot region restoration unit operates to stitch together a restored image without the blind spot obstruction by merging one or more image patches into portions of the images that include the one or more blind spot obstructions.