Vehicle Surround View Streaming With Adaptive 3D Seam Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle Surround View Systems suffer from noticeable seams in stitched images due to noise and white balance variations, geometric and texture distortions, and the inability to accurately represent three-dimensional environments, leading to artifacts that obscure useful visual information and distract drivers, particularly due to limitations in camera placement and visualization of the area under the vehicle.
Innovation Solution
The implementation of dynamic seam placement based on object saliency and ego-object state, an adaptive 3D bowl model that adjusts shape based on distance and direction to detected objects, and real-time reconstruction of the area under the vehicle using cached sensor data and ego-motion, along with streaming capabilities for remote visualization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If conventional image stitching techniques are used to create surround view visualization, then a complete 360° view is achieved, but noticeable seams and stitching artifacts appear that obscure visual information and distract drivers
Solution Approach 1:
The patent introduces an intermediary processing system that includes image capture modules, processing modules, and display modules. The processing module acts as an intermediary that receives images from multiple cameras, performs stitching operations, and outputs processed images to displays. This intermediary processing layer enables sophisticated stitching algorithms to reduce seams and artifacts while maintaining complete 360° coverage.
Solution Approach 2:
The patent implements dynamic seam placement that adapts to detected objects in the environment. The system dynamically adjusts seam locations based on object detection results, moving seams away from important objects and toward less significant areas. This dynamic adaptation reduces visible artifacts and improves visual quality without sacrificing field of view coverage.
2Reliability
If ultrasonic sensors are used to guide seam placement, then seams avoid very close objects, but objects outside the short sensing range are ignored and seams may be placed over important regions
Solution Approach 1:
The patent employs a multi-functional object detection system that combines multiple sensing technologies with different ranges and capabilities. By integrating various detection methods, the system achieves both short-range precision (for reliability in seam placement) and long-range coverage (for adaptability to different object distances). This universal detection approach ensures seams are placed accurately relative to all objects regardless of distance.
Solution Approach 2:
The system implements feedback mechanisms where object detection results continuously inform seam placement decisions. The processing module receives real-time object detection data, analyzes the environment, and adjusts seam locations accordingly. This closed-loop feedback ensures that seams are dynamically positioned to avoid important objects at various distances, improving both reliability and adaptability.
3Device complexity
If fisheye cameras are mounted at standard vehicle locations, then the system structure is simple, but the area under the vehicle remains a blind spot that cannot be visualized
Solution Approach 1:
The patent extends the visualization from traditional 2D planar displays to 3D spatial representation. By mounting cameras on multiple surfaces including the under-vehicle area and projecting images onto a 3D bowl-shaped model, the system recovers information from previously inaccessible dimensions. This dimensional extension allows visualization of the under-vehicle area without significantly increasing overall system complexity.
Solution Approach 2:
The patent segments the vehicle body into multiple surfaces (front, rear, left, right, and under-vehicle surfaces) and places cameras on each segment. Each camera captures images of its specific surface, and the processing module stitches these segmented views together. This segmentation approach enables comprehensive coverage including blind spots while maintaining a modular, manageable camera configuration.
4Area of stationary object
If multiple cameras capture images of moving objects from different perspectives, then complete coverage is achieved, but ghosting and object distortion artifacts appear in the stitched image
Solution Approach 1:
The patent performs preliminary object detection and tracking before the stitching process. The system identifies moving objects in advance, determines their trajectories, and uses this information to guide the stitching algorithm. By preparing object information beforehand, the system can accurately align images of moving objects from different camera perspectives, preventing ghosting and distortion artifacts while maintaining complete environmental coverage.
Data Source
AI summary
In various examples, sensor data may be captured by sensors of an ego-object, such as a vehicle traveling in a physical environment, and a representation of the sensor data may be streamed from the ego-object to a remote location to facilitate various remote experiences, such as streaming to a remote viewer (e.g., a friend or relative), streaming to a remote or fleet operator, streaming to a mobile app configured to self-park or summon an ego-object, rendering a 3D augmented reality (AR) or virtual reality (VR) representation of the physical environment, and/or others. In some embodiments, the stream includes one or more command channels used to control data collection, rendering, stream content, or even vehicle maneuvers, such as during an emergency, self-park, or summon scenario.