Vehicle Surround View Rendering Using Temporal Camera Poses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle surround view systems using multiple cameras face challenges in providing comprehensive views of areas outside the cameras' field of view and are hindered by camera failures, leading to gaps and reduced functionality.
Innovation Solution
A method and system that utilize temporal camera poses and history buffers to render continuous surround views by associating camera images with vehicle locations, storing images in temporal order, and using virtual cameras to fill view gaps based on previous captures, ensuring seamless and robust image rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If multiple cameras are used to provide external views, then the field of view coverage is improved, but the system complexity and cost increase
Solution Approach 1:
The patent creates a virtual camera that copies and processes images from existing physical cameras. Instead of adding more physical cameras to cover blind spots, the system generates a virtual representation by mapping images from multiple physical cameras through a bowl mesh projection, effectively creating a synthetic camera view that expands coverage without additional hardware
Solution Approach 2:
The patent makes existing camera images serve multiple functions by projecting them onto a bowl mesh and enabling them to be viewed from multiple virtual perspectives. The same set of physical camera images is reused to generate both the original camera views and the expanded virtual surround view, eliminating the need for dedicated cameras for each viewpoint
2Area of stationary object
If cameras are positioned to maximize coverage, then blind spots are reduced, but areas outside the field of view of any single camera remain unavailable
Solution Approach 1:
The patent transitions from a 2D plane view to a 3D spherical/bowl representation by projecting camera images onto a bowl mesh. This dimensional transformation allows the system to represent and access areas that would be blind spots in any single 2D camera view, effectively eliminating blind spots by viewing the scene from multiple angular dimensions simultaneously
Solution Approach 2:
The patent merges images from multiple physical cameras into a unified bowl mesh projection. By combining the image data from several cameras and reprojecting them onto a common spherical coordinate system, the system creates a comprehensive view that covers areas invisible to any individual camera, eliminating blind spots through data fusion
3Stability of the object's composition
If historical images are stored in temporal buffers, then view continuity is maintained during camera movement, but memory usage and processing complexity increase
Solution Approach 1:
The patent performs preliminary projection of all camera images onto the bowl mesh in advance, organizing them by their corresponding vehicle locations. This pre-processing creates a structured temporal buffer where images are already mapped to their spatial-temporal coordinates, enabling efficient retrieval and seamless transitions during vehicle movement without requiring complex real-time processing
Solution Approach 2:
The patent implements a dynamic buffer management system that automatically adjusts which historical images are retained based on the vehicle's current location and movement. The buffer dynamically expires older images that are no longer relevant to the current view, maintaining view continuity while preventing unbounded memory growth through adaptive retention policies
4Adaptability or versatility
If the vehicle moves, then new areas become visible, but previously visible areas go out of view of all cameras
Solution Approach 1:
The patent maintains continuous visibility of previously viewed areas by keeping their projected images in the temporal buffer even after the vehicle has moved. The bowl mesh projection system continuously preserves the spatial relationships of all previously captured images, allowing the virtual camera to pan and tilt back to previous locations and maintain an unbroken visual record of the entire trajectory
Data Source
Figure 1A~1B
Figure 2
Figure 3
AI summary
Techniques including obtaining a first location of a vehicle (402), the vehicle (402) having one or more cameras disposed about the vehicle, and wherein each camera is associated with a physical camera pose indicating where each camera is located with respect to the vehicle (402), capturing, by a first camera, a first image of a first area (404), associating the first image with the first location of the vehicle (402) when the first image was captured, moving the vehicle in a direction so that the first area (404) is no longer within a field of view of the first camera, obtaining a second location of the vehicle, determining a temporal camera pose based on the physical camera pose of the first camera and the second location of the vehicle, and rendering a view of the first area based on the temporal camera pose and the first image.