3D Surround View Image-Based Lighting With Real-Time Cube Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D Surround View Monitor (3D SVM) technologies face limitations in generating real-time panoramic images due to performance constraints, resulting in unrealistic reflections on the 3D vehicle model, and the vehicle's camera cannot capture real-time panoramic images efficiently.
Innovation Solution
An image-based lighting method using a lookup table to generate panoramic images in real-time by mapping pixel coordinates, performing alpha blending and saturation adjustment, and converting to a cube map image for application on a 3D vehicle model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a pre-produced panoramic image is applied using environment mapping or reflection mapping, then the processing complexity is reduced and performance requirements are lowered, but the realism and accuracy of the reflected surroundings deteriorate
Solution Approach 1:
A lookup table is pre-calculated and stored, containing coordinate mapping relationships between the input image and panoramic image. During real-time operation, the system simply queries this pre-prepared table rather than performing complex coordinate transformations, thus maintaining low processing complexity while enabling accurate real-time panoramic image generation that reflects actual vehicle surroundings
Solution Approach 2:
The system generates a panoramic image by mapping pixels from the input image captured by the vehicle's camera, creating a real-time copy of the actual surroundings. This panoramic image is then applied to the 3D vehicle model, replacing static pre-produced images with dynamic real-world reflections that accurately represent the current environment
2Measurement precision
If panoramic images are generated in real-time from multiple images using SfM, Stereo Vision, or Omni-Directional Image Generation technologies, then the realism of reflected surroundings is improved, but the processing power requirements exceed the capabilities of vehicle MCU or VPU
Solution Approach 1:
The system extracts only the essential coordinate mapping relationship from the complex panoramic image generation process. By isolating and pre-calculating the coordinate transformation data into a lookup table, the system removes the computationally intensive real-time coordinate transformation operations, leaving only simple table queries that can be executed on vehicle MCUs or VPUs with limited processing power
Solution Approach 2:
The computationally heavy coordinate mapping calculations are performed in advance and stored in a lookup table. During real-time operation, the system only needs to perform simple table lookups and alpha blending operations, reducing the processing power requirements to levels that vehicle MCUs and VPUs can handle while still generating realistic panoramic images
3Speed
If a fixed panoramic image is used for reflection mapping, then the processing speed is maintained, but the heterogeneity between the reflected image and the real environment increases
Solution Approach 1:
The system transitions from using a static fixed panoramic image to generating a dynamic panoramic image in real-time from the vehicle's camera input. By continuously updating the panoramic image based on current camera captures and using alpha blending to handle edge regions, the reflected image dynamically adapts to match the actual real-time environment around the vehicle while maintaining processing speed through the lookup table optimization
Data Source
AI summary
An image-based lighting method includes: receiving an input image obtained by capturing a scene of one of a front, rear and side of a vehicle; generating a panoramic image by mapping a first coordinate of each pixel constituting a region of interest of the received input image to a corresponding second coordinate using a lookup table; analyzing histograms of red, green, and blue of an upper region of the panoramic image to select a color in which a red value, a green value, and a blue value with a highest frequency are combined, and performing alpha blending on the upper region using the selected color; adjusting saturation of the panoramic image; and converting the panoramic image into a cube map image and applying the cube map image to a 3D vehicle model, wherein the lookup table stores a relationship in which a first coordinate of each of the pixels constituting the region of interest of the input image corresponds to a second coordinate of a pixel of the panoramic image.