3D Surround View Image-Based Lighting for Real-Time Vehicle Reflections
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D Surround View Monitor (3D SVM) technologies face limitations in generating real-time panoramic images due to performance constraints, leading to unrealistic reflections on the 3D vehicle model and difficulty in processing images in real-time using vehicle MCU or VPU resources.
Innovation Solution
An image-based lighting method using a lookup table to generate panoramic images in real-time by mapping pixel coordinates, performing alpha blending and saturation adjustment, and converting to a cube map image for application on a 3D vehicle model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a pre-produced panoramic image is applied using fixed environment mapping or reflection mapping, then the 3D vehicle model can be rendered with texture mapping for light reflection and gloss, but the system cannot reflect actual real-time surroundings and gives a sense of heterogeneity different from the real environment
Solution Approach 1:
The patent applies dynamics by transitioning from static pre-produced panoramic images to dynamic real-time image processing. The system captures current surroundings using vehicle cameras and processes them through coordinate mapping and lookup tables to generate updated panoramic images that reflect actual real-time environments, enabling the 3D vehicle model to display accurate, adaptive reflections matching the current surroundings.
Solution Approach 2:
The patent replaces the mechanical limitation of fixed panoramic image application with computational coordinate mapping. Instead of using pre-fixed images, the system uses lookup tables that store coordinate relationships between camera input images and panoramic image outputs, allowing computational substitution to achieve real-time adaptive reflection mapping without mechanical reconfiguration.
2Reliability
If panoramic images are generated in real-time from multiple images using SfM or Stereo Vision, then actual surroundings can be reflected on the 3D vehicle model, but the vehicle MCU or VPU cannot process the images in real time due to computational complexity
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing coordinate mapping relationships in lookup tables during system initialization or offline processing. These lookup tables contain pre-computed transformations between camera coordinate systems and panoramic image coordinate systems, eliminating the need for complex real-time calculations and enabling fast real-time processing on vehicle MCUs or VPUs.
Solution Approach 2:
The patent changes the computational parameters by transforming complex multi-image processing (SfM, Stereo Vision) into simpler coordinate mapping operations using lookup tables. This parameter transformation reduces real-time computational complexity from full structure-from-motion algorithms to efficient pixel-coordinate transformations, making real-time processing feasible on limited vehicle computing resources.
3Reliability
If complex real-time image processing is performed to generate panoramic images, then actual surroundings can be reflected on the 3D vehicle model, but the computational load on vehicle MCU or VPU becomes too high
Solution Approach 1:
The patent extracts the complex coordinate mapping logic into separate lookup table structures that can be pre-computed and stored. By separating the heavy computational work into offline pre-processing and storing only the transformation data in lookup tables, the system extracts complexity from real-time processing, leaving only lightweight coordinate queries and pixel mappings for on-device execution.
Data Source
AI summary
An image-based lighting method includes: receiving an input image obtained by capturing a scene of one of a front, rear and side of a vehicle; generating a panoramic image by mapping a first coordinate of each pixel constituting a region of interest of the received input image to a corresponding second coordinate using a lookup table; analyzing histograms of red, green, and blue of an upper region of the panoramic image to select a color in which a red value, a green value, and a blue value with a highest frequency are combined, and performing alpha blending on the upper region using the selected color; adjusting saturation of the panoramic image; and converting the panoramic image into a cube map image and applying the cube map image to a 3D vehicle model, wherein the lookup table stores a relationship in which a first coordinate of each of the pixels constituting the region of interest of the input image corresponds to a second coordinate of a pixel of the panoramic image.