Ground-Projection Color Harmonization for Surround View Stitching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing surround view systems (SVS) face issues with color mismatches in stitched images due to different lighting conditions and independent image processing, leading to noticeable artifacts at seams, which can distract drivers or autonomous systems.
Innovation Solution
Transfer color statistics from a ground projection of a reference frame to a target frame, rather than using global color statistics from an entire image, to harmonize colors across multiple camera sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If global color statistics from entire images are used for harmonization, then color matching between images improves, but computational complexity and processing time increase
Solution Approach 1:
The patent segments the image into multiple regions (sky region, ground region, and intermediate regions) and computes color statistics separately for each region. This segmentation approach reduces the computational complexity compared to processing entire images while maintaining color harmonization quality by focusing on specific regions that contribute most to color mismatches at seams.
Solution Approach 2:
The patent applies different color harmonization strategies to different regions of the image. Specifically, it computes color statistics for sky and ground regions separately and uses intermediate regions for blending. This local quality approach ensures that color harmonization is optimized for each specific region rather than applying a uniform approach to the entire image, thereby improving efficiency while maintaining quality.
2Productivity
If image processing is applied independently to each frame, then processing speed improves, but color consistency across frames deteriorates
Solution Approach 1:
The patent performs preliminary color statistics computation for each frame before stitching, organizing region-based color data in advance. This preliminary action allows for faster processing during the stitching phase while ensuring color consistency is maintained through pre-computed region-specific color statistics that can be efficiently applied during frame combination.
Solution Approach 2:
The patent introduces intermediate regions between sky and ground regions as mediators for color blending. These intermediate regions serve as a transition zone that harmonizes colors between frames at the seams, maintaining color consistency while allowing independent processing of distinct image regions to preserve processing speed.
3Manufacturing precision
If more comprehensive image processing is applied, then image quality improves, but processing time and latency increase
Solution Approach 1:
The patent segments the image processing task into region-specific color statistics computation, which is more efficient than comprehensive full-image processing. By focusing computation on specific regions (sky, ground, intermediate) rather than the entire image, the system achieves good image quality through targeted processing while reducing overall processing time and latency.
Solution Approach 2:
The patent applies partial action by computing color statistics only for specific regions (sky, ground, and intermediate regions) rather than processing the entire image comprehensively. This partial processing approach achieves sufficient image quality for surround view applications while significantly reducing processing time and latency compared to exhaustive full-image processing.
Data Source
AI summary
In various examples, color statistic(s) from ground projections are used to harmonize color between reference and target frames representing an environment. The reference and target frames may be projected onto a representation of the ground (e.g., a ground plane) of the environment, an overlapping region between the projections may be identified, and the portion of each projection that lands in the overlapping region may be taken as a corresponding ground projection. Color statistics (e.g., mean, variance, standard deviation, kurtosis, skew, correlation(s) between color channels) may be computed from the ground projections (or a portion thereof, such as a majority cluster) and used to modify the colors of the target frame to have updated color statistics that match those from the ground projection of the reference frame, thereby harmonizing color across the reference and target frames.