Deferred Color Correction in Image Pipelines to Reduce Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing pipelines in autonomous vehicles suffer from color degradation and noise due to early application of color correction matrices (CCMs), leading to out-of-range values and artifacts that propagate through subsequent stages, affecting object recognition and tracking accuracy.
Innovation Solution
The proposed solution involves splitting the CCM into two stages within the image processing pipeline, deferring a portion of the color correction to a later stage, thereby reducing and eliminating out-of-range values and artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the color correction matrix (CCM) is applied early in the image processing pipeline, then color accuracy is improved, but out-of-range values and artifacts are generated that propagate through subsequent stages
Solution Approach 1:
The patent divides the color correction process into multiple stages by splitting the CCM into two separate matrices applied at different pipeline stages. The first CCM is applied after demosaicing, and the second CCM is applied after tone mapping, with the sum of their strengths equaling the original full-strength CCM. This segmentation prevents out-of-range values from propagating through the entire pipeline while maintaining color accuracy.
Solution Approach 2:
The patent applies a portion of the color correction (first CCM) before tone mapping as a preliminary action, which prevents out-of-range values from being generated in later stages. By performing partial color correction early, the system prepares the image data for subsequent processing without creating artifacts that would propagate through the pipeline.
2Measurement precision
If the full strength CCM is applied at an earlier stage, then color representation is improved, but false colors and noise are introduced that affect subsequent processing
Solution Approach 1:
The patent segments the full-strength CCM into two partial CCMs applied at different stages. The first partial CCM is applied after demosaicing, and the second partial CCM is applied after tone mapping, with their combined strength equaling the original full-strength CCM. This segmentation maintains color representation accuracy while preventing false colors and noise from propagating through subsequent processing stages.
Solution Approach 2:
The patent applies a preliminary partial color correction (first CCM) after demosaicing but before tone mapping. This preliminary action corrects colors early in the pipeline without generating out-of-range values, as the correction is applied to data that has not yet undergone tone mapping compression. The remaining color correction is applied later after tone mapping.
3Productivity
If traditional color spaces are used with high-dynamic-range sensors, then processing speed is maintained, but colors outside traditional color spaces cannot be represented accurately
Solution Approach 1:
The patent segments the color correction process into two stages with two separate CCMs. The first CCM operates on linearized image data after demosaicing, and the second CCM operates on data after tone mapping. This segmentation allows the system to handle out-of-range colors from high-dynamic-range sensors while maintaining processing efficiency, as each stage processes data in an optimized state without requiring complete color space conversion.
Data Source
AI summary
In various examples, image processing techniques are presented for reducing artifacts in images for autonomous or semi-autonomous systems and applications. Systems and methods are disclosed for deferring at least a portion of a strength associated with a color correction matrix (CCM) of an image processing pipeline to one or more other stages associated with the image processing pipeline. For instance, the CCM is used to determine a first CCM and a second, deferred CCM. The first CCM may be associated with an earlier stage of the image processing pipeline while the deferred CCM is associated with a later stage of the image processing pipeline. In some examples, the deferred CCM is combined with another matrix, such as a color space conversion matrix. By deferring part of the CCM, the image processing pipeline may reduce a number of artifacts and/or eliminate the artifacts that occur when processing image data.