Multi-view Image Coding with Localized Illumination Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-view image coding methods face inefficiencies due to increased code amounts and accuracy issues from illumination and color mismatches between cameras, particularly when using single correction parameters for objects that are not complete diffuse reflectors or have mismatched depth and focus.
Innovation Solution
The method divides images into areas, determines depth information, and uses sample pixel groups from adjacent areas to estimate correction parameters for illumination and color mismatches, creating a view synthesis image that reduces prediction residual without needing to encode these parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If illumination compensation and color correction are performed using single correction parameters for the entire frame, then the prediction residual is reduced, but the coding efficiency deteriorates due to increased code amount
Solution Approach 1:
The image is divided into multiple regions, and correction parameters are determined independently for each region rather than using a single parameter for the entire frame. This segmentation allows localized illumination and color correction while avoiding the need to encode parameters for the whole image, thus reducing the code amount while maintaining prediction accuracy.
Solution Approach 2:
Different correction parameters are applied to different regions of the image based on local illumination and color characteristics. This local quality approach ensures that each region is corrected according to its specific conditions, improving prediction accuracy without requiring uniform correction parameters across the entire frame.
2Measurement precision
If multiple correction parameters are used for different objects, then the illumination and color correction accuracy is improved, but the code amount increases due to encoding multiple parameters
Solution Approach 1:
The image is segmented into multiple regions, each with its own correction parameters. This segmentation enables accurate local correction while managing the code amount by only encoding necessary parameters for each region rather than for all objects universally.
Solution Approach 2:
Correction parameters are changed and adapted for each region based on local characteristics. This parameter change strategy allows accurate correction for different illumination and color conditions in different regions while controlling the overall code amount through selective parameter encoding.
3Measurement precision
If view synthesis prediction is used with depth information, then the prediction accuracy is improved, but the device complexity increases due to depth estimation and triangulation processing
Solution Approach 1:
Depth information is obtained in advance from the coded data, and correction parameters are determined beforehand for each region. This preliminary action approach reduces the processing complexity during decoding by pre-computing correction parameters using depth information, avoiding complex real-time triangulation processing.
Solution Approach 2:
Depth information serves as an intermediary that connects the coded data with the correction parameter determination process. This intermediary enables accurate view synthesis prediction by facilitating the calculation of correction parameters without requiring complex direct triangulation processing during decoding.
4Measurement precision
If correction parameters are determined for each coding target area using sample pixel groups, then the localized illumination and color correction is achieved, but the calculation amount increases
Solution Approach 1:
Correction parameters are determined for each coding target area using sample pixel groups rather than processing the entire image. This partial action approach achieves localized illumination and color correction with reduced calculation amount by focusing computational resources only on relevant regions and sample pixels.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosed multi-view image coding/decoding device first obtains depth information for an object photographed in an area subject to processing. Next, a group of pixels in an already-coded (decoded) area which is adjacent to the area subject to processing and in which the same object as in the area subject to processing has been photographed is determined using the depth information and set as a sample pixel group. Then, a view synthesis image is generated for the pixels included in the sample pixel group and the area subject to processing. Next, correction parameters to correct illumination and color mismatches in the sample pixel group are estimated from the view synthesis image and the decoded image. A predicted image is then generated by correcting the view synthesis image relative to the area subject to processing using the estimated correction parameters.