Multi-view Image Synthesis via Reference Ray Interpolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-view image processing schemes face challenges in providing a natural viewing experience by requiring a finely divided viewing area with a large number of viewable images, especially when the number of reference images is insufficient, leading to gaps in the displayed image.
Innovation Solution
A method that determines the color value of a ray in an image by selecting the closest reference rays from pre-photographed reference images, interpolating based on these values, and reselecting rays from neighboring pixels if the color difference exceeds a threshold, ensuring a seamless and detailed image rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a large number of reference images are used to provide a natural viewing experience, then the viewing quality is improved, but the data storage requirement and processing complexity increase
Solution Approach 1:
The patent pre-calculates and stores color values for specific reference rays in advance. When generating a new viewpoint image, the system retrieves these pre-stored color values and performs interpolation, avoiding the need to process all reference images in real-time. This preliminary preparation reduces both storage requirements and processing complexity while maintaining high viewing quality.
Solution Approach 2:
The patent creates a simplified representation by storing only the color values of reference rays rather than storing complete reference images. This copying approach extracts and stores only the essential information needed for view synthesis, significantly reducing data storage requirements while preserving the ability to generate high-quality multi-view images through interpolation.
2Productivity
If reference images are used to generate new viewpoint images, then the number of operations is reduced, but gaps appear in the displayed image when reference images are insufficient
Solution Approach 1:
The patent implements a feedback mechanism that detects when color values from reference rays cannot be obtained or when interpolation would produce poor results. In such cases, the system identifies edge regions and adjusts the interpolation strategy accordingly, switching to alternative methods for boundary pixels. This feedback loop ensures image completeness and eliminates gaps while maintaining efficient operation for regions where reference data is sufficient.
Solution Approach 2:
The patent dynamically changes the interpolation parameters based on the availability and quality of reference ray color values. When reference images are sufficient, standard interpolation is used for efficiency. When reference images are insufficient or edge regions are detected, the system adjusts parameters to use neighboring pixels for interpolation, ensuring complete image coverage without gaps while adapting the processing approach to local conditions.
3Speed
If color values are interpolated from reference rays, then processing speed is improved, but color accuracy decreases when reference rays are not sufficiently close
Solution Approach 1:
The patent applies different processing strategies to different regions of the image based on local characteristics. In regions where reference rays are sufficiently close, simple interpolation is used for high processing speed. In regions where reference rays are far apart or near image boundaries, the system identifies these as edge regions and applies more sophisticated interpolation using neighboring pixels. This local differentiation maintains color accuracy in critical regions while preserving processing speed in regions where it is sufficient.
Data Source
AI summary
A method of processing an image includes obtaining a position of a viewpoint corresponding to the image, selecting, based on the position, two reference rays closest to a ray included in the image from among a plurality of reference rays included in reference images photographed in advance from different reference viewpoints; determining whether a difference between color values of the selected reference rays is less than or equal to a threshold; and determining a color value of the ray based on the color values of the selected reference rays in response to the difference between the color values of the selected reference rays being determined to be less than or equal to the threshold.