Virtual Viewpoint Image Processing With Adaptive Ray Density
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for estimating three-dimensional information from objects with complex shapes and colors result in poor image quality due to inadequate sampling density outside the depth of field, leading to inaccurate radiance field estimation.
Innovation Solution
An image processing apparatus that extracts high-frequency pattern regions from captured images and generates ray groups with higher density in these regions, using them to learn three-dimensional information, thereby improving estimation accuracy and reducing computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If uniform sampling density is used for all regions, then computational load is reduced, but estimation accuracy of radiance fields in high-frequency pattern regions deteriorates
Solution Approach 1:
The patent applies local quality by differentiating sampling density based on image region characteristics. High-frequency pattern regions (containing edges, textures, or details) receive higher sampling density, while low-frequency regions use lower sampling density. This resolves the contradiction by allocating computational resources locally according to actual needs, improving radiance field estimation accuracy in critical regions without uniformly increasing computational load across all regions.
2Measurement precision
If higher sampling density is used in high-frequency pattern regions, then radiance field estimation accuracy improves, but computational load increases
Solution Approach 1:
The patent changes the sampling density parameter dynamically based on image content analysis. By identifying high-frequency pattern regions through image processing and adjusting sampling density accordingly, the system optimizes the balance between estimation accuracy and computational load. This parameter adaptation resolves the contradiction by making sampling density a variable rather than a fixed value.
3Reliability
If uniform ray density is used for all pixels, then device complexity is reduced, but image quality in complex object regions deteriorates
Solution Approach 1:
The patent segments the image into different regions (high-frequency pattern regions and non-high-frequency regions) and generates ray groups with different densities for each segment. This segmentation approach improves image quality in complex object regions by providing higher ray density where needed, while managing device complexity through systematic region classification and differentiated processing.
Data Source
AI summary
Three-dimensional information which enables generation of a high-quality virtual viewpoint image is estimated while reducing the amount of computations for estimating the three-dimensional information. An image processing apparatus according to this disclosure obtains data on a captured image, extracts a high-frequency pattern region from the captured image, generates a plurality of ray groups based on rays corresponding to each of a plurality of pixels included in the captured image, and learns three-dimensional information on a space using the plurality of ray groups, wherein the image processing apparatus generates the plurality of ray groups including one or more high-frequency pattern region ray groups which are ray groups where a density of rays corresponding to pixels included in the high-frequency pattern region is higher than a density of rays corresponding to pixels included in a region other than the high-frequency pattern region.