Image Processing With High-Density Rays For Radiance Field Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for estimating three-dimensional information about objects with complex shapes and colors result in inaccurate radiance field estimation outside the depth of field, leading to deteriorated image quality of virtual viewpoint images.
Innovation Solution
An image processing apparatus that learns radiance fields based on ray groups selected from high-frequency pattern regions in captured images, using a multilayer perceptron model to enhance the accuracy of three-dimensional information estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the sampling density of rays is uniformly reduced out of the depth of field to reduce computational amount, then the computational amount for estimating radiance fields is reduced, but the radiance fields of space corresponding to objects out of the depth of field cannot be estimated with high accuracy
Solution Approach 1:
The patent applies local quality by differentiating ray sampling strategies based on spatial location. Rays corresponding to objects within the depth of field are sampled at a first density, while rays corresponding to objects outside the depth of field are sampled at a second (lower) density. This localized differentiation allows the system to concentrate computational resources where they are most needed (within depth of field) while reducing unnecessary computations elsewhere, thereby resolving the contradiction between computational efficiency and estimation accuracy.
Solution Approach 2:
The patent segments the three-dimensional space into multiple regions based on depth of field characteristics. By dividing the space into regions within and outside the depth of field, the system can apply different sampling densities to different segments. This segmentation approach enables selective computation allocation, improving overall estimation accuracy for objects within the depth of field while reducing total computational burden through lower sampling density in other regions.
2Measurement precision
If the sampling density of rays within the depth of field is increased to improve estimation accuracy of radiance fields, then the estimation accuracy of radiance fields for objects within the depth of field is improved, but the computational amount for estimating the radiance fields increases
Solution Approach 1:
The patent implements local quality by applying higher sampling density specifically to rays corresponding to objects within the depth of field, while using lower sampling density for rays corresponding to objects outside the depth of field. This localized quality enhancement ensures that computational resources are concentrated where they provide the most value (within depth of field estimation) without uniformly increasing the total computational burden across all rays.
Solution Approach 2:
The patent applies partial action by selectively increasing sampling density only for rays that contribute most significantly to the final image quality (those within the depth of field). Rather than uniformly increasing sampling density for all rays (excessive action), the system applies the enhanced sampling only where necessary, thereby improving estimation accuracy for critical regions while avoiding the computational overhead of uniformly high sampling across the entire scene.
3Device complexity
If uniform sampling density is applied to all rays to simplify the estimation process, then the estimation process is simplified, but the image quality of virtual viewpoint images deteriorates due to insufficient sampling in critical regions
Solution Approach 1:
The patent resolves this contradiction by implementing local quality through position-dependent sampling density. Instead of using a single uniform sampling density for all rays (which would simplify the process but degrade image quality), the system applies different sampling densities based on the ray's position relative to the depth of field. This approach maintains relatively simple estimation procedures while significantly improving image quality by concentrating sampling efforts in critical regions where objects are located.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Three-dimensional information which enables generation of a high-quality virtual viewpoint image is estimated while reducing the amount of computations for estimating the three-dimensional information. An image processing apparatus (102) according to this disclosure obtains data on a captured image (S401), extracts a high-frequency pattern region from the captured image (S402), generates a plurality of ray groups based on rays corresponding to each of a plurality of pixels included in the captured image (S403), and learns three-dimensional information on a space using the plurality of ray groups (S404), wherein the image processing apparatus generates the plurality of ray groups including one or more high-frequency pattern region ray groups which are ray groups where a density of rays corresponding to pixels included in the high-frequency pattern region is higher than a density of rays corresponding to pixels included in a region other than the high-frequency pattern region (S702).