Virtual Viewpoint Image Noise Reduction via Regional Parameter Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional volumetric techniques for generating virtual viewpoint images often result in unnatural colors and noise due to differences in image generation and the need to invalidate images causing noise, leading to complex tasks and data wastage.
Innovation Solution
An information processing apparatus that obtains a first virtual viewpoint image from multiple directional images, determines image coordinates in a specific region, calculates parameters based on pixel values, and generates a second virtual viewpoint image using these parameters to reduce noise and effectively utilize image data outside noise regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional volumetric techniques are used to generate virtual viewpoint images, then images can be generated from multiple shooting apparatuses, but noise and unnatural colors occur due to differences in image generation
Solution Approach 1:
The patent changes the parameter of image selection by introducing a determination unit that evaluates multiple images based on specific criteria (pixel value consistency, noise levels, contribution percentages) and selects the most appropriate image for each region, thereby improving image quality while reducing noise and unnatural colors
Solution Approach 2:
The patent applies different image selection strategies to different regions of the virtual viewpoint image. The determination unit divides the image into multiple regions and selects images locally optimized for each region, ensuring high quality in noise-prone areas while maintaining overall image consistency
2Object-affected harmful factors
If images causing noise are invalidated to reduce noise, then noise in the virtual viewpoint image decreases, but the task becomes complex and image data outside noise regions is wasted
Solution Approach 1:
The patent segments the virtual viewpoint image into multiple regions and processes each region independently through the determination unit. This segmentation allows the system to identify and handle only the specific regions with noise problems, reducing overall task complexity while preserving useful image data in non-noisy regions
Solution Approach 2:
The determination unit acts as an intermediary between the multiple shooting apparatuses and the final image composition. It evaluates and selects the most appropriate image for each region, thereby reducing noise without requiring complex manual intervention or invalidating entire images when only partial regions are affected
3Object-affected harmful factors
If images causing noise are invalidated to reduce noise, then noise in the virtual viewpoint image decreases, but image data outside noise regions is wasted
Solution Approach 1:
The patent applies local quality control by selecting different images for different regions based on their specific characteristics. Images are only invalidated or deprioritized in regions where they cause noise, while the same images can still be used in other regions where they provide valuable data, thereby minimizing image data wastage
Solution Approach 2:
The determination unit dynamically changes the contribution parameter of each image based on regional noise characteristics. Instead of completely invalidating images, the system adjusts their contribution percentages locally, preserving useful image data while reducing noise impact in specific regions
Data Source
AI summary
An information processing apparatus comprising: one or more memories storing instructions; and one or more processors executing the instructions to: obtain a first virtual viewpoint image, wherein the first virtual viewpoint image is generated based on a group of images obtained by shooting a subject from a plurality of directions; obtain at least one image coordinate in a specific region in the first virtual viewpoint image; determine, based on a pixel value at the at least one image coordinate, a parameter for generating a virtual viewpoint image; and generate a second virtual viewpoint image using the determined parameter.