Virtual Viewpoint Rendering Selection for Image Quality and Frame Rate
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing techniques for generating virtual viewpoint images face challenges in achieving uniform performance and high image quality due to varying processing loads based on the number of physical cameras used, which can lead to increased computational demands.
Innovation Solution
An image processing apparatus that determines the color of objects in virtual viewpoint images using a combination of virtual viewpoint-independent and viewpoint-dependent rendering methods, selecting appropriate rendering techniques based on the processing performance of user terminals to maintain high image quality and frame rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the technique in PTL 1 is used to determine color based on line-of-sight direction, then image quality is improved, but processing load increases
Solution Approach 1:
The patent applies dynamics by making the rendering method adaptable and changeable based on terminal performance. The system dynamically selects between viewpoint-dependent rendering (for high image quality) and viewpoint-independent rendering (for low processing load) according to the actual capabilities of the user terminal, allowing the rendering approach to flexibly adjust to different performance requirements
Solution Approach 2:
The patent changes the parameter of rendering method selection based on terminal performance characteristics. By evaluating terminal capabilities and adjusting the rendering approach accordingly, the system optimizes the balance between image quality and processing load, applying viewpoint-dependent rendering when terminals can handle it and viewpoint-independent rendering when they cannot
2Manufacturing precision
If the number of physical cameras is increased to determine object color, then image quality is improved, but processing load increases
Solution Approach 1:
The system dynamically adjusts the rendering strategy based on terminal performance, selecting between viewpoint-dependent and viewpoint-independent methods. This allows the system to handle variable numbers of physical cameras and their associated data without overwhelming terminals with excessive processing requirements
Solution Approach 2:
The patent changes rendering parameters according to terminal capabilities, optimizing the balance between utilizing multiple camera inputs for high image quality and maintaining manageable processing loads on diverse devices
3Manufacturing precision
If viewpoint-dependent rendering is used, then image quality is improved, but processing load increases
Solution Approach 1:
The system dynamically selects between viewpoint-dependent and viewpoint-independent rendering based on terminal performance evaluation. This allows the system to apply the higher-quality viewpoint-dependent method when terminals have sufficient processing power, while switching to the lighter viewpoint-independent method when terminals have limited capabilities
Solution Approach 2:
The patent changes the rendering method parameter according to terminal performance, optimizing the balance between image quality and processing load by selecting the appropriate rendering approach for each terminal's capabilities
Data Source
AI summary
A second image processing apparatus 40 generates a virtual viewpoint image including an object for which a color is determined based on a method selected between a virtual viewpoint-dependent rendering method for determining a color based on virtual viewpoint information for identifying a position of a virtual viewpoint and a line-of-sight direction from the virtual viewpoint, shape information for identifying a three-dimensional shape of the object, and images acquired by a plurality of imaging apparatuses, and a virtual viewpoint-independent rendering method for determining a color based on the shape information for identifying the three-dimensional shape of the object, and color information that identifies a color corresponding to a component element of the shape information.