Virtual Viewpoint Image Transparency for Focus Object Visibility

Solution Overview

Problem

Existing techniques for generating virtual viewpoint images from multiple camera perspectives are insufficient in effectively displaying objects, often resulting in obstruction issues where non-focus objects block the desired focus object.

Innovation Solution

A system that obtains virtual viewpoint information, designates a focus object, and decides which objects to change transparency based on their position relative to the virtual viewpoint, generating a virtual viewpoint image where non-focus objects become transparent to avoid obstruction.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If multiple cameras capture images from different directions to generate virtual viewpoint images, then the completeness of object representation is improved, but non-focus objects may obstruct the focus object in the generated image

Engineering Contradiction:

Improveobject representation completenessVSAvoidobstruction by non-focus objects

Core Design Contradiction:

Loss of informationVSObject-affected harmful factors

Solution Approach 1:

The patent applies local quality by differentiating the treatment of focus objects and non-focus objects. The transparency adjustment unit selectively modifies the transparency parameter only for non-focus objects while maintaining normal appearance for focus objects, thereby removing obstructions without affecting the subject of interest.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent changes the transparency parameter of non-focus objects to resolve the obstruction problem. By adjusting this physical parameter dynamically based on spatial relationships between objects and the virtual viewpoint, the system eliminates blocking while preserving complete object representation information.

Inventive Principle:

Principle #35Parameter changes

2Object-affected harmful factors

If all objects are displayed with normal transparency in virtual viewpoint images, then the overall scene visibility is maintained, but the focus object may be blocked by non-focus objects

Engineering Contradiction:

Improvefocus object visibilityVSAvoidnumber of visible objects

Core Design Contradiction:

Object-affected harmful factorsVSQuantity of substance

Solution Approach 1:

The patent extracts non-focus objects from the normal display state and applies special transparency treatment to them. This separation allows the focus object to be clearly visible while non-focus objects are either made transparent or adjusted in appearance, effectively removing the obstruction without eliminating all objects from the scene.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies local quality by differentiating the treatment of focus objects and non-focus objects. The transparency adjustment unit selectively modifies the transparency parameter only for non-focus objects while maintaining normal appearance for focus objects, thereby removing obstructions without affecting the subject of interest.

Inventive Principle:

Principle #3Local quality

3Object-affected harmful factors

If the transparency of objects is changed to prevent obstruction, then the focus object visibility is improved, but the realism of the virtual viewpoint image may be compromised

Engineering Contradiction:

Improveobstruction removalVSAvoidimage realism

Core Design Contradiction:

Object-affected harmful factorsVSReliability

Solution Approach 1:

The patent applies local quality by differentiating the treatment of focus objects and non-focus objects. The transparency adjustment unit selectively modifies the transparency parameter only for non-focus objects while maintaining normal appearance for focus objects, thereby removing obstructions without affecting the subject of interest.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent performs preliminary action by pre-calculating spatial relationships between objects and the virtual viewpoint before generating the final image. This allows the transparency adjustment unit to make informed decisions about which objects need transparency modification, ensuring the focus object remains visible while maintaining overall image realism through logical and spatially-aware processing.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11017588B2Image processing apparatus that generates a virtual view image from multiple images captured from different directions and method controlling the same

Publication Date: 2021.05.25 CANON KK

AI summary

A system comprises an obtainment unit that obtains virtual viewpoint information relating to a position and direction of a virtual viewpoint; a designation unit that designates a focus object from a plurality of objects detected based on at least one of the plurality of images captured by the plurality of cameras; a decision unit that decides an object to make transparent from among the plurality of objects based on a position and direction of a virtual viewpoint that the virtual viewpoint information obtained by the obtainment unit indicates, and a position of the focus object designated by the designation unit; and a generation unit that generates, based on the plurality of captured images obtained by the plurality of cameras, a virtual viewpoint image in which the object decided by the decision unit is made to be transparent.