Virtual Space Image Generation Using Real Camera Position Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing techniques fail to accurately reflect the natural light environment of a real space in virtual space images generated from captured images, as they either ignore or inadequately represent the light conditions of the real space during image capture.

Innovation Solution

An image generation apparatus that determines the influence of captured images on virtual space images by specifying corresponding camera positions, calculating influence degrees based on camera and virtual camera positions, and using pixel value determination units to generate virtual space images that accurately reflect the natural light environment of the real space.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If white light is applied to the subject or a diffusion plate is arranged in capturing the captured images to reduce the influence of light environment, then the polygon model becomes independent of the light environment of the real space, but the natural light environment of the real space cannot be reflected on the virtual space image

Engineering Contradiction:

Improvepolygon model independence from light environmentVSAvoidnatural light environment information

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The patent applies preliminary action by capturing multiple images under different light environments before generating the polygon model. These pre-captured images with varying lighting conditions are stored and later used to synthesize virtual space images that reflect natural light environments, thus preserving light environment information in advance rather than losing it during the initial model creation

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent utilizes parameter changes by varying lighting conditions across multiple captured images. By capturing images under different light environments and using these varied parameters to generate virtual space images, the system preserves and reproduces natural light environment information rather than eliminating it through standardized lighting

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If computer graphics techniques with light sources are used to express the light environment in the virtual space image, then the polygon model can be displayed in virtual space, but the natural light environment of the real space in actually capturing the captured images cannot be reflected

Engineering Contradiction:

Improvevirtual space image generation capabilityVSAvoidlight environment accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent applies copying by creating virtual space images that replicate the actual light environments captured in the real space. Instead of using abstract computer graphics light sources, the system copies the real captured images with their authentic lighting characteristics into the virtual space, preserving the original light environment accuracy while enabling virtual space display

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent uses captured images as an intermediary between the real space light environment and the virtual space image. These images serve as a mediator that carries the natural light environment information from the real world into the virtual space, allowing accurate light environment representation without relying on computer graphics light source techniques

Inventive Principle:

Principle #24Intermediary (Mediator)

3Loss of information

If multiple captured images are used to generate the virtual space image, then the natural light environment can be better reflected, but the processing complexity and calculation load increase

Engineering Contradiction:

Improvelight environment information retentionVSAvoidimage processing complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The patent applies partial action by selectively using only the necessary captured images and their corresponding camera position information for generating each virtual space image. Rather than processing all captured images equally, the system determines the appropriate subset based on camera positions and viewing angles, reducing processing complexity while still accurately reflecting the natural light environment

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11107267B2Image generation apparatus, image generation method, and program

Publication Date: 2021.08.31 SONY INTERACTIVE ENTERTAINMENT LLC

AI summary

Provided are an image generation apparatus, an image generation method, and a program that can reflect, on a virtual space image, a natural light environment of a real space in actually capturing captured images. A corresponding camera position specification unit (68) specifies a corresponding camera position that is a position in a virtual space equivalent to a captured position that is a position in a real space of a real camera that has captured a captured image. An influence degree determination unit (80) determines a degree of influence on a virtual space image influenced by the captured image captured by the real camera at the captured position equivalent to the corresponding camera position based on a position of a virtual camera in the virtual space and the corresponding camera position. A virtual space image generation unit (86) generates the virtual space image based on the captured image captured by the real camera at the captured position and the degree of influence on the virtual space image influenced by the captured image.