Virtual Viewpoint Image Generation System with Multi-Module Rendering

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing systems for generating virtual viewpoint images face complexity and high costs due to the need for dedicated hardware and software configurations for various image formats, particularly when rendering is performed for different image qualities like 4K and 8K.

Innovation Solution

The system converts virtual viewpoint information into multiple pieces to allocate rendering processing across multiple rendering modules, allowing for efficient generation of virtual viewpoint images in various formats by sharing rendering processing and adapting results to specific image formats.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If dedicated rendering modules are configured for each image format (4K, 8K), then rendering quality and format-specific optimization are improved, but system complexity and hardware/software design costs increase

Engineering Contradiction:
Improverendering qualityVSAvoidsystem complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent implements a universal rendering module that can handle multiple image formats (4K, 8K, and others) through a single unified architecture. The rendering module includes a renderer that can process different virtual viewpoint information according to various image formats without requiring separate dedicated modules for each format, thereby reducing system complexity while maintaining rendering quality.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent employs parameter changes by adjusting rendering parameters such as resolution, frame rate, and virtual viewpoint configuration to adapt the same rendering module to different image formats. The system modifies these parameters dynamically based on the required output format rather than creating hardware-specific configurations for each format.

Inventive Principle:
Principle #35Parameter changes

2Adaptability or versatility

If dedicated rendering modules are configured for each image format, then format-specific optimization is improved, but hardware and software design costs increase

Engineering Contradiction:
Improveformat optimizationVSAvoiddesign cost
Core Design Contradiction:
Adaptability or versatilityVSEase of manufacture

Solution Approach 1:

The rendering module is designed as a universal component that can generate virtual viewpoint images in multiple image formats (4K, 8K, etc.) through a single unified architecture. This eliminates the need for separate dedicated rendering modules for each format, significantly reducing hardware and software design costs while maintaining the ability to optimize for different formats through parameter adjustment.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Manufacturing precision

If multiple virtual viewpoint images in different image formats are generated using dedicated renderers, then image quality and format requirements are met, but the configuration becomes complicated

Engineering Contradiction:
Improveimage qualityVSAvoidconfiguration simplicity
Core Design Contradiction:
Manufacturing precisionVSEase of operation

Solution Approach 1:

The system achieves different image qualities and formats by dynamically changing rendering parameters such as resolution, frame rate, and virtual viewpoint information rather than through complex configuration of multiple dedicated renderers. The single rendering module adjusts these parameters to generate 4K, 8K, or other format images as needed, simplifying the overall system configuration.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS11818323B2Image generation system, method for generating a virtual viewpoint image, and storage medium
Publication Date: 2023.11.14 CANON KK
  • US11818323B2 patent drawing
  • US11818323B2 patent drawing
  • US11818323B2 patent drawing

AI summary

An object is to efficiently generate virtual viewpoint images in different image formats. The image generation system includes a plurality of rendering modules. Then, virtual viewpoint information indicating a virtual viewpoint, for generating a virtual viewpoint image adapted to a predetermined image format, is converted into a plurality of pieces of virtual viewpoint information which indicate a plurality of virtual viewpoints, based on performance of a plurality of rendering modules. Then, based on the plurality of pieces of virtual viewpoint information after being converted, contents of rendering that should be executed are allocated to at least part of the plurality of rendering modules. Then, a virtual viewpoint image adapted to the predetermined image format is generated by using results of rendering processing by the at least part of the rendering modules.