Virtual Viewpoint Imaging With Separated Foreground and Background Capture

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing image processing systems face issues with generating high-quality virtual viewpoint images due to brightness and hue differences in background images captured from multiple cameras, leading to patchy colors and increased processing load.

Innovation Solution

The system separates foreground and background images using dedicated cameras with different viewing angles, processing them separately to minimize the number of cameras needed for each region and reduce processing load, ensuring consistent image quality and reducing the likelihood of patchy backgrounds.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If a large number of cameras are used to capture images from various directions, then the quality of virtual viewpoint image is improved, but the background image becomes patchy due to brightness and hue differences

Engineering Contradiction:

Improvevirtual viewpoint image qualityVSAvoidpatchy background color

Core Design Contradiction:

Manufacturing precisionVSObject-generated harmful factors

Solution Approach 1:

The patent divides the imaging system into two separate groups: a first imaging apparatus group for capturing foreground objects and a second imaging apparatus group for capturing background images. This segmentation allows each group to be optimized for its specific function, with the background group using cameras positioned to capture uniform lighting conditions, thereby avoiding the patchy color problem while maintaining high image quality.

Inventive Principle:

Principle #1Segmentation

2Productivity

If both foreground and background images are generated by each image processing apparatus connected to cameras, then processing is distributed, but the processing load on each apparatus becomes heavier

Engineering Contradiction:

Improveprocessing distributionVSAvoidprocessing load per apparatus

Core Design Contradiction:

ProductivityVSPower

Solution Approach 1:

The patent segments the processing tasks by separating foreground and background image generation into different processing apparatuses. The first imaging apparatus group processes only foreground images, while the second imaging apparatus group processes only background images. This division reduces the processing load on each apparatus compared to handling both types of images simultaneously.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent extracts the background image processing function from the foreground image processing function and assigns it to a separate second imaging apparatus group. This extraction allows each apparatus to focus on a specific task, improving efficiency and reducing the processing burden on individual devices.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS12602840B2Image processing system, image processing method, and storage medium

Publication Date: 2026.04.14 CANON KK

AI summary

The present invention is directed to solving an issue arising when a background image is generated. An image processing system generates a foreground image containing a foreground object based on an image captured by an imaging apparatus included in a first imaging apparatus group. The image processing system generates a background image not containing the foreground object based on an image of the imaging region captured by an imaging apparatus included in a second imaging apparatus group different from the first imaging apparatus group. The image processing system generates a virtual viewpoint image based on the generated foreground image and background image.