Virtual Viewpoint Image Processing to Exclude Vehicle Structures

Solution Overview

Problem

Existing image processing systems for vehicles erroneously recognize projecting structures such as auxiliary mirrors as obstacles in virtual viewpoint images due to their installation positions, leading to potential misinterpretation by drivers.

Innovation Solution

An image processing apparatus that generates a virtual viewpoint image by specifying and excluding regions where structures project from the vehicle, using multiple cameras to synthesize images while avoiding inclusion of these projecting regions, thereby preventing erroneous recognition.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Area of stationary object

If cameras are installed at high positions to image larger road surfaces, then the imaging range is improved, but projecting structures of the host vehicle appear in the camera imaging ranges

Engineering Contradiction:

Improveimaging rangeVSAvoidprojecting structures appearing in imaging range

Core Design Contradiction:

Area of stationary objectVSObject-generated harmful factors

Solution Approach 1:

The patent extracts and removes the harmful projecting regions from the captured images before generating the virtual viewpoint image. The specifying unit identifies regions containing projecting structures (mirrors, antennas, etc.) and the image synthesis unit excludes these regions from the final synthesized image, effectively taking out the harmful elements while preserving the useful imaging range.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies local quality by treating different regions of the captured images differently. Regions containing projecting structures are identified and handled separately from normal road surface regions. The specifying unit detects these local problematic areas and the image synthesis unit processes them differently by excluding them, while maintaining normal synthesis for other regions.

Inventive Principle:

Principle #3Local quality

2Reliability

If auxiliary mirrors are installed at positions projecting from the host vehicle to check blind spots, then the blind spot monitoring function is improved, but the mirrors appear as synthesized images separated from the host vehicle in virtual viewpoint images

Engineering Contradiction:

Improveblind spot monitoring functionVSAvoidmisinterpretation of mirror images as obstacles

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The patent extracts and removes the harmful projecting regions from the captured images before generating the virtual viewpoint image. The specifying unit identifies regions containing projecting structures (mirrors, antennas, etc.) and the image synthesis unit excludes these regions from the final synthesized image, effectively taking out the harmful elements while preserving the useful imaging range.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent converts the harmful effect of mirrors appearing as separate synthesized images into a benefit by using the specifying unit to detect and identify these projecting regions. Instead of allowing them to cause misinterpretation, the system uses the detection capability to actively exclude them, transforming the potential harm into a controlled and resolved situation.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

3Device complexity

If a typical planar or bowl-shaped projection surface is used to generate the virtual viewpoint image, then the image synthesis process is simplified, but the auxiliary mirror appears at a position separated from the host vehicle in the generated virtual viewpoint image

Engineering Contradiction:

Improveimage synthesis processVSAvoidposition accuracy of synthesized objects

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent applies preliminary action by performing region specification and identification before the image synthesis process. The specifying unit detects and marks projecting regions in advance, so that during the subsequent image synthesis using planar or bowl-shaped projection surfaces, these pre-identified regions can be easily excluded without complicating the synthesis process itself.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent extracts and removes the harmful projecting regions from the captured images before generating the virtual viewpoint image. The specifying unit identifies regions containing projecting structures (mirrors, antennas, etc.) and the image synthesis unit excludes these regions from the final synthesized image, effectively taking out the harmful elements while preserving the useful imaging range.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS20250308087A1Image processing apparatus, image processing method, and storage medium

Publication Date: 2025.10.02 CANON KK

AI summary

An image processing apparatus includes: a virtual viewpoint image generation unit configured to generate a virtual viewpoint image on the basis of a plurality of captured images from a plurality of imaging units configured to be mounted on a movable apparatus; and a specification unit configured to specify, as a projecting region, a region where a structure projecting from the movable apparatus has been imaged in the captured images, the virtual viewpoint image generation unit is configured to generate the virtual viewpoint image such that the projecting region specified by the specification unit is not included.