Virtual Camera View Generation With Epipolar Occlusion Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Merging image data from different camera positions to change the perspective of a virtual view in vehicles can result in undesired artifacts due to inverse occlusion regions, where pixels are included or excluded incorrectly.
Innovation Solution
A method and system that determine the actual and desired poses of cameras, define epipolar geometry, and generate a virtual image by selecting pixels based on a minimum matched angle index, resampling depth information, and generating a disparity map to minimize artifacts, while removing indices with significant differences to stabilize the angle measurements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If image data from multiple camera positions are merged to change perspective, then virtual view capability is improved, but artifacts appear due to inverse occlusion regions
Solution Approach 1:
The patent segments the image processing by treating each pixel independently and evaluating its visibility from both actual and virtual camera positions. By processing pixels individually through occlusion analysis, the system can accurately determine which pixels should be visible in the virtual view, thereby avoiding artifacts caused by incorrect pixel inclusion or exclusion.
Solution Approach 2:
The patent replaces traditional geometric projection methods with a ray-casting based occlusion analysis system. Instead of relying on simple perspective transformation, the system uses virtual ray casting from the virtual camera position through each pixel to determine visibility, substituting mechanical geometric operations with a more sophisticated optical simulation approach.
2Measurement precision
If pixels are selected based on occlusion analysis, then accuracy of virtual view is improved, but computational complexity increases
Solution Approach 1:
The patent performs preliminary occlusion analysis by pre-calculating visibility information for each pixel before final image generation. The system determines in advance which pixels are visible from the virtual camera position by casting rays and checking for occlusions, storing this visibility information for use during image synthesis, thereby avoiding repeated complex calculations.
Solution Approach 2:
The patent introduces a new dimension of analysis by evaluating pixel visibility in 3D space through ray casting from the virtual camera position. Instead of working solely in 2D image space, the system adds the depth dimension by checking along ray paths, allowing accurate occlusion detection while maintaining computational efficiency through structured processing.
Data Source
AI summary
Systems and methods for generating a virtual view of a virtual camera based on an input image are described. A system for generating a virtual view of a virtual camera based on an input image can include a capturing device including a physical camera and a depth sensor. The system also includes a controller configured to determine an actual pose of the capturing device; determine a desired pose of the virtual camera for showing the virtual view; define an epipolar geometry between the actual pose of the capturing device and the desired pose of the virtual camera; and generate a virtual image depicting objects within the input image according to the desired pose of the virtual camera for the virtual camera based on an epipolar relation between the actual pose of the capturing device, the input image, and the desired pose of the virtual camera.