Arbitrary Viewpoint Image Generation Using 3D Projection Parallax
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing arbitrary viewpoint image generation technologies face challenges in accurately calculating parallax information, especially in image areas outside the line of sight, leading to low image quality due to geometrically incorrect parallax information.
Innovation Solution
An image processing apparatus and method that obtain image information of a projection position along a straight line passing through the arbitrary viewpoint, define parallax information at pixel positions, and generate the arbitrary viewpoint image using this information, allowing for geometrically correct parallax calculation even in areas without feature points.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If parallax information is detected using feature point matching along the line of sight, then the calculation is simplified and processing is easier, but the image quality deteriorates in areas outside the line of sight due to geometrically incorrect parallax information
Solution Approach 1:
The patent transitions from 2D image plane matching to 3D space calculations by introducing the arbitrary viewpoint position and calculating projection positions in three-dimensional space. This allows parallax information to be obtained geometrically correctly for all pixel positions by projecting 3D world coordinates onto the arbitrary viewpoint image plane, resolving the limitation of line-of-sight-only matching while maintaining calculation feasibility through structured 3D coordinate transformations.
Solution Approach 2:
The patent changes the fundamental parameters used for parallax calculation from feature point coordinates to projection position coordinates derived from 3D world coordinates and camera positions. By using the projection position calculation formula that incorporates arbitrary viewpoint position, input image viewpoint positions, and pixel coordinates, the system obtains geometrically correct parallax information across the entire image area without relying on feature point detection.
2Device complexity
If feature point matching is used to calculate parallax, then processing complexity is reduced, but parallax information cannot be obtained in image areas without feature points
Solution Approach 1:
The patent creates a universal parallax calculation method that works for all pixel positions regardless of whether feature points are present. By using the projection position calculation based on 3D world coordinates and camera geometry, the system can compute parallax information for any pixel in the arbitrary viewpoint image, making the method applicable to the entire image area rather than being limited to regions containing detectable feature points.
Solution Approach 2:
The patent replaces the mechanical feature point detection and matching system with a geometric projection-based calculation system. Instead of relying on image processing algorithms to detect and match features, the system uses mathematical projection formulas that directly compute parallax from known camera positions, world coordinates, and pixel positions, eliminating the need for feature points entirely.
3Reliability
If compensation using adjacent feature point parallax is applied, then image areas without feature points can be processed, but the parallax information remains geometrically incorrect
Solution Approach 1:
The patent introduces 3D world coordinates as an intermediary between the input images and the arbitrary viewpoint image. By calculating projection positions through 3D space rather than directly matching 2D features, the system obtains geometrically correct parallax information. The 3D world coordinate system acts as a mediator that preserves geometric relationships during the projection process, eliminating the need for compensatory methods that propagate geometric errors.
Data Source
AI summary
An image processing apparatus for generating an image of an arbitrary viewpoint using a plurality of input images obtains image information of a projection position at which a point along a first straight line passing through the arbitrary viewpoint and a pixel position on the image of the arbitrary viewpoint is projected onto each of the plurality of input images, defines parallax information of the plurality of input images at the pixel position, using the image information obtained for the point along the first straight line, and generates the image of the arbitrary viewpoint, by defining the image information of the pixel position from the plurality of input images using the parallax information for the pixel position.