View Synthesis Circuit for Depth-Based Image Transformation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for creating imaging special effects, such as parallax and 3D views, require extensive manual effort, expensive software, and multiple cameras, introducing artifacts and being limited by physical camera constraints.

Innovation Solution

A view synthesis circuit that transforms reference images using depth information to simulate camera position or focal length changes, detecting and filling holes through inverse warping and sampling neighboring pixels, allowing for single-camera operation and enhanced computational efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If multiple cameras are used to capture images from multiple viewpoints, then the quality of 3D view synthesis is improved, but the device complexity and cost increase

Engineering Contradiction:

Improve3D view synthesis qualityVSAvoidnumber of cameras

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent uses depth information to create a virtual copy of the scene that can be viewed from multiple angles. Instead of physically capturing multiple viewpoints with multiple cameras, the system synthesizes additional views by warping the reference image based on depth data, effectively creating virtual copies of the scene from different perspectives.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces the mechanical system of multiple physical cameras with a computational system. A single camera captures the reference image, and then computational algorithms process the image along with depth information to generate multiple transformed views, substituting physical hardware with software-based view synthesis.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Manufacturing precision

If extensive manual effort and advanced software tools are used to create imaging special effects, then the quality of visual effects is improved, but the ease of operation and accessibility decrease

Engineering Contradiction:

Improvevisual effects qualityVSAvoidoperator expertise required

Core Design Contradiction:

Manufacturing precisionVSEase of operation

Solution Approach 1:

The system performs automatic view synthesis using depth information and computational algorithms. The process is self-contained, requiring minimal manual intervention or advanced software tools. The device autonomously captures the reference image, processes it with the view synthesis algorithm, and generates the transformed views without requiring operator expertise in complex post-processing software.

Inventive Principle:

Principle #25Self-service

3Stability of the object's composition

If cameras are mounted on rigid rigging and tracks, then stable image capture is achieved, but the adaptability of camera position and motion is limited

Engineering Contradiction:

Improveimage capture stabilityVSAvoidcamera position and motion range

Core Design Contradiction:

Stability of the object's compositionVSAdaptability or versatility

Solution Approach 1:

The patent creates virtual copies of camera positions through computational view synthesis. Instead of physically moving cameras to different locations on rigging and tracks, the system synthesizes images that appear as if captured from various viewpoints, providing unlimited adaptability without physical constraints.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces the mechanical rigging and track system with a computational approach. Rather than physically relocating cameras along fixed paths, the system uses image warping and depth-based synthesis to simulate camera movement, eliminating mechanical constraints on camera position and motion.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Productivity

If multiple cameras and post processing interpolation are used, then transformed views are generated, but computational resources and storage requirements increase

Engineering Contradiction:

Improvetransformed view generationVSAvoidcomputational and storage resources

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent extracts and utilizes depth information from the reference image to enable efficient view synthesis. By leveraging the depth map, the system can perform view transformation through geometric warping rather than requiring multiple full-resolution images and complex interpolation algorithms, reducing computational and storage requirements.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS11153553B2Synthesis of transformed image views

Publication Date: 2021.10.19 TAHOE RES LTD

AI summary

Techniques are provided for synthesis of transformed image views, based on a reference image, using depth information. The transformed image views may simulate a change in position or focal length of a camera that produced the reference image. An example system includes an image transformation circuit configured to transform the reference image corresponding to a first viewpoint, to a transformed image corresponding to a second viewpoint. The system also includes an inverse warping circuit configured to calculate a mapping from the pixels of the transformed image to corresponding pixels of the reference image. The system further includes a hole detection circuit configured to detect holes in the transformed image based on depth discontinuities between the reference and transformed images; and a hole filling circuit configured to in-fill the detected holes using a sampling of selected neighboring pixels from the reference image, to synthesize a view based on the transformed image.