Intermediate View Synthesis for 3D Video Parallax

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current 3D video systems face inefficiencies in data transmission and quality due to the need to transmit multiple views, leading to high bitrates and limitations in supporting head motion parallax and multi-user experiences, especially in home environments where glasses are not acceptable.

Innovation Solution

The use of intermediate view synthesis and multi-view data signal extraction techniques that divide images into non-boundary and boundary regions, allowing for efficient interpolation and rendering of intermediate views using depth information to minimize artifacts and reduce data requirements.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If multiple views are transmitted to support head motion parallax and multi-user experiences, then the 3D video quality and user experience are improved, but the data transmission rate increases significantly

Engineering Contradiction:

Improvesupport for head motion parallax and multi-user experiencesVSAvoiddata transmission rate

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The image is divided into boundary regions and non-boundary regions. Only boundary regions from multiple views are transmitted, while non-boundary regions are synthesized at the receiver. This segmentation allows supporting multiple views and head motion parallax while significantly reducing the quantity of transmitted data.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of transmitting complete multiple views, the system transmits only essential boundary information and synthesizes complete views at the receiver by copying and merging boundary regions with non-boundary regions from adjacent views. This creates full-view copies without transmitting all original data.

Inventive Principle:

Principle #26Copying

2Quantity of substance

If intermediate view synthesis is used to reduce data transmission, then the data rate is reduced, but artifacts may occur along object boundaries

Engineering Contradiction:

Improvedata transmission rateVSAvoidimage quality at boundaries

Core Design Contradiction:

Quantity of substanceVSManufacturing precision

Solution Approach 1:

Different processing is applied to different regions: boundary regions are transmitted with high precision to preserve object boundaries and avoid artifacts, while non-boundary regions are synthesized with lower precision. This local quality differentiation maintains manufacturing precision where needed while reducing overall data transmission.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

Boundary regions are identified and extracted in advance at the transmitter before transmission. This preliminary action ensures that critical boundary information is preserved and available for accurate view synthesis at the receiver, preventing artifacts along object boundaries.

Inventive Principle:

Principle #10Preliminary action

3Manufacturing precision

If complete multiple views are transmitted, then the 3D video quality is maintained, but the system complexity and processing requirements increase

Engineering Contradiction:

Improve3D video qualityVSAvoidsystem complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The essential information (boundary regions) is extracted from complete views for transmission, while the redundant information (non-boundary regions) is removed and reconstructed at the receiver. This extraction approach maintains 3D video quality while reducing system complexity by transmitting only necessary data.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The receiver performs view synthesis by merging transmitted boundary regions with non-boundary regions from adjacent views. This self-service approach at the receiver eliminates the need for the transmitter to process and transmit complete multiple views, reducing system complexity while maintaining quality.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS9214040B2Intermediate view synthesis and multi-view data signal extraction

Publication Date: 2015.12.15 FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV

AI summary

An intermediate view synthesis apparatus for synthesizing an intermediate view image from a first image corresponding to a first view and a second image corresponding to a second view different from the first view, the first and second images including depth information, wherein the second image being is divided-up into a non-boundary portion, and a foreground/background boundary region, wherein the intermediate view synthesis apparatus is configured to project and merge the first image and the second image into the intermediate view to obtain an intermediate view image, with treating the foreground/background boundary region subordinated relative to the non-boundary portion. A multi-view data signal extraction apparatus for extracting a multiview data signal from a multi-view representation including a first image corresponding to a first view and a second image corresponding to a second view being different from the first view is also described, the first and second images including depth information.