Multiview Video Coding Using Depth-Based Region Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional multiview moving image coding methods face inefficiencies when the accuracy of disparity information expressed by a depth map is low, leading to suboptimal prediction and increased bitrate.

Innovation Solution

A moving image encoding and decoding method that utilizes reference view motion information and depth maps to generate disparity vectors and motion information for predicting image regions, allowing for fractional pixel accuracy and efficient coding even with low disparity accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If disparity-compensated prediction is used with low-accuracy depth maps, then multiview moving image coding can be performed, but prediction accuracy deteriorates and bitrate increases

Engineering Contradiction:

Improvecoding efficiencyVSAvoidprediction accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The encoding target region is divided into multiple prediction regions, and different prediction methods are applied to different regions. Specifically, regions with high depth accuracy use disparity-compensated prediction, while regions with low depth accuracy use motion-compensated prediction, optimizing overall coding efficiency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different prediction strategies are applied locally based on depth map quality. The prediction method selection is performed on a per-region basis rather than globally, allowing high-accuracy prediction where depth information is reliable and fallback to motion compensation where it is not.

Inventive Principle:

Principle #3Local quality

Solution Approach 3:

The patent changes the prediction method parameter based on depth accuracy. By evaluating depth map quality and dynamically selecting between disparity-compensated and motion-compensated prediction, the system adapts to varying depth accuracy conditions.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If disparity-compensated prediction is applied to entire encoding target region, then inter-view correlation is utilized, but regions with low depth accuracy produce poor prediction results

Engineering Contradiction:

Improveinter-view correlation utilizationVSAvoidprediction reliability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The encoding target region is segmented into multiple prediction regions based on depth accuracy evaluation. This allows selective application of disparity-compensated prediction only in regions where depth information is sufficiently accurate, preventing degradation in regions with low depth quality.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of applying disparity-compensated prediction to the entire encoding target region, the patent applies it partially only to regions where depth accuracy is sufficient. This partial application maintains prediction reliability while still utilizing inter-view correlation where beneficial.

Inventive Principle:

Principle #16Partial or excessive action

3Measurement precision

If motion-compensated prediction is used instead, then temporal correlation is utilized, but inter-view correlation benefit is lost

Engineering Contradiction:

Improveprediction accuracyVSAvoidcoding efficiency

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent applies motion-compensated prediction locally in regions where depth accuracy is insufficient, while using disparity-compensated prediction in regions with high depth accuracy. This local quality approach ensures optimal prediction accuracy in each region while maintaining overall coding efficiency.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent creates a universal prediction system that can perform both motion-compensated prediction and disparity-compensated prediction. This multi-functional approach allows the system to select the most appropriate method based on local depth accuracy conditions, combining the benefits of both approaches.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS10911779B2Moving image encoding and decoding method, and non-transitory computer-readable media that code moving image for each of prediction regions that are obtained by dividing coding target region while performing prediction between different views

Publication Date: 2021.02.02 NIPPON TELEGRAPH & TELEPHONE CORP

AI summary

A moving image encoding/decoding apparatus that performs encoding/decoding while predicting a multiview moving image including moving images of a plurality of different views includes: a corresponding region setting unit that sets a corresponding region on a depth map for an encoding/decoding target region; a region dividing unit that sets a prediction region that is one of regions obtained by dividing the encoding/decoding target region; a disparity vector generation unit that generates, for the prediction region, a disparity vector for a reference view using depth information for a region within the corresponding region that corresponds to the prediction region; a motion information generation unit that generates motion information in the prediction region from the reference view motion information based on the disparity vector for the reference view; and a prediction image generation unit that generates a prediction image for the prediction region using the motion information in the prediction region.