3D Video Decoder Hole Filling via Depth Edge Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D video coding methods face challenges in maintaining video quality due to holes appearing in decoded images, particularly when view displacement and occlusion occur, leading to reduced encoding/decoding quality.
Innovation Solution
A decoding method and encoder/decoder architecture that generate a first picture from a reference view, fill holes in the first picture using relevant texture and depth information, and then produce a decoded picture with residual data, thereby improving coding efficiency and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a same inter-view motion prediction disparity vector is used for all blocks of a picture, then the coding complexity is reduced, but holes appear in the decoded image reducing video quality
Solution Approach 1:
The picture is divided into multiple blocks, and each block is assigned its own motion prediction disparity vector instead of using a single global vector for the entire picture. This segmentation allows different regions to have different prediction parameters, reducing holes caused by uniform prediction while maintaining manageable coding complexity through block-based processing.
Solution Approach 2:
Different blocks within the picture are allowed to have different motion prediction disparity vectors tailored to their local characteristics. This local optimization improves prediction accuracy for each specific region, reducing holes in areas with varying depth or motion patterns while maintaining overall coding efficiency.
2Loss of information
If view synthesis technology is used to generate virtual view pictures, then inter-view redundancy is reduced, but holes appear due to displacement differences and shield relationships
Solution Approach 1:
The system uses depth information and occlusion detection as feedback to identify hole regions in the synthesized view. This feedback mechanism allows the encoder to detect where prediction failures occur and apply targeted hole filling operations, correcting the quality issues while preserving the redundancy reduction benefits of view synthesis.
Solution Approach 2:
Depth maps serve as an intermediary element between the reference view and the synthesized virtual view. By using depth information as a mediator, the system can perform depth-based hole filling to repair regions where occlusion or displacement caused prediction errors, thereby maintaining video quality while still achieving inter-view redundancy reduction.
3Reliability
If depth information is used for hole filling, then visual reliability is enhanced, but encoding complexity increases
Solution Approach 1:
Depth maps are generated and prepared in advance during the encoding process, before hole filling is performed. This preliminary preparation of depth information allows the hole filling operation to proceed efficiently using pre-computed depth data, reducing the actual encoding complexity during the hole filling stage while still achieving enhanced visual reliability.
Data Source
AI summary
A decoding method, a decoder, and an encoder are provided. A view synthesis technology is used to obtain, according to a reference view image, a synthesized image of a view to be encoded, comprising a texture image and a depth image, and a hole filling technology is used to predict hole area information and perform hole filling on the synthesized image of said view to obtain a predicted image.