Depth Image Multi-Layer Coding Edge Error Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for synthesizing virtual images from depth videos often result in errors around edges, leading to artifacts in the synthesized images due to inaccuracies in depth information, particularly affecting the quality of multiview video plus depth representations.
Innovation Solution
A multi-layered coding scheme for depth images and videos that limits the maximum error for each reconstructed pixel, allowing successive refinement and adapting error limits based on local image characteristics like edges, ensuring accurate depth representation and improved virtual image synthesis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional depth compression techniques are used, then bitrate is reduced, but depth accuracy deteriorates causing artifacts in synthesized images
Solution Approach 1:
The patent applies different error limits to different regions of the depth image based on local characteristics. Edge regions use stricter error limits to preserve depth accuracy at boundaries, while smooth regions allow larger errors to reduce bitrate. This local differentiation resolves the contradiction by maintaining depth accuracy where needed while compressing data where tolerable.
Solution Approach 2:
The patent segments the depth image into multiple layers with different precision levels. The base layer provides coarse depth information for overall scene structure, while enhancement layers add finer details. This segmentation allows the system to allocate bitrate efficiently across different depth information components, reducing total bitrate while maintaining essential depth accuracy.
2Ease of manufacture
If uniform error limits are applied across the entire image, then encoding is simplified, but edge regions suffer from excessive errors causing artifacts
Solution Approach 1:
The patent implements adaptive error limits that vary by spatial location and image characteristics. Edge detection algorithms identify boundary regions where depth accuracy is critical, applying stricter error limits to these areas while using more relaxed limits in smooth regions. This resolves the contradiction by maintaining encoding efficiency while protecting edge quality.
3Measurement precision
If multiple enhancement layers are added to improve depth accuracy, then depth precision increases, but device complexity increases
Solution Approach 1:
The patent structures the multi-layer coding scheme with a clear base layer and hierarchical enhancement layers. Each enhancement layer targets specific frequency components or spatial regions, allowing systematic improvement of depth precision. The segmented structure manages complexity by organizing the encoding process into distinct, manageable stages rather than a monolithic complex system.
Solution Approach 2:
The patent employs adaptive parameter selection where error limits, quantization steps, and coding parameters are dynamically adjusted based on local image characteristics and available bitrate. This dynamic adaptation allows the system to achieve high depth precision when needed while reducing complexity when bandwidth is constrained, resolving the contradiction between precision and complexity.
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
A method reconstructs a depth image encoded as a base layer bitstream, and a set of enhancement layer bitstreams. The base layer bitstream is decoded to produce pixels of a reconstructed base layer image corresponding to the depth image. Each enhancement layer bitstream is decoded in a low to high order to produces a reconstructed residual image. During the decoding of the enhancement layer bitstream, a context model is maintained using an edge map, and each enhancement layer bitstream is entropy decoded using the context model to determine a significance value corresponding to pixels of the reconstructed residual image and a sign bit for each significant pixel, and a pixel value of the reconstructed residual image is reconstructed according to the significance value, sign bit and an uncertainty interval. Then, the reconstructed residual images are added to the reconstructed base layer image to produce the reconstructed depth image.