Spatially Adaptive Video Compression Using Depth Surface Normals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches to reducing network transmission bitrate for color video images with multiple color and depth views are not effective due to substantial redundancy in color views, which is not adequately addressed.
Innovation Solution
Computing a delta quantization parameter (ΔQP) for color images based on the similarity between the depth image surface normal and the view direction associated with a color image, allowing for efficient compression by degrading redundant content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional compression approaches are used for multiple color and depth views, then the compression process is simple, but the bitrate remains high due to substantial redundancy in color views
Solution Approach 1:
The patent segments the compression process into distinct stages: depth image processing to generate surface normals, similarity computation between depth normals and color view directions, and conditional quality adjustment of color images based on redundancy assessment. This segmentation allows complex redundancy reduction to be managed through modular, sequential operations.
Solution Approach 2:
The patent performs preliminary processing of depth images to compute surface normals before compressing color images. This preliminary action enables the system to identify redundant regions in advance, allowing for optimized bitrate allocation before the actual color image compression occurs, thereby reducing overall transmission bitrate efficiently.
2Loss of energy
If uniform quality compression is applied to all color images, then the compression process is simple, but the bitrate cannot be optimized due to unaddressed redundancy
Solution Approach 1:
The patent applies local quality control by computing similarity metrics for each pixel or region between depth surface normals and color view directions. Regions with high similarity (indicating redundancy) receive lower compression quality, while unique regions maintain higher quality. This local differentiation optimizes bitrate by allocating compression strength according to actual redundancy rather than applying uniform quality across all images.
Solution Approach 2:
The patent dynamically adjusts compression parameters (such as quantization levels) based on computed similarity metrics. When redundancy is detected through parameter comparison between depth and color views, the system changes compression parameters to achieve higher compression ratios for those specific regions, thereby optimizing overall bitrate while maintaining necessary quality where needed.
3Loss of energy
If redundancy in color views is not addressed, then the compression process is simple, but the bitrate is substantially higher than necessary
Solution Approach 1:
The patent introduces depth image surface normals as an intermediary element that mediates between the color images and the compression process. By using depth information as an intermediate representation, the system can identify redundant regions across multiple color views without directly comparing all color image pairs, thereby reducing processing complexity while still achieving effective redundancy reduction.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Techniques of compressing color video images include computing a delta quantization parameter (ΔQP) for the color images based on a similarity between the depth image surface normal and the view direction associated with a color image. For example, upon receiving a frame having an image with multiple color and depth images, a computer finds a depth image that is closest in orientation to a color image. For each pixel of that depth image, the computer generates a blend weight based on an orientation of a normal to a position of the depth image and the viewpoints from which the plurality of color images were captured. The computer then generates a value of ΔQP based on the blend weight and determines a macroblock of color image corresponding to the position, the macroblock being associated with the value of ΔQP for the pixel.