Adaptive Multi-View Video Compression via Viewpoint Weight Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-view video compression techniques fail to effectively compress video data for parallax simulation and viewpoint-dependent effects, leading to inefficient network bandwidth usage and suboptimal user experience in 3D TV and immersive applications.
Innovation Solution
A system that uses weight maps to adaptively compress multi-view video by estimating the likelihood of frame portions being used for rendering at the remote end, varying compression rates based on predicted viewer positions and motion, and employing modified codecs like H.264 for efficient encoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional multi-view video compression techniques are used, then compression is achieved, but network bandwidth usage is inefficient and user experience is suboptimal
Solution Approach 1:
The patent applies local quality by differentiating compression rates across different regions of the video frame based on their importance for rendering. Weight maps are generated to identify important regions (those likely to be visible from the target viewpoint) and apply lower compression to these areas, while less important regions receive higher compression. This resolves the contradiction by optimizing compression effectiveness locally rather than uniformly, reducing overall bandwidth usage while maintaining quality where it matters most.
Solution Approach 2:
The patent employs preliminary action by pre-computing weight maps that predict which frame portions will be important for rendering before compression occurs. These weight maps are generated based on camera positions, target viewpoints, and scene geometry analysis. By performing this prediction and weighting analysis beforehand, the system can optimize compression parameters in advance, achieving both efficient bandwidth usage and effective compression for the specific rendering scenario.
2Device complexity
If uniform compression is applied to all video frames, then processing is simplified, but bitrate efficiency is reduced
Solution Approach 1:
The patent applies segmentation by dividing the video frame into multiple regions with different compression characteristics based on weight maps. Each region is identified by its importance weight, and compression parameters are adjusted per region rather than uniformly across the entire frame. This segmentation approach increases processing complexity slightly but achieves significant bitrate reduction by applying appropriate compression levels to different areas, optimizing the trade-off between complexity and bitrate efficiency.
Solution Approach 2:
The patent uses partial action by applying full compression only to less important regions while using reduced compression for important regions identified in the weight maps. This selective application of compression strength optimizes bitrate usage by preserving quality where needed and maximizing compression where acceptable, achieving better bitrate efficiency without requiring excessive processing complexity across the entire frame.
3Manufacturing precision
If all video streams are transmitted for decoding and rendering, then rendering quality is maintained, but network bandwidth consumption increases
Solution Approach 1:
The patent applies local quality by transmitting video data with differentiated compression levels for different spatial regions based on their rendering importance. Weight maps identify which regions will contribute most to the final rendered image from the target viewpoint, and these regions are transmitted with higher quality (lower compression) while less important regions are transmitted with lower quality (higher compression). This resolves the contradiction by maintaining rendering quality locally where it impacts the final image, while reducing overall network bandwidth consumption through selective compression.
4Productivity
If compression is optimized for specific viewpoints, then rendering efficiency improves, but system adaptability to viewpoint changes decreases
Solution Approach 1:
The patent applies dynamics by making the compression optimization adaptive to viewpoint changes. Weight maps are dynamically generated based on the target viewpoint parameters, allowing the system to re-optimize compression for different viewing angles and positions. When viewpoint changes occur, the weight map computation is updated accordingly, and compression parameters are adjusted in response. This dynamic approach maintains rendering efficiency for the current viewpoint while preserving adaptability to future viewpoint changes, resolving the contradiction between optimization and flexibility.
Data Source
AI summary
Multi-view video that is being streamed to a remote device in real time may be encoded. Frames of a real-world scene captured by respective video cameras are received for compression. A virtual viewpoint, positioned relative to the video cameras, is used to determine expected contributions of individual portions of the frames to a synthesized image of the scene from the viewpoint position using the frames. For each frame, compression rates for individual blocks of a frame are computed based on the determined contributions of the individual portions of the frame. The frames are compressed by compressing the blocks of the frames according to their respective determined compression rates. The frames are transmitted in compressed form via a network to a remote device, which is configured to render the scene using the compressed frames.