3D Prediction Method for Video Coding Using Multiple Depth Planes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for compressing and transmitting 3D video data, particularly for Augmented Reality (AR) and Virtual Reality (VR), are inefficient, leading to high computational demands, storage issues, and network latency due to the lack of effective compression techniques for real-time high-quality content delivery, especially over wireless/mobile connections.
Innovation Solution
The implementation of Multi Focal Plane (MFP) and Multiple Depth Plane (MDP) prediction methods in predictive coding systems, which decompose and adjust focal planes/depth planes based on camera viewpoint changes to generate efficient 3D predictions, allowing for improved compression and transmission of 3D video data by summing pixel values along optical axes and coding prediction errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If 3D video data is stored and transmitted without compression, then high quality 3D scene rendering is maintained, but storage requirements and network bandwidth demand increase significantly
Solution Approach 1:
The patent segments the previous frame into multiple depth planes based on distance from the camera viewpoint. Each depth plane contains pixels at similar depths, allowing independent processing and prediction. This segmentation enables efficient compression by exploiting the spatial structure of 3D scenes without sacrificing rendering quality.
Solution Approach 2:
The patent transitions from traditional 2D frame-based prediction to 3D volumetric prediction by adding the depth dimension. By organizing pixels into depth planes and using 3D prediction modes that consider spatial relationships across multiple depths, the system achieves better compression efficiency while maintaining high-quality 3D reconstruction.
2Device complexity
If camera viewpoint changes between frames are handled using traditional 2D prediction methods, then computational complexity is reduced, but prediction accuracy and 3D reconstruction quality deteriorate
Solution Approach 1:
The patent implements dynamic adaptation to camera viewpoint changes by detecting motion between frames and adjusting the 3D prediction process accordingly. The system dynamically selects between different prediction modes (inter-frame, intra-frame, depth-based) and adjusts depth plane configurations based on the magnitude and direction of viewpoint changes, optimizing both accuracy and computational efficiency.
Solution Approach 2:
The patent introduces depth planes as an intermediary structure between the camera viewpoint and the pixel data. By using depth planes to mediate the relationship between different viewpoints, the system can efficiently handle viewpoint changes through depth-based warping and interpolation, achieving high prediction accuracy without excessive computational complexity.
3Loss of time
If real-time 3D video transmission is performed over wireless/mobile networks, then low latency is achieved, but compression efficiency must be increased to handle bandwidth constraints
Solution Approach 1:
The patent performs preliminary organization of pixel data into depth planes and pre-calculates prediction modes based on detected camera motion before actual compression and transmission. By preparing the 3D structure and selecting optimal prediction strategies in advance, the system reduces processing time during real-time transmission, achieving both low latency and high compression efficiency.
Data Source
AI summary
Systems and methods are provided for using a Multiple Depth Plane (MDP) prediction in predictive coding. The system detects a camera viewpoint change between a current frame and a previous frame, decomposes a reconstructed depth map of the previous frame to a plurality of depth planes, adjusts the plurality of depth planes from a previous camera viewpoint to correspond with a current camera viewpoint, generates an MDP prediction by summing pixel values of the adjusted plurality of depth planes along a plurality of optical axes from the current camera viewpoint, determines an MDP prediction error between the MDP prediction and a depth map of the current frame, quantizes and codes the MDP prediction error, and transmits, to a receiver over a communication network, the camera viewpoint change and the coded quantized MDP prediction error for reconstruction of a depth map of the current frame.