3D Video Coding With Multi Focal Plane Viewpoint Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video compression methods are inefficient for 3D virtual environments, leading to high computational intensity and bandwidth demands, especially in real-time delivery of AR and VR content over wireless/mobile connections.
Innovation Solution
Implementing Multi Focal Plane (MFP) and Multiple Depth Plane (MDP) predictions to enhance 3D video coding by adjusting focal and depth planes based on camera viewpoint changes, generating predictions, and transmitting errors for reconstruction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional video compression methods are used for 3D virtual environments, then storage and transmission can be achieved, but computational intensity and bandwidth requirements become excessively high
Solution Approach 1:
The patent segments the 3D video data into multiple depth layers, each representing different depth planes of the scene. This segmentation allows independent processing and compression of each layer, reducing overall computational intensity while maintaining compression efficiency for 3D virtual environments
Solution Approach 2:
The patent introduces depth as an additional dimension beyond traditional 2D video frames, creating a multi-layered representation where each layer corresponds to a specific depth plane. This dimensional transformation enables more efficient compression by exploiting the structural relationships between depth layers, thereby improving productivity while managing computational resources
2Reliability
If uncompressed 3D video data is stored and transmitted, then quality is maintained, but storage system capacity and bandwidth requirements become prohibitively high
Solution Approach 1:
The patent creates simplified copies of the 3D scene at different depth planes rather than transmitting the complete high-resolution 3D data. Each depth layer represents a copy of the scene at a specific depth, allowing selective transmission and reconstruction of quality content while significantly reducing the total data volume that needs to be stored and transmitted
3Ease of operation
If real-time 3D content is delivered over wireless/mobile connections, then accessibility is improved, but network delays and latency increase
Solution Approach 1:
By segmenting 3D content into discrete depth layers, the patent enables selective transmission of only the most critical layers for real-time applications. This segmentation allows mobile and wireless systems to receive and render essential 3D content with reduced latency, improving accessibility while managing network time constraints
Solution Approach 2:
The patent enables dynamic adjustment of depth layer resolution and transmission priority based on network conditions. By changing parameters such as layer detail level and transmission timing, the system optimizes real-time delivery over wireless connections, reducing latency while maintaining accessibility for mobile users
Data Source
AI summary
Systems and methods are provided for using a Multi Focal Plane (MFP) prediction in predictive coding. The system detects a camera viewpoint change between a current frame from a current camera viewpoint to a previous frame from a previous camera viewpoint, decomposes a reconstructed previous frame to a plurality of focal planes, adjusts the plurality of focal planes from the previous camera viewpoint to correspond with the current camera viewpoint, generates an MFP prediction by summing pixel values of the adjusted plurality of focal planes along a plurality of optical axes from the current camera viewpoint, determines an MFP prediction error between the MFP prediction and the current frame, quantizes and codes the MFP prediction error, and transmits, to a receiver over a communication network, the camera viewpoint change and the coded quantized MFP prediction error for reconstruction of the current frame and display of the 3D scene.