Immersive Video Depth Estimation Using Derivable Reference Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high computational complexity and data rate requirements for synthesizing depth maps in immersive video applications, particularly in client-side depth estimation, pose challenges due to the need for complex stereo matching and large data transmission.
Innovation Solution
A method for simplifying depth map calculation on the client side by deriving depth of certain blocks from reference depth blocks using linear combinations or operations, without requiring intrinsic or extrinsic camera parameters, and optimizing data transmission through lossless signaling of derivable depth blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional stereo matching methods are used to calculate depth maps, then depth estimation accuracy is improved, but computational complexity increases significantly
Solution Approach 1:
The patent divides the depth map into multiple blocks and processes each block independently using simplified operations. Instead of performing complex stereo matching on the entire image, the method segments the problem into smaller manageable blocks that can be processed with lightweight computations, thereby reducing overall computational complexity while maintaining depth estimation accuracy.
Solution Approach 2:
The patent copies depth information from reference views to target views for certain blocks without performing full stereo matching. By identifying derivable depth blocks that can be obtained through simple operations on reference depth blocks, the method avoids redundant complex computations while preserving depth map quality.
2Measurement precision
If all captured views and depth maps are transmitted to the client, then synthesis quality is improved, but data transmission rate increases
Solution Approach 1:
The patent extracts only the essential information needed for depth map reconstruction by identifying and transmitting only the necessary reference depth blocks and their corresponding operations. Instead of transmitting all captured views and complete depth maps, the method extracts minimal necessary data that enables the client to reconstruct the target depth map through simple operations.
Solution Approach 2:
The server performs preliminary analysis to identify which depth blocks are derivable from reference blocks before transmission. By pre-processing and identifying derivable blocks on the server side, the method reduces the amount of data that needs to be transmitted while ensuring that the client receives all necessary information for accurate depth map reconstruction.
3Quantity of substance
If depth maps are estimated on the client side from decoded texture views, then data transmission requirements are reduced, but computational complexity on client side increases
Solution Approach 1:
The patent enables the client to copy depth information from reference depth blocks to derivable depth blocks using simple operations. Instead of performing complex stereo matching on the client side, the method allows the client to replicate depth information from already-decoded reference blocks through lightweight operations, significantly reducing client-side computational complexity.
Solution Approach 2:
The patent segments the depth estimation task into identifying derivable blocks and applying simple operations, versus performing full stereo matching. By dividing the problem this way and providing guidance on which blocks are derivable, the method reduces the computational burden on the client while maintaining the ability to reconstruct complete depth maps.
4Measurement precision
If intrinsic and extrinsic camera parameters are used for depth calculation, then depth accuracy is improved, but device complexity increases due to parameter requirements
Solution Approach 1:
The patent copies depth information directly from reference views without requiring recalibration or application of intrinsic and extrinsic camera parameters. By using simple operations that replicate depth blocks from reference views, the method eliminates the need for complex parameter management while maintaining depth accuracy for derivable blocks.
Data Source
AI summary
A method of synthesizing an image includes receiving a coded video stream corresponding to different views of a scene acquired by a plurality of cameras, and identifying one or more derivable depth blocks of a target depth map corresponding to an image of the coded video stream. The method further includes determining a corresponding operation to derive a depth of each of the one or more derivable depth blocks, the operation being applied to a depth of a reference depth block included in a depth map of a reference image in the coded video bitstream. The method further includes obtaining the target depth map corresponding to the image by executing the corresponding operation of the respective derivable depth block, and synthesizing a new image according to the target depth map and at least one decoded image from the coded video stream.