Sender-Side Geometric Fusion of Depth Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches to network transmission of multiple depth views are inefficient due to redundancy and high bitrate requirements, particularly in the presence of occluded regions and noise.
Innovation Solution
The system performs sender-side geometric fusion to generate updated depth views by merging information across multiple depth views, reducing noise and filling in occluded regions, which are then transmitted with lower bitrate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional MVC compression schemes are used to transmit multiple depth views, then the original views are preserved with all artifacts, but the bitrate requirement becomes high and transmission efficiency deteriorates
Solution Approach 1:
The patent merges multiple depth views by performing geometric fusion to create a fused depth map that combines information from all input views. This fusion process integrates depth data from multiple sources to produce a single comprehensive depth representation, reducing the total data volume that needs to be transmitted while preserving the essential 3D structure and features of the original views.
Solution Approach 2:
The patent performs preliminary geometric fusion and occlusion handling before the actual video compression and transmission. By pre-processing the depth views to combine them into a fused depth map and identify occluded regions in advance, the system reduces the complexity and bitrate requirements for subsequent compression stages, improving overall transmission efficiency.
2Measurement precision
If multiple depth views are transmitted with high quality to maintain accuracy, then the 3D representation remains accurate, but network bandwidth consumption increases
Solution Approach 1:
The patent combines multiple depth views into a single fused depth map that preserves depth accuracy by integrating information from all views. This merging process maintains the precision of depth measurements while reducing the total data volume, as the fused map contains consolidated depth information rather than redundant separate views.
Solution Approach 2:
The fused depth map acts as an intermediary representation that bridges the gap between multiple high-precision input views and the compressed transmission stream. This intermediate fused map preserves the essential depth accuracy information in a compact form, allowing for efficient compression while maintaining measurement precision.
3Device complexity
If conventional compression is applied without geometric fusion, then the processing complexity is low, but occluded regions and noise are not reduced
Solution Approach 1:
The patent merges multiple depth views through geometric fusion to eliminate occluded regions and reduce noise. By combining depth information from multiple viewpoints, the fusion process fills in missing data from occlusions and averages out random noise, improving the overall quality and reliability of the depth data.
Solution Approach 2:
The patent performs preliminary geometric fusion and quality improvement operations before compression. This pre-processing step handles occlusion resolution and noise reduction in advance, so that the subsequent compression operates on already-cleaned data, achieving high quality output without excessive complexity during the compression phase.
4Productivity
If sender-side geometric fusion is performed to reduce bitrate, then transmission efficiency improves, but the computational complexity increases
Solution Approach 1:
The patent performs sender-side geometric fusion as a preliminary action before compression and transmission. By completing the computationally intensive fusion operations at the sender end before the data leaves the system, the patent achieves high transmission efficiency with reduced bitrate, while the computational complexity is concentrated in a single pre-processing stage rather than distributed throughout the transmission pipeline.
Data Source
AI summary
Systems and methods are described for utilizing an image processing system with at least one processing device to perform operations including receiving a plurality of depth views of an object, each of the plurality of depth views being captured from a respective viewpoint of the object, each of the plurality of depth views including respective depth data associated with a depth image of the object captured from the respective viewpoint, performing an aggregation operation on the plurality of depth views, and generating an image of the object from a target viewpoint based on the updated depth views, the target viewpoint being different from each of the respective viewpoints from which each of the plurality of depth views are captured.