3D Scene Reconstruction With Geometry-Only Transmission for VR Broadcasts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D scene reconstruction methods require large bandwidth resources due to the transmission of high-quality images from multiple cameras, leading to inefficiencies in data transmission and potential delays, especially in live broadcasting scenarios.
Innovation Solution
Disentangle geometry and appearance features from target images, transmitting only the geometry feature when quality requirements are met, and updating the reference appearance feature when necessary to reduce data transmission, using a neural network-based disentangling model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-quality images from multiple cameras are transmitted for 3D scene reconstruction, then image quality and reconstruction accuracy are improved, but bandwidth consumption increases and transmission delays occur
Solution Approach 1:
The patent segments the image data into two independent parts: geometry features and appearance features. The geometry feature captures spatial structure information while the appearance feature captures visual appearance information. This segmentation allows selective transmission of only the geometry feature to the terminal device, reducing data volume while maintaining reconstruction accuracy.
Solution Approach 2:
The patent extracts only the essential geometry feature from the complete image data and transmits it to the terminal device, while keeping the appearance feature locally stored. This extraction approach eliminates redundant data transmission and focuses only on the critical information needed for 3D reconstruction.
2Measurement precision
If multiple original images from cameras are transmitted for 3D scene reconstruction, then reconstruction quality is improved, but transmission speed decreases due to large data volume
Solution Approach 1:
By segmenting image data into geometry and appearance features, the patent enables transmission of only the compact geometry feature representation, dramatically reducing data size and increasing transmission speed while preserving reconstruction quality.
Solution Approach 2:
The patent uses a pre-stored reference appearance feature (copy) at the terminal device to reconstruct the final image. Instead of transmitting the complete original images, only the geometry feature is transmitted and combined with the local appearance feature copy, achieving fast transmission without quality loss.
3Measurement precision
If complete image data is transmitted for real-time 3D scene reconstruction, then image quality is maintained, but bandwidth resources are wasted
Solution Approach 1:
The patent segments image information into geometry features (transmitted) and appearance features (stored locally). This segmentation enables efficient use of bandwidth resources by transmitting only the essential geometric information needed for reconstruction, while the appearance information is retrieved from local storage.
Solution Approach 2:
The patent extracts and transmits only the geometry feature, removing the redundant appearance information from the transmission stream. This extraction approach optimizes bandwidth utilization by sending minimal necessary data while maintaining image quality through local appearance feature references.
Data Source
AI summary
The present disclosure proposes a method and apparatus for image transmission for three-dimensional (3D) scene reconstruction. A target image may be obtained. A geometry feature and an appearance feature may be disentangled from the target image. An intermediate image may be reconstructed based on the geometry feature and a reference appearance feature. A difference between the intermediate image and the target image may be determined. The geometry feature may be transmitted to a receiving device for 3D scene reconstruction in response to determining that the difference is lower than a predetermined threshold.