3D Stream Timing Adaptation for Synchronized XR Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The streaming of 3D image data over wireless networks is bandwidth-intensive and suffers from synchronization issues due to mismatched processing times of mesh and texture components, leading to distorted renderings, especially in mobile networks.
Innovation Solution
A processing entity is employed to add transmission time indications to 3D image data streams, synchronize and adapt the frame rates of mesh and texture components based on network conditions and user activity, using cloud-based processing entities to optimize the 3D image data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If 3D image data stream is transmitted over wireless network, then extended reality content can be delivered to users, but bandwidth requirements become excessively high (600 MB/sec for 480x640 resolution at 60fps)
Solution Approach 1:
The patent segments the 3D image data stream into two separate components: a first image data stream representing geometric structure (mesh/point cloud) and a second image data stream representing visual appearance (texture/UV map). This segmentation allows independent processing and transmission of each component, enabling optimized compression and adaptive bitrate allocation for each stream type, thereby reducing overall bandwidth requirements while maintaining 3D content delivery capability
Solution Approach 2:
The patent changes the parameter of data representation by separating geometric and visual components, transmitting them as distinct streams with different compression schemes and bitrate allocations. This parameter change enables adaptive streaming where each component can be optimized independently for wireless transmission efficiency
2Productivity
If mesh and texture are processed at different speeds, then rendering efficiency is improved, but synchronization issues occur causing distorted renderings
Solution Approach 1:
The patent applies preliminary action by adding transmission time indications to each frame of both the first image data stream (geometric) and second image data stream (visual) before transmission. These time indications are inserted in advance, allowing the receiving device to pre-calculate and adjust playback timing to compensate for differential processing speeds, ensuring synchronized rendering without distortion
Solution Approach 2:
The patent implements feedback mechanisms where transmission time indications are sent along with each stream, and the receiving device uses these indications to monitor and adjust the rendering timing of mesh and texture components. This feedback loop ensures that even though processing speeds differ, the final rendered output remains synchronized
3Reliability
If transmission time indications are added to each stream, then synchronization is improved, but processing complexity increases
Solution Approach 1:
The patent introduces transmission time indications as an intermediary element that mediates between the differential processing speeds of mesh and texture streams. These time indications serve as a synchronization signal that simplifies the coordination process at the receiving device, allowing automatic timing adjustment without complex real-time synchronization algorithms
Data Source
AI summary
The application relates to a method for operating a processing entity configured to process a 3D image data stream for transmission over a wireless network to an extended-reality device, including receiving the 3D stream with a first image data stream representing a structure of a real-world object and a second image data stream representing a visual appearance of the real-world object, adding a corresponding transmission-time indication to each of the first image data stream and the second image data stream, resulting in an amended stream comprising an amended first stream and an amended second stream, transmitting the amended stream over the network to the extended reality device, receiving a processing time indication for each of the amended first stream and the amended second stream, and initiating an adapting of at least one of the 3D image data stream and the amended stream based on the processing-time indication.