AR/VR Depth Data Transmission Thresholding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In VR/AR systems, excessive data transmission occurs when depth information for all frames is sent from the electronic device performing rendering to the HMD device performing reprojection, even when there is little or no change in depth information.
Innovation Solution
An electronic device that transmits color information and depth information only when the amount of change in depth information exceeds a specified threshold, thereby reducing unnecessary data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If depth information for all frames is transmitted from the electronic device to the HMD device, then reprojection performance is maintained, but data transmission volume increases excessively
Solution Approach 1:
The patent extracts only the necessary depth information for transmission by comparing depth maps between frames and identifying only those pixels whose depth values have changed beyond a threshold. This selective extraction approach maintains reprojection quality by transmitting updated depth data while avoiding unnecessary transmission of unchanged depth information, thereby reducing overall data transmission volume.
Solution Approach 2:
Instead of transmitting complete depth information for every frame, the patent applies partial action by transmitting only the portion of depth information that has actually changed. This is achieved through frame-to-frame depth comparison and threshold-based filtering, sending depth updates only when necessary to maintain reprojection performance.
2Reliability
If depth information is transmitted for every frame, then accurate reprojection is achieved, but bandwidth consumption increases
Solution Approach 1:
The system extracts only the changed portions of depth information by comparing current frame depth maps with previous frame depth maps. By identifying and extracting only those pixel depth values that exceed a change threshold, the system maintains reprojection accuracy where needed while minimizing bandwidth consumption by excluding unchanged depth data from transmission.
Solution Approach 2:
The patent implements partial transmission of depth information by sending only the necessary updates rather than complete depth maps. This selective partial transmission preserves reprojection accuracy for changed regions while significantly reducing overall bandwidth consumption compared to transmitting all depth data every frame.
3Reliability
If complete depth information is transmitted continuously, then reprojection quality is maintained, but battery consumption increases
Solution Approach 1:
The patent extracts only the essential depth information that has changed between frames, avoiding transmission of redundant data. This selective extraction reduces processing requirements and transmission activity, thereby lowering battery consumption while maintaining reprojection quality through targeted depth updates.
Solution Approach 2:
By transmitting only partial depth information updates rather than continuous complete depth maps, the system reduces processing load and communication activity. This partial transmission approach maintains reprojection quality for changed regions while significantly reducing battery consumption associated with continuous full-depth-information transmission.
Data Source
AI summary
According to an embodiment, an electronic device for providing augmented reality and operating method includes: transmitting color information indicating colors of a plurality of pixels forming a first frame and depth information indicating depths of the plurality of pixels to a first external electronic device connected through the communication circuit; obtaining color information and depth information of a second frame following the first frame; calculating an amount of change of depth information of at least one object included in the first frame and the second frame based on the depth information of the first frame and the depth information of the second frame; and transmitting the depth information of the second frame together with the color information of the second frame to the first external electronic device based on that the amount of change of the depth information is greater than or equal to the specified depth threshold.