AR Split Rendering with Pose Prediction for Low-Latency Imagery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR rendering technologies face challenges with UE performance issues, latency in split rendering, and distortion in stereoscopic effects due to unpredictable user poses and fluctuating latency in server-UE communication.
Innovation Solution
A method and device for AR split rendering that predicts user poses and geometric information, using scene descriptions to generate and transmit two-dimensional images efficiently, reducing distortion and latency by employing predicted poses and geometric transformations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If AR rendering is performed on UE (user equipment), then rendering quality can be maintained, but UE performance requirements increase and cause processing bottlenecks
Solution Approach 1:
The patent divides the AR rendering process into two segments: server-side rendering (generating base images) and client-side composition (overlaying on real-world views). This segmentation allows complex rendering to be performed on the server while the UE only handles lightweight composition tasks, resolving the contradiction between rendering quality and UE performance requirements.
2Device complexity
If split rendering is implemented with server-UE communication, then rendering complexity is reduced on UE, but latency increases due to communication overhead
Solution Approach 1:
The patent performs preliminary rendering actions on the server side to generate AR images in advance before they are needed at the UE. By pre-computing and preparing rendering content on the server, the system reduces the time required for real-time processing at the UE, thereby mitigating latency introduced by server-UE communication.
3Speed
If user pose predictions are used for AR rendering, then rendering speed improves, but distortion occurs due to unpredictable user poses
Solution Approach 1:
The patent implements a dynamic approach by continuously tracking and predicting user poses rather than using static pre-rendered images. The system adapts rendering content based on predicted user movements, allowing it to maintain rendering speed while minimizing distortion through real-time pose estimation and dynamic image adjustment.
4Adaptability or versatility
If multiple poses are pre-rendered to handle user movement, then rendering adaptability improves, but data transmission volume and processing overhead increase
Solution Approach 1:
The patent applies local quality by rendering different levels of detail for different regions of the AR scene based on predicted user focus areas. Instead of uniformly rendering all possible poses with equal quality, the system concentrates rendering resources on critical regions and angles, reducing overall data transmission volume while maintaining adaptability for user movement.
Data Source
AI summary
The present disclosure relates to a 5G or 6G communication system for supporting higher data transmission rates. According to various embodiments of the present disclosure, an operation method of a terminal in a wireless communication system is provided, the method comprising the steps of: receiving, from a server, information of a plurality of terminal prediction poses obtained by prediction, and information of a two-dimensional image projected from one prediction pose of the plurality of terminal prediction poses; selecting one prediction pose from among the plurality of terminal prediction poses; performing spatial geometric transformation on each part of the two-dimensional image in a three-dimensional space according to information of the selected prediction pose; and transforming, on the basis of a final terminal pose, the spatial geometric-transformed two-dimensional image from a space of the selected prediction pose to a space of the final terminal pose, thereby generating a final two-dimensional image.