Dynamic Performance Adjustment for Split XR Applications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In split extended reality (XR) applications, the fixed performance settings in the rendering pipeline lead to sub-optimal display performance due to issues like frame repeats, misses, and increased motion-to-render-to-photon (M2R2P) latencies, resulting in a less than optimal user experience.
Innovation Solution
Implementing dynamic performance adjustments and dynamic content quality adjustments using machine learning models to predict and adjust frame processing times, workload distribution, and content resolution, allowing for real-time optimization of the rendering pipeline.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If fixed performance settings are used in the rendering pipeline, then device complexity is reduced and ease of operation is improved, but display performance deteriorates due to frame repeats, misses, and increased M2R2P latencies
Solution Approach 1:
The patent implements dynamic performance adjustments that allow the rendering pipeline to adapt its performance settings in real-time based on actual frame processing times and system conditions. This transforms the static fixed performance settings into dynamic adjustable parameters, resolving the contradiction by enabling both ease of operation (through automated adaptation) and improved display performance (through real-time optimization).
Solution Approach 2:
The system incorporates feedback mechanisms that monitor actual frame processing times and use this information to adjust performance settings for subsequent frames. This closed-loop feedback approach enables the system to automatically optimize display performance while maintaining ease of operation, as the adjustments are driven by actual performance data rather than manual intervention.
2Reliability
If dynamic performance adjustments are implemented using machine learning models, then display performance and user experience are improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent introduces machine learning models as intermediary components that bridge the gap between raw performance data and actionable adjustments. These models process frame processing time data and generate performance adjustment recommendations, thereby managing the complexity of dynamic optimization while delivering improved user experience. The intermediary nature of the ML models allows the system to handle complexity in a modular fashion.
3Manufacturing precision
If frame processing times are extended to improve quality, then content quality is improved, but M2R2P latencies increase and user experience deteriorates
Solution Approach 1:
The system dynamically adjusts frame processing parameters based on actual performance data, allowing it to optimize the balance between content quality and latency in real-time. Rather than using fixed extended processing times, the system adapts processing intensity based on current system conditions and performance requirements, thereby improving user experience while maintaining quality where possible.
Solution Approach 2:
The patent employs parameter changes in the rendering pipeline, adjusting performance settings such as resolution, texture quality, and processing intensity based on monitored frame processing times. This allows the system to reduce processing time and latency when needed while maintaining high content quality when system conditions permit, effectively resolving the contradiction between quality and speed.
Data Source
AI summary
Aspects presented herein relate to methods and devices for frame processing including an apparatus, e.g., client or server. The apparatus may estimate a set of frame processing times for at least one first frame in a set of frames. The apparatus may also detect a set of actual frame processing times for the at least one first frame in the set of frames. The apparatus may also output, based on the set of estimated frame processing times and the set of actual frame processing times, an indication to adjust a set of second frame processing times for at least one second frame in the set of frames, where the at least one second frame is subsequent to the at least one first frame in the set of frames.