HMD Error Concealment Using Contextual Buffer Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual, augmented, and mixed reality (xR) applications in connectivity-constrained environments face challenges with forward channel errors, such as motion data and synchronization errors, which affect the quality of the immersive experience due to limited processing capabilities in Head-Mounted Devices (HMDs) and the need for efficient error concealment and synchronization techniques.
Innovation Solution
The implementation of contextual error concealment and synchronization methods in HMDs, utilizing Simultaneous Localization And Mapping (SLAM) sensors, Internal Measurement Units (IMU), and buffers like Motion Smoothing (MS) and Asynchronous Reprojection (ARP) to handle motion data errors and synchronization issues, ensuring seamless video and audio experiences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If error concealment and synchronization techniques are implemented in HMDs, then the quality of xR content is improved, but the device complexity increases
Solution Approach 1:
The system performs preliminary error detection and concealment actions by maintaining multiple buffers (display buffer, motion smoothing buffer, asynchronous reprojection buffer) that are pre-filled with decoded video data. When errors are detected in incoming data, the system can immediately switch to using pre-prepared data from these buffers without requiring complex real-time processing, thus improving reliability while managing device complexity.
Solution Approach 2:
The patent introduces intermediary buffer structures (motion smoothing buffer and asynchronous reprojection buffer) that mediate between the incoming encoded video data and the display output. These buffers act as intermediaries that can provide error-concealed data when the primary data stream contains errors, improving content quality without requiring the HMD processor to perform complex real-time error correction.
2Reliability
If multiple buffers and contextual information processing are used, then error concealment effectiveness is improved, but processing requirements increase
Solution Approach 1:
The system changes the parameter of data readiness by maintaining multiple buffers with different levels of processing completion. The display buffer contains fully decoded data, the motion smoothing buffer contains data with motion compensation applied, and the asynchronous reprojection buffer contains data with timing adjustments. This parameter variation allows the system to select the most appropriate buffer based on error conditions without requiring complex real-time processing.
Solution Approach 2:
The patent applies partial error concealment by using multiple buffers that provide different levels of error protection. Rather than implementing complex real-time error correction for all data, the system uses pre-prepared buffers that provide sufficient error concealment for most scenarios, reducing processing requirements while maintaining effectiveness.
3Ease of operation
If real-time error handling is implemented, then user experience is improved, but synchronization complexity increases
Solution Approach 1:
The system performs preliminary synchronization by maintaining buffers that are pre-filled with data at different timing offsets. The motion smoothing buffer contains data with motion compensation pre-calculated, and the asynchronous reprojection buffer contains data with timing adjustments pre-applied. This allows real-time error handling without complex synchronization calculations during active display.
Solution Approach 2:
The patent uses copying by maintaining multiple copies of video data in different buffers with different processing states. When errors are detected, the system copies data from an alternative buffer rather than performing complex real-time reconstruction, improving user experience while managing synchronization complexity.
Data Source
AI summary
Embodiments of systems and methods for forward channel contextual error concealment and sync for virtual, augmented, or mixed reality (xR) content in connectivity-constrained environments are described. In some embodiments, a Head-Mounted Device (HMD) may include a processor; and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution by the processor, cause the HMD to: decode content received from an Information Handling System (IHS) coupled to the HMD; and handle an error in the decoded content using context information received from a sensor mounted on the HMD.