Hierarchical Modulation XR Split Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing wireless communication systems, particularly in 5G NR, face challenges in efficiently processing and rendering graphical content for extended reality (XR) applications, leading to high power consumption and processing overhead in devices like head-mounted displays (HMDs).
Innovation Solution
The implementation of a wireless communication system that utilizes hierarchical modulation and foveated rendering in a split or hybrid processing system. This system involves obtaining a hierarchical modulated image from a network entity and using gaze information from a second device to process and prioritize the rendering of the image, focusing on higher spatial and temporal resolutions in the foveal region while reducing processing in other areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If full-resolution rendering is performed at the HMD for XR applications, then image quality is improved, but power consumption and processing overhead increase significantly
Solution Approach 1:
The patent applies foveated rendering to provide different image qualities in different regions of the display. The foveal region (center of vision) receives high-resolution rendering while peripheral regions receive lower-resolution rendering. This resolves the contradiction by maintaining high image quality where the user actually looks while reducing overall processing requirements and power consumption.
Solution Approach 2:
The patent segments the image processing workload between two devices: the HMD performs lightweight processing and transmission of hierarchical modulated images, while a separate computing device performs the complex processing of low-priority bits and foveated rendering. This segmentation reduces the processing burden and power consumption at the HMD while maintaining high image quality.
2Manufacturing precision
If high-resolution graphical content is processed and transmitted wirelessly for XR, then visual fidelity is improved, but bandwidth consumption and latency increase
Solution Approach 1:
The patent transmits only the essential high-priority bits required for basic image rendering at the HMD, while deferring the processing and transmission of low-priority bits (which enhance visual fidelity) to a separate computing device. This partial action approach reduces bandwidth consumption and latency for the critical path while still achieving high visual fidelity through subsequent processing of low-priority bits.
3Speed
If the HMD performs all graphics processing locally, then processing speed is improved, but device complexity and power requirements increase
Solution Approach 1:
The patent segments the graphics processing pipeline into two parts: the HMD handles high-priority bit processing and image transmission, while a separate computing device handles low-priority bit processing and foveated rendering. This segmentation allows each device to have optimized, simpler processing requirements while maintaining overall high processing speed through coordinated operation.
4Quantity of substance
If hierarchical modulation is used to prioritize image data transmission, then bandwidth efficiency is improved, but processing complexity at the receiving device increases
Solution Approach 1:
The patent uses hierarchical modulation to segment image data into high-priority and low-priority bits. The HMD processes only the simpler high-priority bits for immediate rendering, while a separate computing device handles the more complex low-priority bit processing. This segmentation distributes processing complexity across multiple devices, reducing the burden on any single device while maintaining bandwidth efficiency.
Data Source
AI summary
Aspects are provided including one or more techniques for graphics processing involving hierarchical modulation and foveated rendering. A first wireless device obtains, from a network entity, a hierarchical modulated image including higher and lower priority bits. The first wireless device also obtains gaze information from a second wireless device indicating an associated foveal region. In response to the gaze information, the first wireless device processes a portion of the hierarchical modulated image associated with the lower priority bits using the gaze information. For example, the device decodes low priority bits in the hierarchical modulated image in regions closer to the fovea. The first wireless device subsequently provides an image including the processed portion to be rendered at the second wireless device. For example, the processed image may have a higher resolution in the regions closer to the fovea but a lower resolution in regions farther from the fovea.