Edge XR Task Allocation for Low-Latency Lightweight Clients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR technologies face challenges in maintaining low latency and delivering high-quality user experiences on lightweight client devices due to limitations in offloading processing to server sides, power consumption, computing performance, and high latency, as well as issues with initialization and visual tracking methods.
Innovation Solution
A distributed XR computing system utilizing an edge node that optimizes processing distribution by assessing network and device metrics to determine optimal function allocation between the edge node and client devices, including methods for visual tracking and rendering, thereby enhancing user experience on lightweight devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If full XR processing is offloaded to a centralized server, then computing requirements of the client device are reduced, but latency between the client and the edge node becomes undesirably high
Solution Approach 1:
The patent segments XR processing functions into multiple categories (visual tracking, device pose tracking, rendering, initialization) and distributes them across different locations (client device and edge node) based on processing intensity and latency requirements. This segmentation allows lightweight client devices to offload intensive tasks while maintaining low-latency responsiveness for critical functions.
Solution Approach 2:
The patent introduces an edge node as an intermediary between the centralized server and client devices. This edge node performs intermediate processing of XR data, reducing the latency burden on both the server and client devices while maintaining the benefits of cloud-based rendering and tracking services.
2Use of energy by moving object
If XR processing is offloaded to an edge node, then client device power consumption is reduced, but computing performance of the edge node becomes a limiting factor
Solution Approach 1:
The patent implements dynamic processing distribution where the edge node adaptively adjusts which XR processing functions are performed locally versus offloaded based on real-time assessment of network conditions, edge node computing performance, and client device capabilities. This dynamic approach optimizes the balance between power consumption and processing performance.
Solution Approach 2:
The system changes operational parameters (processing location, data transmission volume, feature descriptor extraction methods) based on assessed performance metrics of the edge node and network conditions, allowing the system to adapt to varying computing performance capabilities of different edge nodes.
3Quantity of substance
If simplified feature descriptor extraction is used, then data transmission bandwidth is reduced, but accuracy of visual tracking deteriorates due to sensitivity to sensor characteristics and lighting changes
Solution Approach 1:
The patent applies partial feature descriptor extraction where only essential visual features are extracted and transmitted to the edge node, rather than transmitting full image data. This partial action reduces bandwidth consumption while maintaining sufficient tracking accuracy through selective feature extraction that is optimized for the specific application requirements.
Solution Approach 2:
Instead of transmitting original full-resolution images, the system creates and transmits simplified copies in the form of feature descriptors (such as ORB features). These copied representations retain the essential visual information needed for tracking while occupying minimal bandwidth.
4Measurement precision
If deep learning is applied to visual feature extraction, then tracking accuracy is improved, but real-time performance lags compared to traditional SLAM systems
Solution Approach 1:
The patent segments the feature extraction process into traditional SLAM-based extraction (for speed) and deep learning-based verification/refinement (for accuracy). This segmentation allows the system to maintain real-time performance through efficient traditional methods while incorporating deep learning enhancements where computational resources are available.
Solution Approach 2:
The system applies different quality levels of feature extraction to different parts of the visual processing pipeline. Critical real-time tracking uses lightweight traditional methods, while non-critical enhancement and verification tasks use more computationally intensive deep learning approaches, creating local quality variations optimized for different functional requirements.
Data Source
AI summary
Methods and systems are described for distributed extended reality (XR) computing optimization for an edge node connectable to a client device and an XR content server across a network. XR computing is apportioned to the edge node and the client device so that a wide variety of client devices, from light to heavy, are accommodated based on metrics of the edge node, the client device, the XR content server, and the network. Some of the apportionments include extraction of feature descriptors by the client device to avoid transmitting full image data from the client device to the edge node. The feature descriptors are used for tracking initialization and frame to frame tracking. Overall system performance is improved delivering an improved user experience. Related apparatuses, devices, techniques, and articles are also described.