Shared Feature Tensor Caching for Multitask XR Decoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The deployment and execution of multiple concurrent machine learning models in extended reality (XR) applications impose significant resource demands, particularly in terms of memory access bandwidth and power consumption, due to frequent context switching and external memory access.
Innovation Solution
Implementing a multitask machine learning model with a common feature encoder that caches universal feature tensors in local memory, allowing multiple tasks to access and decode these tensors without external memory access, using an encoding algorithm with specified spatial resolution and frame rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple concurrent ML models are deployed for XR applications, then task functionality and user experience are improved, but memory bandwidth consumption and power usage increase significantly
Solution Approach 1:
The patent merges the feature encoding functionality into a shared common encoder that is jointly used by multiple decoder tasks. This consolidation eliminates redundant feature extraction operations across different tasks, reducing overall computational load and power consumption while maintaining the ability to perform multiple ML tasks concurrently
Solution Approach 2:
The common encoder is designed as a universal component that serves multiple decoder tasks simultaneously. By making the feature encoding capability universal and shared across tasks rather than task-specific, the system reduces the total number of encoding operations needed, thereby lowering power consumption and resource usage
2Adaptability or versatility
If multiple concurrent ML models are deployed for XR applications, then task functionality is improved, but external memory access frequency increases
Solution Approach 1:
The system performs preliminary feature encoding by the common encoder and stores the encoded features in local memory before they are needed by multiple decoder tasks. This advance preparation eliminates the need for repeated external memory accesses during task execution, as decoders can retrieve pre-encoded features from local memory
Solution Approach 2:
The patent introduces local memory as an intermediary buffer between the common encoder and multiple decoder tasks. This intermediary storage layer allows encoded features to be cached locally, reducing direct external memory access requirements and bandwidth consumption while enabling efficient data sharing among tasks
3Speed
If context switching between multiple ML tasks is performed frequently, then task responsiveness is improved, but memory access latency increases
Solution Approach 1:
By merging the feature encoding operation into a single common encoder that serves all tasks, the system eliminates redundant encoding operations during context switching. The shared encoder maintains feature encoding state across tasks, reducing the overhead and latency associated with switching between multiple independent model instances
Data Source
AI summary
Aspects of the disclosure are directed to multitask machine learning (ML). In accordance with one aspect, the disclosure includes executing, by a first machine learning (ML) task, a machine learning (ML) feature encoding of a selected feature to generate a common feature tensor, without an external memory access, accessing, by a second machine learning (ML) task, the common feature tensor from a local non-transitory memory; and decoding the common feature tensor for completion of the second ML task.