ML Accelerator Virtual Memory for Shared SRAM Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning accelerators in ambient computing devices face challenges due to high memory requirements, leading to cost-prohibitive memory allocation, wasted memory, and excessive power consumption, especially in devices designed for low-power consumption.
Innovation Solution
Implementing virtual memory addresses to access external memory, allowing the ML accelerator to simulate local memory access, thereby reducing the need for dedicated large memory and minimizing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If large amounts of memory are allocated to the ML accelerator to accommodate larger machine learning models, then the ML accelerator can process more complex models, but the memory cost becomes prohibitive and physical space is wasted
Solution Approach 1:
The patent merges the ML accelerator's memory requirements with the device's main memory system by implementing a shared memory architecture. The ML accelerator accesses memory through a memory interface that allows it to use the device's existing memory resources, eliminating the need for separate dedicated memory allocation. This combining approach enables the ML accelerator to access large memory capacities without requiring physical memory dedicated solely to it.
Solution Approach 2:
The memory system is designed to serve multiple functions and multiple processing components simultaneously. The same memory resource can be allocated to the ML accelerator when needed for model processing, while also being available for other device operations. The memory interface enables dynamic allocation and sharing, making the memory system universal rather than dedicated to a single function.
2Adaptability or versatility
If large amounts of memory are allocated to the ML accelerator, then larger machine learning models can be processed, but power consumption increases because memory not in use still draws power
Solution Approach 1:
By merging the ML accelerator's memory needs with the device's main memory system, the patent eliminates idle memory allocation. The shared memory architecture ensures that memory is only powered and active when actually being used by the ML accelerator or other processing components, preventing the wasteful power consumption of dedicated memory that remains allocated but unused.
Solution Approach 2:
The memory allocation is made dynamic rather than static. The memory interface allows the ML accelerator to access memory on-demand based on actual processing needs, enabling the system to allocate memory resources dynamically during operation. This dynamic approach ensures memory is active only when needed, reducing power consumption compared to static pre-allocation.
3Productivity
If memory is allocated to the ML accelerator, then machine learning models can be processed, but the allocated memory is not available to other processing components of the device
Solution Approach 1:
The patent merges the ML accelerator's memory access with the device's main memory system through a shared memory interface. This merging allows multiple processing components to access the same memory resource simultaneously or sequentially, enabling memory sharing rather than exclusive allocation. The ML accelerator can process models while other components retain access to the same memory pool.
Solution Approach 2:
The memory system is designed as a universal resource that can serve multiple processing components for different functions. The memory interface enables the same memory to be allocated to the ML accelerator for model processing while also being available for other device operations, making the memory system multi-functional rather than dedicated to a single component.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for virtualizing external memory as local to a machine learning accelerator. One ambient computing system comprises: an ambient machine learning engine; a low-power CPU; and an SRAM that is shared among at least the ambient machine learning engine and the low-power CPU; wherein the ambient machine learning engine comprises virtual address logic to translate from virtual addresses generated by the ambient machine learning engine to physical addresses within the SRAM.