Distributed Temporal Cache for SoC Memory Sharing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computing systems, particularly artificial reality systems, there is a challenge in efficiently sharing on-die memory distributed among compute elements as local memory, leading to issues with latency and energy consumption when sharing memory across different subsystems, especially when some subsystems are not continuously powered on.
Innovation Solution
Implementing a set-associative cache system that maps cacheable memory locations to multiple local memories, allowing data blocks to be fetched from either the cache or the memory based on availability, and using a TAG memory to manage data storage and retrieval efficiently, thereby reducing the need for frequent data copying and minimizing energy expenditure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If on-die SRAM is distributed as local memory close to compute elements, then bandwidth and latency are improved, but memory sharing capability deteriorates
Solution Approach 1:
The SRAM is segmented into multiple distributed local memory banks, each associated with specific compute elements. This segmentation allows each local memory to serve its nearby compute elements with low latency while the cache controller manages sharing across subsystems by selecting appropriate local memories for cache operations.
Solution Approach 2:
A cache controller is introduced as an intermediary between the distributed local memories and compute elements. The cache controller receives memory requests, determines which local memory contains the requested data, and retrieves data from the appropriate local memory or system memory, enabling efficient sharing without compromising local access performance.
2Adaptability or versatility
If distributed local memory is shared with remote compute elements, then memory sharing capability is improved, but latency increases
Solution Approach 1:
Different quality levels of memory access are provided based on location: local compute elements access their associated local memory with low latency, while remote compute elements access the same local memory through the cache controller with higher latency. This local quality approach allows memory sharing while maintaining optimal performance for local accesses.
3Use of energy by moving object
If narrow connections are used to share distributed memory, then energy consumption is reduced, but bandwidth decreases
Solution Approach 1:
The connection width and activation of memory sharing pathways are made dynamic. When a local memory is not being used by its associated compute elements, the cache controller activates connections to share that memory with remote compute elements. The connection width is adjusted based on the current usage state, providing high bandwidth when needed and low energy consumption when memory is idle.
Data Source
AI summary
A system and method for accessing cache lines of an N-way set associative cache distributed across local memory of compute elements. The set associative cache includes a plurality of sets, with each location in cacheable local memory mapped to one of the sets and each set including N locations for caching data blocks read from the cacheable memory. Each set is mapped to one of the local memories, when that local memory is not in use by local compute elements. A cache controller is configured to receive a read request, to identify a data block in the cacheable memory associated with the address, to determine if the identified data block is in cache in one of the local memories, and, if the identified data block is in cache in one of the local memories, to fetch the identified data block from the cache.