Systolic Storage Tile for Cache Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cache memory systems face a trade-off between access time and the number of frequently-accessed values, with larger L1 caches increasing access time and smaller caches increasing the number of misses, leading to slower higher-order caches and inefficient request handling.
Innovation Solution
A storage array tile design that supports systolic move-to-front and push-backward operations without global control, using a spiral cache architecture with distributed control logic to dynamically reorganize values and reduce worst-case access latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the L1 cache size is increased to store more frequently-accessed values, then the number of frequently-accessed values available at short access times is improved, but the access time increases due to physical wiring constraints and signal propagation speed limits
Solution Approach 1:
The cache is divided into multiple independent tiles that can operate in parallel. Each tile is a self-contained unit with its own storage locations and control logic, allowing the system to handle multiple access requests simultaneously without increasing the access time for individual tiles.
Solution Approach 2:
The patent transitions from a traditional single-cache-hierarchy model to a two-dimensional tiled array architecture. This spatial reorganization allows values to be distributed across multiple tiles, enabling parallel access operations and reducing the effective access time through dimensional expansion of the cache structure.
2Speed
If the L1 cache size is reduced to decrease access time, then the access time is improved, but the number of frequently-accessed values that can be stored decreases, leading to more cache misses
Solution Approach 1:
By segmenting the cache into multiple smaller tiles, each tile can maintain fast access times while the collective array of tiles provides increased total storage capacity for frequently-accessed values.
Solution Approach 2:
The patent implements replication of cache functionality across multiple tiles, where each tile contains duplicate control logic and storage structures. This allows the system to serve multiple values simultaneously through parallel tile operations, effectively increasing capacity without proportionally increasing access time.
3Device complexity
If traditional cache control algorithms are used to handle one outstanding request at a time, then the control logic is simple, but the productivity decreases due to stalled pipelines and severe performance penalties
Solution Approach 1:
The control logic is segmented and distributed across individual tiles rather than centralized. Each tile independently handles its own access requests, enabling parallel processing of multiple requests simultaneously without requiring complex centralized arbitration or stalling mechanisms.
Solution Approach 2:
Each tile is self-sufficient with embedded control logic that autonomously manages its own storage locations and access requests. This self-service capability eliminates the need for complex global control algorithms and allows simultaneous independent handling of multiple requests across different tiles.
4Quantity of substance
If higher-order cache levels are used to store values not in L1 cache, then the storage capacity is improved, but the access time increases by 10-100 times due to propagation speed constraints
Solution Approach 1:
The patent creates a parallel cache structure through tiled arrays that operates at the same hierarchical level as traditional L1 cache, providing extended capacity without the latency penalty of higher-order cache levels. This dimensional approach allows fast access to a larger set of frequently-used values.
Solution Approach 2:
By dividing the extended cache into multiple tiles with local control logic, the system provides large storage capacity distributed across many small, fast-access units, avoiding the need for a single large slow cache or multiple hierarchical levels.
Data Source
AI summary
A tile for use in a tiled storage array provides re-organization of values within the tile array without requiring sophisticated global control. The tiles operate to move a requested value to a front-most storage element of the tile array according to a global systolic clock. The previous occupant of the front-most location is moved or swapped backward according to the systolic clock, and the new occupant is moved forward according to the systolic clock, according to the operation of the tiles, while providing for multiple in-flight access requests within the tile array. The placement heuristic that moves the values is determined according to the position of the tiles within the array and the behavior of the tiles. The movement of the values can be performed via only next-neighbor connections of adjacent tiles within the tile array.