Segmented Gather Cache with Parallel Hit Logic for Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer architectures face significant challenges in moving data between processors and memory, leading to constrained performance and capacity due to limited memory bandwidth access, particularly in compute-intensive operations like data flow computing.
Innovation Solution
Implementing a high bandwidth cache system with multiple cache segments and independent hit logic units to facilitate parallel cache operations, allowing simultaneous or near-simultaneous access to multiple memory locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If conventional shared memory architecture is used, then system simplicity is maintained, but memory bandwidth is limited and access time increases
Solution Approach 1:
The cache system is divided into multiple independent cache segments (first cache segment, second cache segment, etc.), each capable of independent operation. This segmentation allows parallel memory access operations to proceed simultaneously across different segments, thereby increasing memory bandwidth and access speed without requiring a complete redesign of the entire memory architecture.
Solution Approach 2:
The patent introduces multiple cache segments operating in parallel dimensions, transforming sequential memory access into parallel operations. By organizing cache segments that can be accessed simultaneously, the system adds a temporal and spatial dimension to memory access, effectively increasing throughput without proportionally increasing complexity.
2Productivity
If single cache architecture is used, then device complexity is low, but cache bandwidth is insufficient for compute-intensive operations
Solution Approach 1:
The cache is segmented into multiple independent units (first cache segment, second cache segment) that can operate in parallel. Each segment can service different memory requests simultaneously, thereby increasing the overall cache bandwidth and supporting higher throughput for compute-intensive operations like data flow computing and neural network processing.
Solution Approach 2:
Multiple cache segments are merged into a unified cache system that operates in parallel. The segments share common control logic and memory hierarchy but maintain independent data paths, combining their individual bandwidth capabilities to achieve high throughput while managing complexity through modular integration.
3Loss of time
If sequential cache access is used, then access control is simple, but latency increases due to sequential processing
Solution Approach 1:
The cache access control is segmented into multiple independent controllers or logic units, each managing a specific cache segment. This allows simultaneous access control decisions to be made for different segments in parallel, reducing the time required to service multiple memory requests without requiring complex centralized arbitration.
Solution Approach 2:
Multiple cache segments enable continuous parallel memory access operations, ensuring that useful action (data retrieval) continues without interruption across different segments. This maintains high throughput and reduces latency by keeping all cache segments actively servicing requests simultaneously rather than sequentially.
Data Source
AI summary
Disclosed in some examples are methods, systems, and machine readable mediums that provide increased bandwidth caches to process requests more efficiently for more than a single address at a time. This increased bandwidth allows for multiple cache operations to be performed in parallel. In some examples, to achieve this bandwidth increase, multiple copies of the hit logic are used in conjunction with dividing the cache into two or more segments with each segment storing values from different addresses. In some examples, the hit logic may detect hits for each segment. That is, the hit logic does not correspond to a particular cache segment. Each address value may be serviced by any of the plurality of hit logic units.