Segmented Gather Cache With Parallel Hit Logic for Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer architectures face limitations in memory access performance and bandwidth due to the time and energy required for data transfer between processors and memory, constraining compute performance beyond transistor scaling.
Innovation Solution
Implementing a compute-near-memory (CNM) system with a high bandwidth cache that utilizes multiple cache segments and independent hit logic units to facilitate parallel cache operations, reducing cache-associated delays for workloads requiring multiple memory locations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional shared memory architecture is used, then system simplicity is maintained, but memory access performance and bandwidth are limited
Solution Approach 1:
The patent divides the cache into multiple segments (first cache segment, second cache segment, etc.) with each segment having independent hit logic units. This segmentation allows parallel processing of multiple memory addresses simultaneously, increasing memory bandwidth and access performance without requiring a complete architectural redesign.
Solution Approach 2:
The patent introduces a multi-dimensional cache structure where multiple cache segments are organized in parallel, each accessible through independent paths. This dimensional expansion from a single-cache structure to a multi-segment parallel structure enables simultaneous memory accesses and resolves the bandwidth limitation.
2Productivity
If multiple memory locations are accessed simultaneously, then workload throughput increases, but cache hit detection complexity increases
Solution Approach 1:
The hit logic is segmented into multiple independent hit logic units, each responsible for detecting cache hits for specific memory addresses or address ranges. This segmentation enables parallel hit detection for multiple locations simultaneously, increasing throughput while keeping each individual hit logic unit relatively simple.
Solution Approach 2:
The patent implements more hit logic units than the number of simultaneous memory accesses required, allowing the system to process multiple addresses in parallel. This excessive provision of hit logic units ensures that all possible simultaneous accesses can be handled without bottlenecking, maximizing throughput.
3Speed
If data transfer between processors and memory is performed, then data access is enabled, but time and energy consumption increase
Solution Approach 1:
The cache stores frequently accessed data in advance, allowing processors to access this data locally without waiting for transfers from main memory. By pre-loading and caching data, the system reduces the frequency and duration of high-energy data transfer operations between processors and memory.
Solution Approach 2:
The multi-segment cache acts as an intermediary between processors and main memory, providing a high-speed buffer that reduces the burden on the memory bus. This intermediary structure enables faster data access for frequently used data while minimizing the energy-intensive transfer operations required for less frequently accessed data.
Data Source
AI summary
Disclosed in some examples are methods, systems, and machine readable mediums that provide increased bandwidth caches to process requests more efficiently for more than a single address at a time. This increased bandwidth allows for multiple cache operations to be performed in parallel. In some examples, to achieve this bandwidth increase, multiple copies of the hit logic are used in conjunction with dividing the cache into two or more segments with each segment storing values from different addresses. In some examples, the hit logic may detect hits for each segment. That is, the hit logic does not correspond to a particular cache segment. Each address value may be serviced by any of the plurality of hit logic units.