Multi-Region Cache Memory for Sparse Matrix Outer-Product Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing devices face inefficiencies in performing outer-product computing algorithms with sparse matrices due to irregular memory access sequences, leading to excessive data traffic and suboptimal use of cache systems.
Innovation Solution
A computing device with a cache memory optimized for outer-product algorithms, featuring a first N-way set associative memory region for dense regions and a second fully associative or M-way set associative memory region for sparse regions, along with a computing circuit to perform reduction operations, and a management system to balance data traffic and reduce main memory accesses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If outer-product computing algorithms are used for sparse matrices, then the number of memory accesses is reduced, but irregular memory access sequences cause excessive data traffic to main memory
Solution Approach 1:
The cache memory is divided into multiple regions with different associativity characteristics. The first region uses N-way set associative mapping while the second region uses fully associative or M-way set associative mapping (where M≥N). This segmentation allows different types of memory accesses to be handled by appropriately optimized regions, reducing overall data traffic to main memory while maintaining high computing productivity.
Solution Approach 2:
Different regions of the cache memory are assigned different associativity levels tailored to specific access patterns. The first region is optimized for certain types of accesses while the second region handles irregular accesses with higher associativity. This local optimization reduces data traffic for each access pattern type, resolving the contradiction between computing speed and energy loss.
2Adaptability or versatility
If conventional cache systems are used, then hardware complexity is kept low, but they are not well adapted to processing irregular memory access sequences
Solution Approach 1:
The cache memory is segmented into multiple regions with different associativity characteristics. The first region uses N-way set associative mapping while the second region uses fully associative or M-way set associative mapping (where M≥N). This segmentation allows the system to handle both regular and irregular memory access patterns efficiently, improving adaptability without excessive complexity increase.
Solution Approach 2:
The multi-region cache structure serves multiple functions: the first region handles regular access patterns with lower complexity, while the second region handles irregular access patterns with higher associativity. This multi-functionality allows a single cache system to adapt to various access patterns, resolving the contradiction between adaptability and device complexity.
3Quantity of substance
If CSR or CSC storage formats are used, then memory size is reduced, but browsing tables with indices increases processing complexity
Solution Approach 1:
The cache memory acts as an intermediary between the compressed sparse matrix storage format and the processing units. By maintaining cached copies of matrix elements and their indices in the multi-region cache structure, the system reduces the need to repeatedly access the compressed storage format, thereby reducing processing complexity while maintaining compact storage.
4Stability of the object's composition
If inner-product algorithms are used, then regular memory access is achieved, but more memory accesses are required compared to outer-product algorithms
Solution Approach 1:
The cache memory is segmented into multiple regions with different associativity characteristics. The first region uses N-way set associative mapping while the second region uses fully associative or M-way set associative mapping (where M≥N). This segmentation allows the system to handle both regular and irregular memory access patterns efficiently, resolving the contradiction between computing speed and energy loss.
Data Source
AI summary
The present description concerns a computing device comprising a computing unit; a main memory; a cache memory configured to exchange data with the computing unit and the main memory, and comprising a circuit for calculating reduction operations between partial products derived from values of a sparse matrix and an input vector, and an output vector, wherein the cache memory comprises a first N-way set associative memory region storing, with a first word granularity TD, values of results of reduction operations performed by the computing circuit based on partial products derived from values of a dense region of the matrix, and a second fully associative or M-way set associative memory region storing, with a second word granularity TS, values of results of reduction operations performed by the computing circuit based on partial products derived from values of a sparse region of the matrix, with M≥N, TD≥TS, and also M≥N if TD=TS and TD>TS if M=N.