Cache Memory Partitioning for Sparse Matrix Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computing devices struggle with optimizing sparse matrix multiplication operations due to inefficient memory access patterns, particularly when dealing with large sparse matrices, leading to high data traffic and suboptimal performance of outer-product algorithms.
Innovation Solution
A computer computing system with a cache memory configured to handle sparse matrix operations by segregating dense and sparse regions, using N-way and M-way associative memory regions with specific granularities, and implementing reduction operations efficiently through a combination of read/write and atomic operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If outer-product algorithms are used to perform sparse matrix-vector multiplication, then the number of memory accesses is reduced, but the performance deteriorates due to irregular memory access patterns and high data traffic to main memory
Solution Approach 1:
The patent segments the cache memory into multiple associative memory regions (first N-way associative region, second fully/M-way associative region, third region) to handle different types of data and access patterns. This segmentation allows each region to be optimized for specific operations, resolving the contradiction by enabling efficient memory access patterns while maintaining reduced access count benefits of outer-product algorithms.
Solution Approach 2:
The patent applies local quality by assigning different memory access characteristics to different cache regions. The first region handles dense matrix regions with N-way associativity, the second region handles sparse matrix regions with fully/M-way associativity, and the third region serves as a buffer. This local optimization enables the system to maintain both reduced memory accesses and high performance by matching memory access patterns to appropriate regional characteristics.
2Device complexity
If conventional cache memory is used for sparse matrix operations, then the device complexity is low, but the productivity deteriorates due to inefficient handling of irregular memory access sequences
Solution Approach 1:
The patent segments the cache memory into multiple specialized regions with different associativity levels and access characteristics. This segmentation increases device complexity but enables efficient handling of irregular memory access sequences by directing different data types to appropriate regions, thereby resolving the contradiction between structural simplicity and processing efficiency.
Solution Approach 2:
The patent introduces dynamic characteristics through multiple associative memory regions that can handle different access patterns flexibly. The system dynamically routes memory accesses to appropriate regions based on data characteristics, enabling high productivity while maintaining manageable complexity through structured organization of the dynamic memory hierarchy.
3Manufacturing precision
If dense regions of sparse matrices are processed separately from sparse regions, then the manufacturing precision of memory access patterns improves, but the device complexity increases due to separate memory regions and granularities
Solution Approach 1:
The patent segments the cache memory into specialized regions for dense and sparse matrix processing with different granularities. The first N-way associative region handles dense regions with finer granularity for regular access patterns, while the second fully/M-way associative region handles sparse regions with coarser granularity. This segmentation achieves manufacturing precision in memory access patterns while managing device complexity through hierarchical organization.
Solution Approach 2:
The patent applies local quality by optimizing each memory region for specific matrix regions. The first region is optimized for dense matrix regions with regular access patterns, while the second region is optimized for sparse matrix regions with irregular access patterns. This localized optimization achieves high precision for each region's access patterns while the overall system manages complexity through clear functional differentiation.
Data Source
Figure 1~2
Figure 3a~3i
Figure 4a~4j
AI summary
The present description relates to a computer computing device (100), comprising: - a computing unit (102); - a main memory (104); - a cache memory (106) configured to exchange data with the computing unit and the main memory, and comprising a computing circuit (108) for reduction operations between partial products from values of a sparse matrix (118) and an input vector, and an output vector;in which the cache memory includes a first N-way associative memory region (112) storing, with a first granularity of words TD, results of reduction operations performed by the computation circuit from partial products from values in a dense region (120) of the matrix, and a second memory region (114) that is completely associative or M-way associative storing, with a second granularity of words TS, results of reduction operations performed by the computation circuit from partial products from values in a sparse region (124) of the matrix, with M ≥ N, TD ≥ TS, and also M > N if TD = TS and TD > TS if M = N.;