PIM Architecture Hybrid Partitioning Sparse Matrix Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing PIM architectures are inefficient in supporting sparse matrix operations like SpMV and SpMSpV due to limitations in handling random accesses and power-law column length distributions, leading to data movement overhead and load imbalance.
Innovation Solution
The implementation of a Processing-in-Memory (PIM) architecture that includes accumulation dispatching and hybrid partitioning, enabling column-oriented processing and reducing remote accumulations and load imbalance by distributing long columns among all memory segments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If row-oriented processing is used in PIM architecture, then simple matrix operations can be performed, but sparse matrix operations like SpMV and SpMSpV cannot be efficiently supported
Solution Approach 1:
The patent divides the processing architecture into two distinct modes: row-oriented processing units for dense matrix operations and column-oriented processing units for sparse matrix operations. This segmentation allows each unit to be optimized for its specific function, resolving the contradiction between simplicity and versatility.
Solution Approach 2:
The PIM architecture dynamically switches between row-oriented and column-oriented processing modes based on the operation type. The system can adaptively select the appropriate processing mode for SpMV, SpMSpV, or dense matrix operations, providing both simplicity when needed and versatility when required.
2Power
If data is transferred from memory to external accelerators, then computation can be performed, but data movement overhead increases
Solution Approach 1:
The patent implements self-service by enabling computation directly within the memory device. The PIM architecture performs sparse matrix operations in-place within the memory array, eliminating the need to transfer data to external accelerators and thereby reducing data movement overhead and energy consumption.
Solution Approach 2:
The patent introduces an intermediary column buffer structure within the memory device that facilitates in-place column-oriented processing. This intermediary structure enables the memory device to perform computation internally without requiring external accelerator involvement, reducing energy loss from data transfer.
3Ease of operation
If long columns are concentrated in single memory segments, then column-oriented processing can be simplified, but load imbalance occurs
Solution Approach 1:
The patent segments long columns across multiple memory segments rather than concentrating them in a single segment. Each memory segment handles a portion of the column data, which distributes the computational load evenly and prevents load imbalance while maintaining column-oriented processing efficiency.
Solution Approach 2:
The patent applies local quality by assigning different portions of long columns to different memory segments based on their capacity and workload. Each segment processes its assigned portion with optimized local operations, maintaining simplicity while achieving balanced load distribution across the system.
4Adaptability or versatility
If remote accumulations are performed frequently, then column-oriented processing can be implemented, but processing efficiency decreases
Solution Approach 1:
The patent performs preliminary action by pre-fetching and buffering column data into local column buffers before processing. This preliminary preparation reduces the need for remote accumulations during actual computation, maintaining column-oriented processing capability while improving processing efficiency.
Solution Approach 2:
The patent creates local copies of column data in column buffers within each memory segment. These copies enable local processing without requiring frequent remote accumulations, thus maintaining versatility for column-oriented operations while significantly improving processing efficiency by reducing remote memory access.
Data Source
AI summary
An integrated circuit memory device can include a plurality of banks of memory, each of the banks of memory including a first pair of sub-arrays comprising first and second sub-arrays, the first pair of sub-arrays configured to store data in memory cells of the first pair of sub-arrays, a first row buffer memory circuit located in the integrated circuit memory device adjacent to the first pair of sub-arrays and configured to store first row data received from the first pair of sub-arrays and configured to transfer the row data into and/or out of the first row buffer memory circuit, and a first sub-array level processor circuit in the integrated circuit memory device adjacent to the first pair of sub-arrays and operatively coupled to the first row data, wherein the first sub-array level processor circuit is configured to perform column oriented processing a sparse matrix kernel stored, at least in-part, in the first pair of sub-arrays, with input vector values stored, at least in part, in the first pair of sub-arrays to provide output vector values representing products of values stored in columns of the sparse matrix kernel with the input vector values.