Processing-in-Memory Fused-Multiply-Add Logic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computer-based computations for fused-multiply-add (FMA) and matrix multiplication operations are processor and memory intensive, requiring frequent data transfers between compute cores and memory arrays, which leads to inefficiencies and increased latency.
Innovation Solution
The implementation of processing-in-memory (PIM) capable devices that perform FMA operations directly within the memory device, allowing data to be reused across multiple computations by loading operands into a sequencer and multiplying them with rows of data stored in a sense amplifier array, thereby reducing the need for external data transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional processor-based computations are used for FMA and matrix multiplication, then processing capability is provided, but data transfer frequency between compute cores and memory arrays increases leading to higher latency and reduced efficiency
Solution Approach 1:
The patent merges the processing function with the memory function by implementing FMA operations directly within the memory device. The sense amplifier array performs multiplication and accumulation operations on data stored in the memory array, eliminating the need for separate processor-based computation and reducing data transfer latency between compute cores and memory arrays.
Solution Approach 2:
The patent introduces a sense amplifier array as an intermediary component between data storage and processing. This intermediary performs the FMA operations locally within the memory device, acting as a bridge that eliminates direct processor-memory data transfer for computation operations and reduces overall system latency.
2Productivity
If frequent data transfers are performed between compute cores and memory arrays, then processing operations can be executed, but the number of memory loads and stores increases reducing overall efficiency
Solution Approach 1:
The memory device performs self-computation by executing FMA operations internally using its own resources (sense amplifier array). This self-service capability eliminates the need for external processor intervention and reduces the number of memory loads and stores required, thereby decreasing energy consumption associated with frequent data transfers.
3Productivity
If data is loaded and reused across multiple computations within the same memory device, then processing efficiency improves, but the complexity of managing data reuse operations increases
Solution Approach 1:
The sense amplifier array is designed with multi-functionality, serving both as a data storage interface and as a computational unit capable of performing FMA operations. This universal design enables data reuse across multiple computations without requiring separate specialized hardware, managing complexity through a single integrated component that handles both storage and processing functions.
Data Source
AI summary
Methods, apparatuses, and systems for in-or near-memory processing are described. Strings of bits (e.g., vectors) may be fetched and processed in logic of a memory device without involving a separate processing unit. Operations (e.g., arithmetic operations) may be performed on numbers stored in a bit-parallel way during a single sequence of clock cycles. Arithmetic may thus be performed in a single pass as numbers are bits of two or more strings of bits are fetched and without intermediate storage of the numbers. Vectors may be fetched (e.g., identified, transmitted, received) from one or more bit lines. Registers of a memory array may be used to write (e.g., store or temporarily store) results or ancillary bits (e.g., carry bits or carry flags) that facilitate arithmetic operations. Circuitry near, adjacent, or under the memory array may employ XOR or AND (or other) logic to fetch, organize, or operate on the data.