Compute Memory Selection Logic for Faster In-Memory GEMM
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional computer architectures face significant delays in performing calculations due to high read and compute times, particularly in AI neural-network computations, which are exacerbated by the inefficiencies in traditional memory architectures during parallel general matrix multiplication (GEMM) operations.
Innovation Solution
The integration of logic gates within a memory array, where each logic gate is coupled to pairs of memory cells and controlled by selection logic, allowing for efficient performance of logical operations by reducing the need for separate read and compute steps through direct data processing within the memory array.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data are stored in separate memory and read before calculation, then data storage capacity is improved, but read time and compute time increase causing performance delays
Solution Approach 1:
The patent merges memory storage and logic computation into a unified structure where logic gates are integrated directly within the memory array. Each memory cell is associated with logic gates that can perform computations on stored data without requiring data to be read out to external processors, thereby eliminating the separation between storage and computation functions.
Solution Approach 2:
The patent introduces a new dimensional aspect to memory architecture by adding vertical integration of logic gates alongside horizontal memory cell arrangements. This creates a three-dimensional computing memory structure where computation occurs in the same physical space as storage, rather than in sequential steps across different components.
2Device complexity
If traditional memory architecture is used for AI neural-network computations, then implementation simplicity is maintained, but parallel GEMM operations suffer from substantial delays
Solution Approach 1:
The patent segments the memory array into multiple independently controllable blocks, each capable of performing parallel GEMM operations. By dividing the computation into smaller segments that can execute simultaneously, the overall GEMM operation speed is accelerated while maintaining a modular architecture that builds upon traditional memory structures.
Solution Approach 2:
The patent implements dynamic control mechanisms where select lines and word lines can be selectively activated to enable different computation modes. The architecture can dynamically switch between traditional memory access modes and accelerated GEMM computation modes, allowing optimal performance for different workloads while maintaining implementation flexibility.
Data Source
AI summary
A memory includes an array with rows and columns of memory cells. The rows include a first row and a second row. The memory also includes a plurality of logic gates in the array. Each logic gate of the plurality of logic gates includes a first input coupled to a respective memory cell in the first row, a second input coupled to a respective memory cell in the second row, and an output. The memory further includes selection logic coupled to the plurality of logic gates. The selection logic includes a two dimensional (2D) associative array to generate select lines associated with a first pair of rows of memory cells. The select lines are configured to provide enable signals to the plurality of logic gates to control compute operations of the array.