In-Memory Computing Array Sub-Clock Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional Von Neumann structured computers face performance bottlenecks in executing large amounts of multiplication and accumulation (MAC) operations due to mismatches between system memory and processor performance, leading to inefficiencies in neural network training and inference.
Innovation Solution
An in-memory computing (IMC) unit that stores weight vectors as columns and applies input vectors row-wise, using a timing generator to generate sub-clock signals for sequential loading and processing of weight bits, enabling a dynamic logic circuit to perform single-bit and multi-bit matrix product operations efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional Von Neumann structure is used for neural network computations, then system architecture is simple and well-established, but performance of MAC operations is limited due to memory-processor data exchange bottlenecks
Solution Approach 1:
The patent merges the memory storage function and MAC computation function into a single integrated structure. Weight vectors are stored in memory cells arranged in columns, while MAC logic circuits perform computations directly on the stored data, eliminating the need to transfer data between separate memory and processor units. This integration directly resolves the data exchange bottleneck problem.
Solution Approach 2:
The patent introduces sub-clock signals as an intermediary control mechanism to coordinate the sequential loading of weight bits from memory to MAC logic circuits. These timing signals act as mediators that synchronize data flow and computation operations, enabling efficient pipelined processing without requiring complex interconnect structures.
2Productivity
If large amounts of MAC operations are performed in traditional computers, then computational throughput is limited by memory bandwidth, but increasing memory bandwidth increases system complexity and cost
Solution Approach 1:
The patent segments the weight vector storage into multiple columns in the memory cell array, with each column corresponding to a specific weight bit position across multiple weight vectors. This segmentation allows parallel access to corresponding bits from different vectors simultaneously, increasing computational throughput without requiring proportional increases in memory bandwidth or complexity.
Solution Approach 2:
The patent employs periodic sub-clock signals to sequentially enable different columns of the memory cell array in a repeating cycle. This periodic activation pattern allows the system to process multiple weight vectors through the same MAC logic circuits in a pipelined manner, achieving high computational throughput using the same physical resources repeatedly over time.
3Productivity
If weight vectors and input vectors are processed simultaneously in traditional architecture, then data exchange between memory and processor creates bottlenecks, but reducing data parallelism decreases computational efficiency
Solution Approach 1:
The patent performs preliminary organization of weight vectors in the memory cell array before computation begins. Weight vectors are pre-stored in columnar format where corresponding bits from multiple vectors are aligned in the same column. This preliminary arrangement enables immediate parallel processing when input vectors are applied, eliminating the need for runtime data reorganization and reducing data exchange time.
Solution Approach 2:
The patent implements continuous pipelined processing where MAC logic circuits continuously compute matrix products as weight bits are sequentially loaded from memory. The sub-clock signals maintain continuous operation by constantly enabling the next column of weight bits, ensuring that computation resources are never idle and useful action continues without interruption or bottlenecks.
Data Source
AI summary
An in-memory method and apparatus are included. An in-memory computing (IMC) macro includes an IMC array that includes an IMC configured to share sub-clock signals that are generated based on an external clock signal and control respective columns having a crossbar structure, the IMC is further configured to perform a matrix product operation between weight bits by units of columns thereof and input bits of an input vector, the weight bits being sequentially loaded, according to the sub-clock signals, from a memory cell array comprising memory cell units, and an enabling circuit configured to generate enabling signals for enabling the weight bits included in each of the plurality of columns, for each of the plurality of memory cells.