In-Memory Computing Integrated Circuit for AI Workloads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The conventional Von Neumann architecture of computers faces a performance bottleneck due to the time and power required for shuttling data between data storage and processing units, especially with the increased data demands in artificial intelligence applications.
Innovation Solution
An integrated circuit that operates in both memory and computation modes, featuring a memory array with programmable weights, bit lines, and a page buffer, allowing for in-memory computing and near-memory summation of products without the need for data transfer between processing and storage units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is shuttled between data storage units and data processing units through inputs/outputs and buses, then data transfer is enabled, but time and power are consumed and performance bottleneck occurs
Solution Approach 1:
The patent merges data storage units (memory array with memory cells) and data processing units (page buffer with computation circuitry) into a single integrated circuit structure. The memory array stores weights while the page buffer performs multiplication and summation operations on these weights, eliminating the need for data shuttling between separate storage and processing units, thus resolving the performance bottleneck caused by data transfer time
Solution Approach 2:
The integrated circuit is designed to perform multiple functions: it can operate in memory mode for data storage and in computation mode for performing sum-of-products functions. The same memory array and page buffer are used for both storing data and processing data, making the system universal and eliminating the need for separate storage and processing units
2Productivity
If data is shuttled between data storage units and data processing units, then data transfer is enabled, but power consumption increases
Solution Approach 1:
By combining storage and processing functions in one integrated circuit, the patent eliminates the energy-consuming data transfer process between separate units. The memory array and page buffer are integrated such that computation is performed directly on stored data without requiring data to be moved through buses and I/O interfaces, thus reducing power consumption while maintaining computation throughput
Solution Approach 2:
The patent extracts the computation function from separate processing units and integrates it directly into the memory structure. The page buffer is coupled to the memory array and performs computation operations in-place, removing the need for energy-intensive data shuttling between independent storage and processing components
3Adaptability or versatility
If separate data storage units and data processing units are used, then functional separation is achieved, but performance bottleneck occurs with massive data transmission
Solution Approach 1:
The integrated circuit provides operational flexibility by supporting both memory mode for data storage and computation mode for performing sum-of-products functions using the same hardware resources. This multi-functionality maintains adaptability while eliminating the performance bottleneck of data transmission between separate units, as the system can process data in-place without requiring massive data transmission
Data Source
AI summary
An integrated circuit and a computing method thereof are provided. The integrated circuit includes a memory array, word lines, bit lines and a page buffer. The memory array includes memory cells, each configured to be programmed with a weight. The word lines respectively connect a row of the memory cells. The bit lines are respectively connected with a column of the memory cells that are connected in series. More than one of the bit lines in a block of the memory array or more than one of the word lines in multiple blocks of the memory array are configured to receive input voltages. The memory cells receiving the input voltages are configured to multiply the weights stored therein and the received input voltages. The page buffer is coupled to the memory array, and configured to sense products of the weights and the input voltages.