Digital In-Memory Computing Circuit for AI Data Transmission Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The current Von Neumann architecture faces a bottleneck in data transmissions between the CPU and memory, leading to inefficiencies in deep neural network computations due to frequent and massive data exchanges, which limits energy efficiency and recognition speed in AI applications.
Innovation Solution
A circuit based on in-memory computing in the digital domain is introduced, featuring an array of computational storage cells with single-bit multipliers, an adder tree, and multi-bit input transfer logic, enabling direct multiply-accumulate operations on weight and feature data within the memory, thus reducing bus interactions and improving energy efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data transmissions between CPU and memory are performed via limited bus bandwidth, then data exchange can be achieved, but transmission efficiency and energy efficiency deteriorate due to frequent and massive data exchanges
Solution Approach 1:
The patent merges storage and computation functions into a single memory structure. Computational storage cells integrate storage elements with single-bit multipliers, allowing data to be stored and processed in the same location. This eliminates the need for separate data transmission between CPU and memory, directly resolving the bottleneck of frequent data exchanges over limited bus bandwidth.
Solution Approach 2:
The patent introduces an intermediary in-memory computing architecture that sits between traditional CPU and memory. The computational storage cells and adder trees perform partial computation within memory, acting as an intermediary processing layer that reduces the volume of data needing transmission over the bus, thereby improving both transmission efficiency and energy efficiency.
2Adaptability or versatility
If single-bit multipliers are used in computational storage cells, then in-memory multiplication can be performed, but handling multi-bit data requires multiple operations increasing complexity
Solution Approach 1:
The patent segments multi-bit data into individual single-bit components for processing. Each computational storage cell handles one bit of the weight data, and multiple cells work in parallel to process different bits. The adder tree then combines these single-bit products to generate the final multi-bit result, breaking down complex multi-bit multiplication into simpler single-bit operations.
Solution Approach 2:
The patent transitions from processing multi-bit data in a single cell to processing single-bit data across multiple cells simultaneously. By adding the dimension of parallelism across computational storage cells and using the adder tree to aggregate results, the system achieves multi-bit computing capability through single-bit building blocks, effectively moving the complexity from the bit-level to the cell-level parallelism.
Data Source
AI summary
In an embodiment of the disclosure, disclosed is a circuit based on in-memory computing in a digital domain, including: an array of computational storage cells, the computational storage cells including a preset number of data storage cells and a preset number of single-bit multipliers in one-to-one correspondence; an adder tree configured to accumulate products output by respective computational storage cells to obtain an accumulated result; and a multi-bit input transfer logic configured to convert accumulated results output by the adder tree and corresponding to respective single bits included in the input feature data into a multiply-accumulate result of multi-bit input feature data and multi-bit weight data. An in-memory multiply-accumulation is implemented or multi-bit weight data and input feature data, so that efficiency and energy efficiency density of in-memory computing is improved, “read disturb write” issue caused by a voltage change on bit lines is avoided, and computing stability is improved.