Memory Apparatus with Logic Units for Parallel AI Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current memory architectures, such as Von Neumann architecture, are inefficient in performing AI operations as they primarily act as passive data storage and writing devices, leading to significant data transfer bottlenecks between processing chips and memory apparatuses, which hampers operation speed and increases power consumption.
Innovation Solution
A memory apparatus with built-in logic operation units, such as MAC units, that can arrange data according to weight matrices for neural network operations, allowing for parallel processing of feature values and weight coefficients to generate intermediate layer feature values, thereby reducing the load on hidden layers and minimizing data transfer.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data reading and writing between processing chip and memory apparatus is increased to achieve machine learning effect, then the data transfer amount increases, but the operation speed decreases and power consumption increases due to Von Neumann architecture bottleneck
Solution Approach 1:
The patent merges the memory apparatus with logic operation units (such as MAC units) to create a unified structure that can both store data and perform computations. This integration allows the memory apparatus to actively participate in AI operations rather than merely passively storing data, thereby reducing the need for frequent data transfers between separate memory and processing units, and improving operation speed while managing data transfer amounts efficiently.
Solution Approach 2:
The memory apparatus is designed with multi-functionality, serving both as a data storage device and as a computation device with built-in logic operation units. This universal design enables the same hardware structure to handle both memory functions and AI computation tasks, reducing the overall system complexity and data transfer requirements between specialized components.
2Quantity of substance
If data reading and writing between processing chip and memory apparatus is increased to achieve machine learning effect, then the data transfer amount increases, but the power consumption increases due to frequent data transfers
Solution Approach 1:
By combining memory storage and computation capabilities into a single apparatus, the patent eliminates the need for repeated data transfers between separate memory and processing units. The built-in logic operation units can directly process data stored in the memory array, significantly reducing the energy consumed by data movement operations while maintaining the required data transfer amounts for machine learning.
3Device complexity
If passive memory apparatus is used for data storage and writing, then the memory apparatus structure is simple, but the capability to perform AI operations is lacking
Solution Approach 1:
The patent integrates logic operation units directly into the memory apparatus structure, creating a unified device that maintains relatively simple architecture while gaining advanced AI computation capabilities. This merging approach avoids the complexity of separate specialized processing units while enabling versatile AI operations through the combined memory-computation structure.
Data Source
AI summary
A memory apparatus and an operation method thereof are provided. The memory apparatus includes a mode configuration register, a system memory array, a pointer and an arithmetic circuit including logic operation units. The mode configuration register stores weight matrix information and a base address. The system memory array stores feature values in a feature map from the base address according to the weight matrix information. The pointer stores the base address and a weight matrix size to provide pointer information. The arithmetic circuit sequentially or parallelly reads the feature values according to the pointer information. The arithmetic circuit parallelly arranges weight coefficients of a selected weight matrix and the corresponding feature values in each of the corresponding logic operation units according to the weight matrix information, and causes the logic operation units to perform computing operations parallelly to output intermediate layer feature values to an external processing unit.