In-Memory Vector-Matrix Multiplication for Neural Network Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix multiplication, a complex operation in computational procedures, particularly in artificial neural networks, consumes significant computational resources due to its high complexity, especially in matrix-matrix and vector-matrix multiplications.
Innovation Solution
Implementing in-memory vector-matrix multiplication using an associative memory device with a memory array divided into sections, where the operands are stored in the same column, allowing for concurrent computation and reducing the number of connections between operands, thereby lowering the computational complexity to linear or constant time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If matrix multiplication is performed using conventional computational methods, then the operation can be completed with standard processing, but the computational complexity is high and consumes significant computational resources
Solution Approach 1:
The patent replaces conventional mechanical/computational processing with an in-memory computation approach using associative memory devices. The memory device itself performs the multiplication operation through its physical structure and properties, eliminating the need for traditional sequential computational processing and reducing computational complexity from O(mpn) to linear or constant time.
Solution Approach 2:
The patent introduces an associative memory device as an intermediary between data storage and computation. This intermediary performs the matrix multiplication operation directly within the memory array, acting as a bridge that combines storage and computation functions to reduce the complexity of the overall system.
2Device complexity
If operands are stored in separate locations in memory, then data organization is simple, but the number of connections between operands increases computational complexity
Solution Approach 1:
The patent merges the storage locations of operands by storing them in the same column of the memory array. This combining of operands in a unified spatial location reduces the number of connections required between them and simplifies the computation process, as the associative memory device can access both operands through the same memory column.
Solution Approach 2:
The patent changes the spatial dimensionality of operand storage by organizing data in a three-dimensional memory array structure. Instead of storing operands in separate two-dimensional locations, the invention uses the third dimension (column depth) to co-locate operands, reducing the connection complexity while maintaining ease of access.
Data Source
AI summary
A device for in memory vector-matrix multiplication includes a memory array and in-memory logic. The memory array has at least two sections and stores a multiplier matrix. The memory array also receives and stores an input multiplicand arranged in a vector such that the operands of the vector-matrix multiplication are located on a same column of the memory array. Each of the sections is one of: a volatile memory array, a non-volatile memory array, a destructive memory array and a non-destructive memory array. The in-memory logic computes an output of the vector-matrix multiplication using the stored input vector and the stored multiplier matrix. The memory array is one of the following type of memory array: RAM, DRAM, SRAM, Re-RAM, ZRAM, MRAM and Memristor.