Convolution Memory Routing to Avoid Redundant Matrix Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Redundant storage of element data in different memories during convolution operations in neural network models affects memory usage efficiency.
Innovation Solution
A convolution operation device that stores different parts of a matrix in separate memories and uses a routing and shift register circuit to transmit elements between multiply-accumulate circuits as needed, avoiding redundant storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If element data is copied multiple times and stored in different memories corresponding to different MAC operators, then computing efficiency is improved, but memory usage efficiency deteriorates
Solution Approach 1:
The patent segments the memory storage system into multiple independent memory banks (first memory, second memory, third memory, etc.), each storing different portions of the matrix data. This segmentation allows different MAC operators to access different memory banks simultaneously, improving computing efficiency while avoiding redundant storage of the same data elements across all memory banks.
Solution Approach 2:
The routing and shift register circuit serves multiple functions: it routes data elements from appropriate memory banks to the required MAC operators, performs shifting operations to align data correctly, and manages data flow timing. This multi-functional circuit enables the system to achieve both high computing efficiency and good memory usage efficiency through intelligent data management.
2Speed
If multiple memories store copies of the same element data, then data access speed for different MAC operators is improved, but hardware resource consumption increases
Solution Approach 1:
The system dynamically routes data elements from memory banks to MAC operators based on real-time computational requirements. The routing and shift register circuit adaptively controls data flow, selecting which memory bank to access and how to shift data, thereby achieving fast data access without requiring static redundant storage in all memory banks simultaneously.
Solution Approach 2:
The routing and shift register circuit acts as an intermediary between the segmented memory banks and the MAC operators. It manages data transfer efficiently by selecting the appropriate source memory bank and performing necessary shifting operations, eliminating the need for each MAC operator to have direct access to all memory banks or for all memory banks to store all data elements.
Data Source
AI summary
The invention provides a convolution operation device, which includes a first memory, a second memory, a third memory, a first multiply-accumulate circuit, a second multiply-accumulate circuit, and a routing and shift register circuit. Different elements of a same matrix are stored in different memories. The first multiply-accumulate circuit and the second multiply-accumulate circuit access a convolution kernel from the first memory. During a first period, the routing and shift register circuit transmits a first element of the matrix from the second memory to the first multiply-accumulate circuit, and transmits a second element of the matrix from the third memory to the second multiply-accumulate circuit. During a second period, the routing and shift register circuit transmits the second element of the third memory to the first multiply-accumulate circuit.