Convolution Engine Parallel Weight Loading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current convolution computation engines in neural network models consume a significant portion of computation time due to the sequential processing of convolution operations and weight data loading, which hampers performance and efficiency.
Innovation Solution
Implementing a convolution computation engine with a separate weight storage unit and data computation unit, allowing for parallel processing of convolution operations and weight data loading, thereby hiding the loading duration and reducing overall computation time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If weight data loading is performed sequentially before convolution operations, then data loading is simple to implement, but computation time is significantly increased
Solution Approach 1:
The convolution computation engine is segmented into functionally independent modules: a weight storage unit for storing weight data, a data computation unit for performing convolution operations, and a control unit for coordinating operations. This segmentation enables parallel execution of weight loading and computation tasks, resolving the contradiction between reducing computation time and maintaining structural simplicity.
Solution Approach 2:
The weight storage unit pre-loads weight data from external storage during the convolution operation of the first network layer. This preliminary action allows weight data to be ready before it is needed for the second network layer, enabling parallel processing and hiding the loading duration, thus reducing overall computation time without significantly complicating the engine structure.
2Productivity
If weight data is loaded during convolution operations of subsequent layers, then storage efficiency is improved, but data loading may become a bottleneck
Solution Approach 1:
The weight storage unit acts as an intermediary buffer between external storage and the data computation unit. It receives and stores weight data from external storage during convolution operations, then supplies the data to the computation unit when needed. This intermediary mechanism decouples the data loading speed from processing efficiency, allowing the system to maintain high productivity without being bottlenecked by external storage speed.
3Productivity
If separate weight storage and computation units are used, then parallel processing is enabled, but device area increases
Solution Approach 1:
The weight storage unit is designed with multi-functionality: it stores weight data for multiple network layers, serves as a buffer for data transfer, and enables parallel processing operations. By consolidating these functions into a single versatile component rather than creating separate dedicated units for each function, the engine achieves high computation performance through parallel processing while minimizing the increase in device area.
Data Source
AI summary
This application discloses a convolution computation engine of a neural network model. The neural network model includes at least first and second network layers. The convolution computation engine includes a weight storage unit implemented by a first processing circuit and a data computation unit implemented by a second processing circuit. The weight storage unit pre-loads weight data of the second network layer from an external storage unit located outside the convolution computation engine during a convolution operation of the first network layer. The data computation unit obtains target input data of the second network layer in response to an operation instruction of the second network layer, and reads the weight data of the second network layer from the weight storage unit; and performs a convolution operation on the target input data by using the weight data of the second network layer.