Convolution Reuse Buffer Layout for Lower Memory Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning technologies, particularly CNN-based models, require significant computational resources and increased training data, leading to inefficiencies in resource utilization.
Innovation Solution
Implement a processor-implemented method using a loader to load input data into a reuse buffer, where data is stored and transmitted by senders to an executer for convolution operations, with overlapping data portions being spatially and temporally shared to reduce redundancy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If input data is loaded and stored completely in the reuse buffer, then all data is available for convolution operations, but memory capacity requirements increase and redundant storage occurs
Solution Approach 1:
The patent segments the input data into multiple blocks and loads them in a sequential manner according to the loading order. The buffer controller manages these segments by rotating through the reuse buffer, loading only the necessary segment at each step rather than loading all data at once. This reduces the peak memory requirement while ensuring data is available when needed for convolution operations.
Solution Approach 2:
The patent implements preliminary action by pre-determining the loading order of input data blocks based on the convolution operation requirements. The buffer controller is configured to load data segments in advance according to this predetermined sequence, ensuring that data is ready before it is needed for computation. This allows efficient memory utilization by loading only what is needed next rather than all data simultaneously.
2Reliability
If all input data is loaded into memory, then sufficient data is available for processing, but memory bandwidth is saturated creating a bottleneck
Solution Approach 1:
The patent divides the input data into sequential segments and loads them one at a time according to a predetermined loading order. This segmentation approach allows the memory subsystem to process data in manageable chunks rather than attempting to load all data simultaneously, thereby avoiding saturation of the memory bandwidth while still providing sufficient data for processing when needed.
Solution Approach 2:
The patent uses preliminary action by pre-calculating the loading order of data segments based on the convolution operation requirements. The buffer controller then loads data segments in advance according to this sequence, ensuring data is ready before computation begins. This eliminates the need to load all data simultaneously, reducing memory bandwidth pressure while maintaining data sufficiency for processing.
3Reliability
If input data is stored redundantly in the reuse buffer, then data is readily available for multiple operations, but storage efficiency decreases
Solution Approach 1:
The patent segments input data into distinct blocks and manages them through a rotational loading mechanism. Each segment is loaded only when needed based on the current convolution operation requirements, rather than maintaining redundant copies of all data in the buffer. This segmentation approach ensures data is accessible when required while minimizing unnecessary storage duplication.
Solution Approach 2:
The patent implements preliminary action by pre-determining which data segments need to be loaded based on the convolution operation requirements. The buffer controller loads only those specific segments in advance according to the predetermined order, ensuring data accessibility for upcoming operations while avoiding the storage of redundant data that would result from loading all possible data segments.
4Ease of manufacture
If data is loaded in a fixed order, then the loading process is simple, but memory access patterns become less efficient
Solution Approach 1:
The patent introduces dynamics by making the data loading order adaptive rather than fixed. The buffer controller dynamically determines the loading sequence based on the current convolution operation requirements and the state of the reuse buffer. This dynamic approach allows the system to optimize memory access patterns by loading data in the most efficient sequence possible for the current computation, while maintaining relative simplicity through automated control.
Data Source
AI summary
A computing method and device with data sharing re provided. The method includes loading, by a loader, input data of an input feature map stored in a memory in loading units according to a loading order, storing, by a buffer controller, the loaded input data in a reuse buffer of an address rotationally allocated according to the loading order, and transmitting, by each of a plurality of senders, to an executer respective input data corresponding to each output data of respective convolution operations among the input data stored in the reuse buffer, wherein portions of the transmitted respective input data overlap other.