Neural Network Buffering to Reduce Global Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network data processing methods are inefficient due to high computational time costs and memory access delays, particularly in forward and reverse calculations, which hinder performance improvements.
Innovation Solution
The method involves performing a forward calculation to obtain intermediate data, storing it in a buffer unit, and then performing the reverse calculation on this data, reducing the need to store intermediate data in global memory and minimizing global memory access, thereby optimizing neural network structure and increasing data processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If intermediate data is stored in global memory for reverse calculation, then data availability is ensured, but memory access time increases and processing speed decreases
Solution Approach 1:
The patent segments memory into multiple levels: global memory for storing input data and final results, and local memory (buffers) for storing intermediate calculation results. This segmentation allows frequent access data to be stored closer to processing units, reducing memory access time while maintaining data availability where needed.
Solution Approach 2:
The patent introduces local memory buffers as intermediary storage between global memory and processing units. These buffers act as mediators that temporarily hold intermediate results, reducing the need for frequent global memory accesses and thereby improving processing speed while ensuring data is available when needed.
2Adaptability or versatility
If intermediate data is stored in global memory, then data can be accessed for reverse calculation, but memory usage increases and access delays occur
Solution Approach 1:
The patent performs preliminary storage of intermediate calculation results in local memory buffers during the forward propagation phase. This preliminary action ensures that when reverse propagation begins, the intermediate data is already available in fast local memory, eliminating the need for time-consuming global memory accesses during the reverse calculation.
3Measurement precision
If forward calculation results are stored for reverse calculation, then calculation accuracy is maintained, but computational time increases
Solution Approach 1:
The patent applies different storage strategies to different types of data based on their access patterns: input data and final results are stored in global memory, while intermediate results that require frequent access during reverse propagation are stored in local memory buffers. This local quality approach optimizes both accuracy (by preserving all necessary data) and efficiency (by strategically placing data in fast memory).
Data Source
AI summary
A data processing method and device are provided. The method includes: performing a forward calculation of a neural network on global data to obtain intermediate data for a reverse calculation of the neural network; storing the intermediate data in a buffer unit; reading the intermediate data from the buffer unit; and performing the reverse calculation of the neural network on the intermediate data to obtain a result of the reverse calculation. According to embodiments, in the reverse calculation of the neural network, the number of accessing the global memory is reduced, thereby reducing the computational time cost and increasing the data processing speed.