Neural Accelerator Data Transmission via Zero-Value Weight Skipping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning neural networks, particularly convolutional neural networks, require significant computational resources for image classification and recognition, leading to increased resource demands as training data volume grows, and existing acceleration methods do not efficiently manage data reuse in neural accelerators.
Innovation Solution
A data transmission method and apparatus that utilize a loader, sender, buffer controller, and reuse buffer to load and store input data based on kernel shape and zero-value information, selectively skipping zero-value data points to reduce memory access and transmission, thereby accelerating convolution operations in neural accelerators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If all input data is loaded and transmitted for convolution operations, then complete data processing is achieved, but memory access and transmission overhead increase
Solution Approach 1:
The patent extracts and identifies zero-value weights from the kernel through pruning techniques, separating them from non-zero weights. This extraction allows the system to exclude zero-value data from loading and transmission processes, reducing memory access overhead and power consumption while maintaining complete processing of necessary data
Solution Approach 2:
The patent implements partial action by loading only the necessary portion of input data corresponding to non-zero weights rather than all input data. The loader selectively fetches data based on kernel sparsity patterns, and the sender selectively transmits relevant data to the executer, avoiding unnecessary data processing and reducing overall system overhead
2Productivity
If all input data is loaded into memory, then data availability is ensured, but memory access overhead increases
Solution Approach 1:
The patent performs preliminary action by pre-identifying and storing zero-value weight positions in zero-value information structures before the convolution operation begins. The loader uses this pre-computed information to directly skip zero-value data regions during data loading, eliminating the need to read and process unnecessary data from memory
Solution Approach 2:
The system extracts zero-value weight position information and uses it to create a filtering mechanism that removes unnecessary data access operations. The sender extracts only the necessary data portions based on zero-value information, preventing wasteful memory access cycles
3Productivity
If complete input data is transmitted to the executer, then processing accuracy is maintained, but transmission overhead increases
Solution Approach 1:
The patent introduces zero-value information structures as intermediary data structures that mediate between the loader and sender components. These intermediaries contain pre-computed zero-value weight position information that guides the sender in selectively transmitting only necessary data portions to the executer, reducing transmission overhead while maintaining processing accuracy
Solution Approach 2:
The sender performs partial action by transmitting only the subset of input data that corresponds to non-zero weights rather than all input data. This selective transmission reduces bandwidth requirements and transmission complexity while ensuring that all necessary data for accurate convolution computation is delivered
Data Source
AI summary
A data transmission method for a convolution operation, and a convolution operation apparatus including a fetcher that includes a loader, at least one sender, a buffer controller, and a reuse buffer. The method includes loading, by the loader, input data of an input feature map according to a loading order, based on input data stored in the reuse buffer, a shape of a kernel to be used for a convolution operation, and two-dimensional (2D) zero-value information of weights of the kernel; storing, by the buffer controller, the loaded input data in the reuse buffer of an address cyclically assigned according to the loading order; and selecting, by each of the at least one sender, input data corresponding to each output data of a convolution operation among the input data stored in the reuse buffer, based on one-dimensional (1D) zero-value information of the weights, and outputting the selected input data.