Neural Network Apparatus for Convolution Operation Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network architectures face inefficiencies in processing and computation, particularly in convolution operations, due to the need for extensive data alignment and the complexity of multiply-accumulation (MAC) structures.
Innovation Solution
The proposed solution involves a neural network apparatus that splits input node data and weights into smaller predetermined sizes, allowing for efficient storage and processing in node and weight buffers. This apparatus includes a multiplexer and shifter to output the split data on an index-by-index basis, facilitating convolution operations with reduced computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If input node data is split into smaller predetermined sizes and stored in node buffers, then the number of adders required in the MAC structure is reduced, but the device complexity increases due to the need for multiple buffers and data alignment operations
Solution Approach 1:
The input node data is segmented into multiple smaller data portions, each stored in separate node buffers. This segmentation allows the MAC structure to process data in smaller chunks, reducing the number of adders required while maintaining processing efficiency through parallel operations on different data segments.
Solution Approach 2:
The patent introduces a data dimension by splitting input data into multiple buffers, transforming the traditional single-buffer approach into a multi-dimensional data storage structure. This enables index-by-index data output and facilitates more efficient convolution operations with reduced computational complexity.
2Measurement precision
If data alignment operations are performed extensively to prepare input data for convolution, then the accuracy of convolution operations is improved, but the processing time increases
Solution Approach 1:
The input node data is pre-split into smaller predetermined sizes and stored in node buffers before the convolution operation. This preliminary action organizes the data in advance, eliminating the need for extensive data alignment operations during the actual convolution process and reducing processing time while maintaining accuracy.
Solution Approach 2:
The system dynamically outputs data from node buffers on an index-by-index basis, allowing flexible data retrieval without rigid alignment requirements. This dynamic data access pattern maintains convolution accuracy while significantly reducing the time needed for data preparation and alignment.
3Productivity
If the number of adders in the MAC structure is reduced to improve processing speed, then the computational complexity is decreased, but the precision of convolution operations may be compromised
Solution Approach 1:
By segmenting input data into smaller portions stored in multiple node buffers, the system can use fewer adders in the MAC structure while maintaining precision. Each adder processes a specific data segment, and the cumulative result across segments preserves the full precision of the convolution operation.
Solution Approach 2:
The patent compensates for reduced adder count by introducing a data dimension through multi-buffer storage. This allows the system to perform multiple parallel operations that collectively provide the same computational precision as a larger single MAC structure, thereby maintaining accuracy while improving processing speed.
Data Source
AI summary
A neural network apparatus includes a plurality of node buffers connected to a node lane and configured to store input node data by a predetermined bit size; a plurality of weight buffers connected to a weight lane and configured to store weights; and one or more processors configured to: generate first and second split data by splitting the input node data by the predetermined bit size, store the first and second split data in the node buffers, output the first split data to an operation circuit for a neural network operation on an index-by-index basis, shift the second split data, and output the second split data to the operation circuit on the index-by-index basis.