Operation Accelerator for Neural Network Convolution Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current convolution operations in neural networks face challenges such as insufficient matrix buffer space, inflexible calculation, and low calculation efficiency, particularly when dealing with large matrices.
Innovation Solution
The proposed operation accelerator includes a matrix converter and two memories, which perform preprocessing on input data by sliding, concatenating, or padding to create a target matrix, allowing for efficient conversion and calculation by dividing large matrix operations into smaller ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large matrices are used for convolution operations, then calculation accuracy is improved, but buffer space requirements increase and calculation efficiency decreases
Solution Approach 1:
The patent divides large input feature maps and weight parameters into multiple smaller blocks. The input feature map is segmented into N blocks along the channel dimension, and weight parameters are segmented into M blocks. This segmentation allows convolution operations to be performed on smaller matrix blocks individually, reducing the buffer space required for each operation while maintaining overall calculation accuracy through systematic combination of results.
Solution Approach 2:
The patent introduces a new block dimension index k (where k ranges from 0 to M-1) to organize the segmented weight parameter blocks. This dimensional organization transforms the traditional single large matrix operation into multiple smaller block-wise operations indexed by k, enabling efficient memory management and parallel processing while reducing peak buffer space requirements.
2Reliability
If large matrices are used for convolution operations, then calculation completeness is improved, but calculation efficiency decreases
Solution Approach 1:
The patent segments the convolution operation into multiple block-wise operations. Each block operation processes a subset of channels independently, completing partial results that are then systematically combined. This segmentation enables parallel execution of multiple block operations, significantly improving calculation efficiency while ensuring completeness through the structured combination of all block results.
Solution Approach 2:
The patent performs preliminary organization of input feature maps and weight parameters into blocked structures before execution. Input features are pre-organized into N blocks, and weight parameters are pre-organized into M blocks with corresponding index mappings. This preliminary blocking structure enables efficient parallel processing and eliminates the need for large intermediate buffers during computation, improving overall calculation efficiency.
3Adaptability or versatility
If matrix conversion is performed before calculation, then operation flexibility is improved, but buffer space consumption increases
Solution Approach 1:
The patent performs matrix conversion in a segmented manner, converting input feature maps into blocked representations where each block corresponds to a specific channel range. This segmented conversion allows the system to process and store only the necessary block data in buffers at any given time, reducing overall buffer space consumption while maintaining the flexibility to perform various convolution operations through the structured block organization.
4Volume of stationary object
If data is processed in smaller blocks, then buffer space requirements are reduced, but operation complexity increases
Solution Approach 1:
The patent applies consistent segmentation to input features, weight parameters, and output feature maps, where all are divided into blocks with corresponding index relationships. This uniform segmentation approach simplifies the overall operation complexity by establishing regular patterns in data access and computation, even though individual block operations are performed separately. The systematic indexing scheme (with k ranging from 0 to M-1) provides a clear framework that manages complexity while enabling efficient buffer utilization.
Data Source
AI summary
An operation accelerator, a processing method, and a related device, the operation accelerator including a first memory configured to store an input dataset, a matrix converter configured to perform reading M row vectors from the input dataset, generating a first instruction, and sending the M row vectors and the first instruction to a second memory configured to perform, according to the first instruction, preprocessing on the M row vectors to obtain n row vectors, and storing the n row vectors, where the n row vectors include the M row vectors and (n−M) padding row vectors, the n row vectors are N row vectors in a target matrix, and a storage sequence of the n row vectors in the second memory is consistent with a sequence of the N row vectors in the target matrix.