Neural Network Processing Device for SE Block Speed Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The SE block improves CNN accuracy but significantly reduces processing speed due to a large overhead caused by the H×W times copy process in the Scale layer, especially when the number of elements in the 1 dimensional data is large.
Innovation Solution
A neural network processing device and method that generates 1 dimensional data by setting other dimensions to 1, reduces elements, copies the data to restore original dimensions, performs a convolution layer process with a 1×1 filter size, and executes an element-wise product operation to achieve the same CNN accuracy as the SE block while reducing processing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the SE block is used to improve CNN accuracy, then the accuracy is improved, but the processing speed is significantly reduced due to the H×W times copy process overhead
Solution Approach 1:
The patent extracts the essential function of the SE block (channel-wise feature modulation) from its original implementation that includes H×W copy operations. By separating the core excitation mechanism from the redundant copying steps, the invention achieves the same accuracy improvement without the processing speed penalty.
Solution Approach 2:
The patent applies partial action by performing the copy process only when necessary (when H×W > 1) and using optimized copy strategies. The invention selectively applies the squeeze-and-excitation operations only to the channels that contribute to accuracy improvement, avoiding unnecessary full H×W copying for all channels.
2Reliability
If the H×W times copy process is performed in the Scale layer, then the channel-wise feature modulation is achieved, but the processing overhead increases significantly
Solution Approach 1:
The patent performs preliminary actions by pre-computing the channel-wise weights in the excitation network before the scaling operation. By preparing the modulation factors in advance and storing them in an efficient format, the invention eliminates the need for repeated H×W copy operations during the actual scaling phase, significantly reducing processing overhead.
Solution Approach 2:
The patent uses optimized copying strategies where instead of copying the entire H×W feature map for each channel, it efficiently replicates only the necessary channel-wise weights. The invention implements a streamlined copy mechanism that reduces memory bandwidth requirements and processing time while maintaining the same feature modulation效果.
Data Source
AI summary
The 1 dimensional data generation means 92 generates 1 dimensional data, by setting the number of elements of each dimension other than predetermined one dimension to 1 based on multidimensional data corresponding to one input data. The number of elements reducing means 93 reduces the number of elements included in the 1 dimensional data. The copy means 95 generates multidimensional data, by copying the 1 dimensional data whose number of elements has been reduced multiple times. The convolution layer processing means 96 performs a convolution layer process with a filter size of 1×1 on the multidimensional data generated by the copy means 95. The element-wise product operation means 98 performs an element-wise product operation, based on the multidimensional data corresponding to one input data and multidimensional data generated by the above process.