Convolution Calculation Method for Deep Learning Bandwidth Bottleneck
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models face inefficiencies in convolution calculation due to bandwidth bottlenecks caused by poor data reuse, making it difficult for models to adapt to frequent updates and upgrades.
Innovation Solution
A convolution calculation method that adjusts the sizes of blocks in convolution layers to reduce bandwidth consumption by blocking input channel data and performing convolution calculations in a way that minimizes data access from memory, allowing for adaptive deep learning model updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If conventional convolution calculation method is used, then calculation can be performed, but bandwidth bottleneck occurs due to poor data reuse
Solution Approach 1:
The patent divides the convolution calculation process into multiple stages, separating weight data loading from input data processing. Weight data is loaded once and stored in high-speed memory, while input data is processed in blocks, reducing repeated memory accesses and improving data reuse efficiency.
Solution Approach 2:
The patent performs preliminary loading of weight data into high-speed memory before the main convolution calculation. This preliminary action ensures that weight data is readily available during computation, eliminating the need for repeated memory accesses and reducing bandwidth consumption.
2Adaptability or versatility
If deep learning model is updated and upgraded frequently, then model performance can be improved, but data handling and power consumption increase
Solution Approach 1:
The patent introduces a dynamic block size adjustment mechanism that adapts to different deep learning model configurations. The block size is determined based on the specific model architecture and data characteristics, allowing the system to optimize performance and power consumption for each model update without requiring complete redesign.
Solution Approach 2:
The patent changes the block size parameter dynamically based on the deep learning model being executed. By adjusting this parameter, the system can optimize the balance between data reuse and computational efficiency for different models, reducing power consumption while maintaining adaptability.
3Adaptability or versatility
If block size of convolution layer is fixed, then calculation is simple, but different deep learning models with different efficiency performance cannot adapt to current situations
Solution Approach 1:
The patent implements dynamic adjustment of block size parameters based on the specific deep learning model being executed. This allows the system to adapt to different model architectures and data characteristics, improving efficiency without requiring complex hardware modifications.
Solution Approach 2:
The patent makes the block size dynamic rather than fixed, allowing it to change based on the computational requirements of different deep learning models. This dynamic approach enables better adaptability while keeping the adjustment mechanism relatively simple.
Data Source
AI summary
The present application provides a convolution calculation method, a convolution calculation apparatus, a terminal device, and a computer readable storage medium. The method includes: inputting an image to be processed into a deep learning model, and obtaining a to-be-blocked convolution group and a target size of a block from all convolution layers of the deep learning model; blocking all input channel data of a first to-be-blocked convolution layer in said convolution group according to the target size, a size of each block being the target size; obtaining an output result of said convolution group according to all blocks of all input channel data of said first convolution layer; inputting the output result of said convolution group to a specified network of the deep learning model. Sizes of blocks of the to-be-blocked convolution layer and bandwidth consumption can be adjusted to adapt to frequently updating and upgrading the deep learning model.