Memory Padding Allocation for Deep Learning Convolution Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models on terminal devices face significant overhead due to frequent memory allocation and deallocation during convolution operations, leading to high memory consumption and processing inefficiencies, especially when resources are limited.
Innovation Solution
The method involves determining the sizes of input and output data for convolution operations, grouping them into sets with no increasing output sizes, and allocating fixed memories in advance, allowing for uniform release after processing, thereby reducing memory management overhead and conserving processing capability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If memory is allocated for each convolution operation to perform value padding, then padding can be performed correctly, but memory allocation and deallocation overhead increases significantly
Solution Approach 1:
The patent applies preliminary action by allocating all necessary padding memories before the convolution operations begin. Specifically, it determines the maximum padding size needed, allocates sufficient memory space in advance, and initializes it with padding values. This eliminates the need for repeated memory allocation and deallocation during the convolution process, reducing overhead while ensuring padding correctness is maintained throughout all operations.
2Reliability
If memory is allocated for each convolution operation, then padding can be performed, but processing time is consumed by repeated allocation and deallocation
Solution Approach 1:
The patent performs memory allocation preliminary by calculating the maximum padding requirements before convolution operations start. It allocates a single contiguous memory block sufficient for the largest padding need, initializes it once, and reuses it across all convolution operations. This approach maintains full padding capability while eliminating repeated allocation/deallocation cycles, significantly reducing the time lost to memory management operations.
3Quantity of substance
If multiple memories are allocated for different convolution operation sets, then memory capacity can be optimized, but the number of memory management operations increases
Solution Approach 1:
The patent applies merging by consolidating multiple potential memory allocations into a single unified padding memory. Instead of allocating separate memories for each convolution operation or operation set, it determines the maximum padding size required across all operations and allocates one sufficiently large memory block. This single memory is shared across all convolution operations, optimizing memory capacity utilization while reducing the number of memory management operations from multiple allocations to a single allocation.
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
Provided are a method, a device and a storage medium for processing overhead of memory access. The method includes: applying for memories for padding; performing a deep learning algorithm on input data; and releasing the memories for padding after the deep learning algorithm is completed. In the disclosure, the memories for padding can be preset in advance and can be uniformly released after the deep learning algorithm is completed, thereby reducing the processes of applying for and releasing the memories, and saving the processing capability.