Neural Network Convolution Memory Reuse for Overflow Prevention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network convolution operations face inefficiencies in memory utilization, leading to potential overflow and suboptimal performance, particularly in deep learning tasks with large amounts of training data.
Innovation Solution
A method and apparatus that utilize a line data structure for storing and processing convolution operations, allowing for efficient reuse of memory space by overwriting output lines to unused input feature elements, and employing cumulative convolution operations to optimize memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional memory allocation methods are used for convolution operations, then memory space is allocated separately for input and output feature maps, but this leads to memory overflow and reduced efficiency in deep learning tasks with large datasets
Solution Approach 1:
The patent merges the storage spaces for input and output feature maps by allowing the output feature map to be stored in the same memory region as the input feature map. This is achieved by overwriting unused portions of the input feature map memory with the output feature map data, thereby combining two separate memory allocations into one unified space and preventing memory overflow in deep learning tasks with large datasets
Solution Approach 2:
The patent recovers and reuses memory space by overwriting unused portions of the input feature map with the output feature map. After the convolution operation completes, the memory space that previously held input data but is no longer needed is recovered and reused for storing the output feature map, thereby improving memory space utilization and preventing overflow
2Productivity
If separate memory spaces are allocated for input and output feature maps, then data integrity is maintained, but memory efficiency is reduced and computational speed is limited
Solution Approach 1:
The patent combines the memory spaces for input and output feature maps, allowing the output to be stored in the same memory region as the input. This merging of memory spaces eliminates the need for separate memory allocations, improves memory efficiency, and enables faster computational operations by reducing memory access overhead in deep learning tasks
Solution Approach 2:
The patent performs preliminary identification of unused memory portions in the input feature map before overwriting with output data. By pre-determining which memory regions can be safely overwritten, the system optimizes memory space utilization and prepares the memory layout for efficient subsequent computations, thereby improving computational speed
Data Source
AI summary
A processor-implemented neural network method includes: generating a first output line of an output feature map by performing a convolution operation between a first input line group of an input feature map and weight kernels; generating a first output of an operation block including the convolution operation based on the first output line; and storing the first output in a memory in which the input feature map is stored by overwriting the first output to a memory space of the memory.