Convolution Processing Block Segmentation for Memory Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolution operations in neural networks are compute-intensive and power-inefficient, leading to high memory access and bandwidth consumption, which restricts their use in mobile devices due to increased power consumption and memory footprint.
Innovation Solution
A method and apparatus for processing convolution operations by dividing input data into blocks, using a processor to perform the convolution operation with a kernel, generating feature maps, and storing them in memory, while minimizing memory access and internal network bandwidth consumption, and using a buffer to accumulate and convert output data to reduce data traffic and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If convolution operations are performed using traditional methods, then recognition rate improves, but power consumption and memory footprint increase
Solution Approach 1:
The input data is divided into multiple blocks, and convolution operations are performed on each block separately. This segmentation allows processing to be distributed across multiple memory banks simultaneously, reducing the frequency of memory access operations and thereby lowering power consumption while maintaining the same recognition rate.
Solution Approach 2:
The patent introduces a new dimension of parallel processing by mapping different input blocks to different memory banks. This spatial distribution across multiple banks enables concurrent access, transforming a sequential operation into a parallel one, which reduces overall execution time and power consumption.
2Reliability
If convolution operations are performed using traditional methods, then recognition rate improves, but memory access frequency increases
Solution Approach 1:
By segmenting input data into blocks and assigning them to different memory banks, the system reduces the total number of memory access operations needed. Each bank can be accessed independently and in parallel, decreasing the overall memory access frequency while preserving recognition accuracy.
Solution Approach 2:
The patent ensures continuous processing by maintaining data in memory buffers and performing convolution operations without repeated memory access. Once data is loaded into the banks, processing continues uninterrupted, eliminating the need for frequent memory access cycles.
3Reliability
If convolution operations are performed using traditional methods, then recognition rate improves, but bandwidth consumption increases
Solution Approach 1:
Segmenting data into blocks distributed across multiple memory banks reduces the total bandwidth required. Each bank handles a portion of the data independently, allowing parallel processing that decreases the aggregate bandwidth consumption while maintaining the same overall recognition rate.
Solution Approach 2:
The patent merges multiple convolution operations into a single parallel processing stage by distributing work across memory banks. This consolidation reduces redundant data transfers and bandwidth consumption that would occur in sequential processing.
Data Source
AI summary
Provided are a method and apparatus for processing a convolution operation in a neural network. The apparatus may include a memory, and a processor configured to read, from the memory, one of divided blocks of input data stored in a memory; generate an output block by performing the convolution operation on the one of the divided blocks with a kernel; generate a feature map by using the output block, and write the feature map to the memory.