Parallel Convolution Processing with Dedicated Memory Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolution operations in image processing algorithms, particularly in neural networks, face inefficiencies due to redundant data copying across processing elements, leading to increased computational load and resource utilization when handling dependencies between convolutional layers.
Innovation Solution
The technique involves performing convolution operations in a manner where processing engines operate on different layers in parallel, with each engine generating output for the next layer and storing results in dedicated memory, thereby minimizing data duplication and optimizing processing efficiency by handling dependencies in a temporal rather than spatial manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If convolution operations are performed sequentially with data copied across processing elements, then correctness of layer dependencies is maintained, but processing speed and computational efficiency deteriorate
Solution Approach 1:
The patent divides the convolutional layer processing into independent segments that can be executed in parallel. Each processing element handles specific convolution operations without requiring data copying from other elements, segmenting the computational workload to eliminate redundant data transmission while maintaining layer dependency correctness.
Solution Approach 2:
The patent performs preliminary preparation of convolution data within each processing element before execution. By pre-positioning required data locally and organizing computation pipelines in advance, the system eliminates the need for runtime data copying between processing elements, thereby improving processing speed without increasing complexity.
2Productivity
If data is copied across processing elements to handle layer dependencies, then correctness of convolution operations is ensured, but resource utilization and processing overhead increase
Solution Approach 1:
Each processing element is designed to be self-sufficient, containing all necessary data and computation resources to perform convolution operations independently. This self-service approach eliminates the need for inter-element data copying and reduces processing overhead, allowing parallel execution without communication costs.
Solution Approach 2:
The patent reorganizes the computational architecture from a spatial data distribution model to a temporal parallel execution model. By changing the dimension of parallelism from spatial (data parallelism requiring communication) to temporal (instruction parallelism without communication), the system achieves higher productivity with reduced energy loss.
3Productivity
If processing engines operate on the same layer simultaneously, then parallel processing is achieved, but data duplication and memory usage increase
Solution Approach 1:
The patent segments the convolutional layer computation into independent operational units that can be parallelized without duplicating data. Each processing element works on a distinct computational segment with its own local data requirements, achieving parallel processing capability while minimizing data duplication through careful segmentation of the input feature maps and filter applications.
Data Source
AI summary
A technique for performing convolution operations is disclosed. The technique includes performing a first convolution operation based on a first convolutional layer input image to generate at least a portion of a first convolutional layer output image; while performing the first convolution operation, performing a second convolution operation based on a second convolutional layer input image to generate at least a portion of a second convolutional layer output image, wherein the second convolutional layer input image is based on the first convolutional layer output image; storing the portion of the first convolutional layer output image in a first memory dedicated to storing image data for convolution operations; and storing the portion of the second convolutional layer output image in a second memory dedicated to storing image data for convolution operations.