Convolutional Residual Structure for Low-Latency GPU Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The current GPU architecture for Resnet50 neural networks experiences communication delays due to frequent data reading and writing between global and shared memory, which acts as a bottleneck for computation speed, especially in convolution operations.
Innovation Solution
Optimize convolutional residual structures by dividing data into areas and performing convolutions in shared memory, reducing the number of data exchanges between global and shared memory through a combination of convolution operations and linear rectification, utilizing Tensorcore computation cores for efficient matrix multiplication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is frequently read and written between global memory and shared memory for convolution operations, then computation accuracy is maintained, but communication latency increases and computation speed decreases
Solution Approach 1:
The patent loads convolution kernel data and picture data into shared memory before computation begins, performing preliminary data preparation. This allows the computation units to access data locally without repeated global memory accesses during the actual convolution operations, thereby reducing communication latency while maintaining computational accuracy.
Solution Approach 2:
The patent introduces shared memory as an intermediary between global memory and computation units. Data is first transferred from global memory to shared memory, and then computation units access this intermediate storage for processing. This intermediary layer reduces the frequency of global memory accesses, lowering communication overhead and latency.
2Reliability
If convolution operations are performed with frequent data exchange between global and shared memory, then computation completeness is ensured, but computation density decreases
Solution Approach 1:
The patent divides the picture data into multiple blocks and processes them in parallel using multiple computation units. Each computation unit handles a specific block independently, allowing simultaneous processing of different data segments. This segmentation increases computation density by utilizing multiple processing units concurrently while ensuring complete processing of all picture data through the blocking approach.
Solution Approach 2:
The patent transforms the computation approach by adding a spatial dimension through block division and parallel processing. Instead of sequential processing of the entire picture, the system processes multiple blocks simultaneously across different computation units, effectively utilizing the parallel dimension of the GPU architecture to increase computation density.
3Speed
If data is divided into areas and processed in shared memory blocks, then computation speed increases, but device complexity increases
Solution Approach 1:
The patent segments picture data into fixed-size blocks with defined dimensions (e.g., 16x16 or 32x32 pixels per block). This systematic segmentation provides a regular structure that simplifies memory allocation and computation unit assignment, reducing the actual implementation complexity despite the increased data division granularity that improves computation speed.
Solution Approach 2:
The patent employs configurable parameters for block size, number of computation units, and shared memory allocation. By making these parameters adjustable, the system can optimize for different picture sizes and hardware configurations without requiring complex adaptive logic, thus managing device complexity while maintaining high computation speed through parameter-driven optimization.
Data Source
AI summary
Disclosed is a method for optimizing a convolutional residual structure of a neural network, including: obtaining picture data and convolution kernel data of each group of residual structures from a global memory as inputs, calculating a first convolution to obtain a first result, storing the first result in a shared memory; determining the size of a picture according to the picture data of the first result, dividing the picture into a plurality of first regions, allocating a corresponding block to each first region in the shared memory, calculating a second convolution in the blocks to obtain a second result; determining the size of a second picture, dividing the second picture into a plurality of second regions, allocating each second region to a corresponding block, calculating a third convolution in the blocks to obtain an output; adding the output and the inputs and performing linear rectification to obtain a final result.