GPU Pooling Operations with Row-Column Processing for Reduced Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current GPU-based pooling operations in deep learning models are inefficient, with high computational complexity and memory access requirements, limiting performance improvements in convolutional neural networks.
Innovation Solution
A method for computing pooling operations on GPUs by reducing time complexity, memory accesses, and SIMD instructions through dimension-specific processing and contiguous memory access techniques, specifically using row and column pooler circuitry to process dimensions in a set order.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional pooling operations are used on GPU, then pooling functionality is provided, but computational complexity and memory access requirements are high
Solution Approach 1:
The pooling operation is segmented into two distinct phases: a first pooling pass that processes the input feature map to generate intermediate results, and a second pooling pass that processes the intermediate results to generate final output. This segmentation allows each pass to operate on reduced data volumes, thereby reducing computational complexity while maintaining the required pooling functionality.
Solution Approach 2:
The first pooling pass performs preliminary processing of the input feature map to generate intermediate results that are then used in the second pooling pass. By performing this preliminary action, the data volume is reduced before the final pooling operation, reducing the computational complexity of subsequent operations.
2Productivity
If conventional pooling operations are used on GPU, then pooling functionality is provided, but memory access requirements are high
Solution Approach 1:
The pooling operation is divided into two passes, where intermediate results are stored in GPU memory after the first pass. This segmentation allows the second pass to operate on smaller intermediate results rather than the original large feature map, thereby reducing the quantity of memory accesses required.
Solution Approach 2:
Intermediate results serve as an intermediary data structure between the input feature map and the final pooling output. By introducing this intermediary layer, the patent reduces the amount of data that needs to be accessed from and written to GPU memory during the pooling operation.
3Ease of manufacture
If brute force pooling approach is used, then simplicity of implementation is maintained, but performance improvement is limited
Solution Approach 1:
The pooling operation is segmented into two distinct passes with different purposes: the first pass generates intermediate results with reduced spatial dimensions, and the second pass produces the final pooling output. This segmentation improves performance by reducing redundant computations while maintaining clear implementation logic.
Solution Approach 2:
The two-pass pooling approach maintains continuous useful action by ensuring that each pass processes data that is directly relevant to the final output. The first pass reduces spatial dimensions, and the second pass completes the pooling operation, eliminating wasted computational steps present in brute force approaches.
Data Source
AI summary
An example apparatus includes interface circuitry, machine-readable instructions, and at least one processor circuit to be programmed by the machine-readable instructions to distribute a computing task across one or more cores based on a batch dimension, execute a row pooling operation to generate a first intermediate data output based on the batch dimension and a first pooling size, execute a column pooling operation to produce a second intermediate data output based on the batch dimension and a second pooling size, and generate a final output based on the first intermediate data output and the second intermediate data output.