Neural Network Pooling Output Buffers for Low-Latency Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory allocation methods for artificial neural network (ANN) processing, particularly in hardware accelerators, face inefficiencies in memory usage and computational latency due to repeated evaluations and handling of edge cases in pooling operations, leading to increased computational costs and latency.
Innovation Solution
A method involving the computation of an output buffer size based on pooling parameters, initializing it with an absorbing element, and associating input elements with output elements in a streaming manner to reduce memory usage and computational latency by reusing input values across overlapping pooling regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional memory allocation methods are used for pooling operations, then memory operations can be performed, but memory allocation overhead and computational latency increase
Solution Approach 1:
The patent pre-computes the output buffer size using the formula (input_size - pooling_window_size) / stride + 1 before the pooling operation begins. This preliminary calculation eliminates the need for repeated size computations and dynamic memory allocation during the actual pooling operation, thereby reducing computational latency and improving processing efficiency.
Solution Approach 2:
The patent extracts the memory allocation step from the main pooling operation by pre-allocating the output buffer with the correctly computed size before processing. This separation allows the pooling operation to proceed without interruption for memory management tasks, reducing computational latency and improving overall productivity.
2Measurement precision
If traditional pooling operations are performed with repeated evaluations, then complete pooling computation is achieved, but memory usage and computational costs increase
Solution Approach 1:
The patent pre-computes the exact output buffer size using the formula (input_size - pooling_window_size) / stride + 1 before the pooling operation. This preliminary calculation ensures that the buffer is allocated with the precise size needed, avoiding both memory waste from over-allocation and computation errors from under-allocation, thereby maintaining pooling computation accuracy while optimizing memory usage.
Solution Approach 2:
The patent changes the parameter of buffer size from being dynamically determined during processing to being pre-determined through a mathematical formula. This parameter change allows the system to allocate exactly the right amount of memory ( (input_size - pooling_window_size) / stride + 1 elements ), reducing memory usage while maintaining complete and accurate pooling computation.
Data Source
AI summary
A computer-implemented method applies a pooling operator to an input array of data, the pooling operator having an absorbing element value and a set of pooling parameters. A size of an output buffer is computer as a function of the set of pooling parameters. The elements of the output buffer are initialized to the value of the absorbing element of the pooling operator. The output array of data is generated by, for a plurality of iterations associated with respective pooling windows: associating, as a function of the pooling parameters, elements of the input array of a pooling window with output elements of the output buffer; and combining, for each output element of the output buffer, the respective input elements associated with the output element. The combining may include determining a combination of respective elements of the output buffer with the input elements associated with the output elements.