Neural Network Layer Processing with Rolling Buffer Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks, particularly convolutional neural networks (CNNs), face challenges in reducing memory and computational requirements, limiting their ability to process high-resolution images and videos efficiently on cost-effective embedded devices.
Innovation Solution
The proposed solution involves processing more than a single neural network layer simultaneously by using a method that discards unnecessary input data, employs counters to control data flow, and utilizes a rolling buffer to reduce memory requirements, thereby enabling efficient high-resolution image and video processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional CNNs process layers sequentially with full frame buffers, then computational accuracy is maintained, but memory requirements increase significantly
Solution Approach 1:
The patent segments the input data into smaller non-overlapping blocks rather than processing entire frames at once. This allows the system to process high-resolution images and videos by dividing the data into manageable chunks that can fit in limited memory, while maintaining computational accuracy through proper block processing and stitching.
Solution Approach 2:
The patent introduces a temporal dimension by processing multiple neural network layers simultaneously across different time steps. This allows the system to accumulate results from multiple layers without requiring all intermediate data to be stored in memory at once, effectively transforming a memory-intensive spatial problem into a more manageable temporal processing approach.
2Reliability
If full frame buffers are reserved for intermediate data, then processing completeness is ensured, but device complexity and cost increase
Solution Approach 1:
The patent applies partial action by processing only the necessary portion of data at each time step rather than requiring complete frame buffers. The system processes blocks of data and accumulates results progressively, eliminating the need for expensive high-capacity memory and complex buffer management while ensuring processing completeness through systematic block iteration.
3Productivity
If multiple neural network layers are processed simultaneously, then processing speed increases, but memory requirements increase
Solution Approach 1:
The patent implements preliminary action by pre-defining the processing sequence and block structure before execution. The system prepares the data into appropriate blocks and establishes the temporal-spatial mapping in advance, allowing multiple layers to be processed simultaneously without requiring additional memory for intermediate representations, thus achieving high throughput with minimal memory footprint.
Data Source
AI summary
Described herein are systems and methods for efficiently processing large amounts of data when performing complex neural network operations, such as convolution and pooling operations. Given cascaded convolutional neural network layers, various embodiments allow for commencing processing of a downstream layer prior to completing processing of a current or previous network layer. In certain embodiments, this is accomplished by utilizing a handshaking mechanism or asynchronous logic to determine an active neural network layer in a neural network and using that active neural layer to process a subset of a set of input data of a first layer prior to processing all of the set of input data.