Stripe-Based Neural Network Data Processing for Memory Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks, particularly pixel-to-pixel neural networks, face significant resource burdens due to high memory and processing demands, leading to increased costs and inefficiencies, especially in lower-end devices where memory buffers become oversubscribed, causing spikes in bandwidth and power usage.
Innovation Solution
Implementing a stripe and raster based data processing scheme that processes data in vertical stripes, where each layer processes data sequentially along depth and horizontal directions before moving vertically, reducing redundant calculations and memory storage needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural networks process large amounts of data using conventional methods, then processing accuracy is improved, but memory bandwidth and power consumption increase significantly
Solution Approach 1:
The input data is divided into multiple non-overlapping stripes that can be processed independently. Each stripe contains a subset of the input pixels, allowing the neural network to process one stripe at a time rather than the entire input simultaneously, thereby reducing peak memory bandwidth requirements and power consumption while maintaining processing accuracy.
Solution Approach 2:
The patent introduces a new processing dimension by organizing data into stripes and processing them sequentially across the stripe dimension. This transforms the conventional single-batch processing into multi-stripe processing, enabling better resource utilization and reduced power consumption without sacrificing accuracy.
2Reliability
If neural networks use conventional processing methods, then complete data processing is achieved, but device resource burden increases
Solution Approach 1:
By segmenting the input data into multiple stripes and processing them sequentially, the patent reduces the memory buffer size required at any given time. This lowers the device resource burden while ensuring complete processing of all input data through systematic iteration over all stripes.
Solution Approach 2:
The patent processes data in partial batches (individual stripes) rather than requiring the entire dataset to be loaded into memory simultaneously. This partial processing approach reduces resource requirements while maintaining processing completeness through multiple passes.
3Quantity of substance
If memory buffers are increased to handle neural network processing, then data processing capacity is improved, but bandwidth spikes and power consumption increase
Solution Approach 1:
The patent segments the data processing into multiple stripes that can be handled with smaller memory buffers. This segmentation allows the system to maintain high data processing capacity while using less memory bandwidth at any given moment, thereby reducing power consumption associated with memory access.
Data Source
AI summary
Technologies are provided for processing data in neural networks. An example method can include processing, by each layer of a neural network, a row in a first stripe of a data input, the row being processed sequentially in a horizontal direction and according to a layer-by-layer sequence; after processing the row, processing, by each layer, subsequent rows in the first stripe on a row-by-row basis, each subsequent row being processed sequentially in the horizontal direction and according to the layer-by-layer sequence; generating an output stripe based on the processing of the row and subsequent rows; processing, by each layer, a second stripe of the data input, each row in the second stripe being processed in the horizontal direction and according to the layer-by-layer sequence, wherein rows in the second stripe are processed on a row-by-row basis; and generating another output stripe based on the processing of the second stripe.