Neural Network Processing Block-by-Block Local Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network processing requires significant memory bandwidth due to the need to transfer large amounts of data between operations, leading to inefficiencies in memory usage and processing speed.
Innovation Solution
Implementing a method where neural network processing is performed on a block-by-block basis, with intermediate data stored and transferred locally within the processor's storage instead of main memory, reducing the need for extensive data transfer between operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural network processing transfers large amounts of data between operations using main memory, then complete data availability is ensured, but memory bandwidth consumption increases and processing efficiency decreases
Solution Approach 1:
The patent segments the large input data array into multiple smaller blocks that can be processed independently. Each block is processed through the sequence of operations separately, allowing intermediate results to be stored in local storage rather than transferring complete large datasets to main memory. This segmentation reduces the volume of data requiring main memory bandwidth while maintaining processing completeness.
Solution Approach 2:
The patent implements a nested storage structure where local storage (faster, smaller capacity) is embedded within the processor and嵌套ed with main memory (slower, larger capacity). Intermediate processing results are stored in the nested local storage, avoiding repeated transfers to and from the outer main memory layer. This nested architecture enables efficient data hierarchy management that reduces overall memory bandwidth consumption.
2Speed
If intermediate data is stored in main memory between operations, then data capacity requirements are met, but data transfer time increases and processing speed decreases
Solution Approach 1:
The patent performs preliminary action by loading necessary data blocks into local storage before processing begins. The processor pre-positions data in the faster local storage medium, so that during the actual processing sequence, data can be accessed rapidly without time-consuming transfers to main memory. This preliminary data positioning eliminates bottlenecks during critical processing phases.
Solution Approach 2:
Local storage acts as an intermediary between the processor and main memory. Instead of direct communication between the processor and slow main memory, the local storage intermediary buffers and manages data transfers, providing fast access to intermediate results and reducing the time penalty associated with main memory bandwidth limitations.
3Ease of operation
If complete input data arrays are processed at once, then operational simplicity is maintained, but memory bandwidth requirements increase significantly
Solution Approach 1:
The patent automatically segments the input data array into manageable blocks without requiring complex manual intervention. The segmentation is performed systematically to ensure that each block can be processed independently through the same sequence of operations, maintaining operational simplicity while dramatically reducing the memory bandwidth required at any given time compared to processing the complete array simultaneously.
Data Source
AI summary
A sequence of operations to process an initial input data array for the sequence of operations to generate a final output data array of the sequence of operations on a processor operable to execute neural network processing, the sequence are performed for respective blocks of the initial input data array on a block-by-block basis, and when performing an operation in the sequence whose output data is input data for another operation in the sequence, the output data is used as input data for another operation of the sequence is stored in local storage of the processor that is performing the neural network processing, and provided as input data for the another operation in the sequence from the local storage, but for the final operation in the sequence, the final output data array is stored in a main memory of the data processing system.