Aligned Neural Network Blob Data to Reduce Memory Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural networks and deep neural networks inefficiencies in data processing lead to excessive processing time and resource consumption due to inadequate management of data reads and writes, particularly in memory operations, which results in suboptimal performance.
Innovation Solution
The implementation of virtualized hardware iterators that physically pad data to align it in memory components, optimizing memory management and reducing the number of memory operations by specifying data dimensions and using data padding techniques to minimize reads and writes during processing cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is read from memory during neural network processing operations, then processing can be performed, but a substantial amount of time and processing resources are consumed
Solution Approach 1:
The patent applies preliminary action by padding input data with dummy values before processing to align data with memory block boundaries. This preprocessing step ensures that subsequent memory read operations can efficiently load complete blocks without requiring additional reads, thereby reducing the time lost to memory access during neural network processing operations
2Productivity
If data padding is applied to align data in memory components, then the number of memory reads and writes is minimized, but additional processing operations are required to pad the data
Solution Approach 1:
The patent changes the parameter of data dimensions by adding padding values to modify the shape and size of input data tensors. This parameter transformation aligns the data with memory block boundaries, optimizing memory access patterns and reducing the number of read/write operations required during neural network processing
3Productivity
If virtualized hardware iterators are used to manage data, then memory management is optimized and reads are minimized, but the system complexity increases
Solution Approach 1:
The patent introduces virtualized hardware iterators as intermediary components that manage the iteration over padded input data. These iterators coordinate between the padded data structure and the neural network processing units, enabling efficient memory access patterns while abstracting the complexity of data management from the core processing operations
Data Source
AI summary
The performance of a neural network (NN) and/or deep neural network (DNN) can be limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. By inserting a selected padding in the input data to align the input data in memory, data read/writes can be optimized for processing by the NN/DNN thereby enhancing the overall performance of a NN/DNN. Operatively, an operations controller/iterator can generate one or more instructions that inserts the selected padding into the data. The data padding can be calculated using various characteristics of the input data as well as the NN/DNN as well as characteristics of the cooperating memory components. Padding on the output data can be utilized to support the data alignment at the memory components and the cooperating processing units of the NN/DNN.