Directed Line Buffer Layout for Fewer NN Memory Reads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial neural networks (NN) and deep neural networks (DNN) face performance issues due to inefficient data processing and memory management, leading to increased latency and unnecessary computations, particularly in managing data across memory components like the line buffer.
Innovation Solution
The implementation of a directed line buffer in NN/DNN environments that uses calculated shifting bits to minimize memory operations by allowing data to be written and read as a single contiguous block, optimizing memory management and reducing the number of reads and writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional memory management practices are used in NN/DNN, then data processing can be performed, but the number of memory reads and writes increases, reducing overall performance
Solution Approach 1:
The patent applies preliminary action by pre-calculating shifting bits based on data dimensions and continuity characteristics before data processing begins. The directed line buffer pre-organizes data layout and computes the necessary bit shifts in advance, so that during actual data processing, memory operations can proceed efficiently without real-time calculation overhead. This preparatory organization of data in the line buffer reduces the number of memory reads and writes required during neural network operations.
2Productivity
If traditional memory management practices are used in NN/DNN, then data can be processed, but avoidable computations and neural processor operations are required, stressing overall performance
Solution Approach 1:
The patent extracts the complex computation of determining data layout and shifting requirements from the neural processor and places it in the directed line buffer. By separating this preprocessing function into the line buffer, the neural processor is freed from performing avoidable computations and can focus on its primary function of neural network operations. The line buffer handles the extraction and organization of data with appropriate shifting bits, reducing the computational burden on the neural processor.
3Productivity
If data is not organized with shifting bits in the line buffer, then memory operations increase, but implementing directed line buffer requires additional control logic
Solution Approach 1:
The directed line buffer applies self-service by autonomously calculating and inserting the appropriate shifting bits based on data dimension and continuity characteristics without requiring external intervention. The line buffer controller automatically determines the number of continuous data elements, computes the required bit shifts, and organizes data accordingly. This self-organizing capability reduces memory operations while the control logic, though added, manages itself without requiring complex external control mechanisms.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The performance of a neural network (NN) and/or deep neural network (DNN) can limited by the number of operations being performed as well as management of data among the various memory components of the NN/DNN. Using a directed line buffer that operatively inserts one or more shifting bits in data blocks to be processed, data read/writes to the line buffer can be optimized for processing by the NN/DNN thereby enhancing the overall performance of a NN/DNN. Operatively, an operations controller and/or iterator can generate one or more instructions having a calculated shifting bit(s) for communication to the line buffer. Illustratively, the shifting bit(s) can be calculated using various characteristics of the input data as well as the NN/DNN inclusive of the data dimensions. The line buffer can read data for processing, insert the shifting bits and write the data in the line buffer for subsequent processing by cooperating processing unit(s).