Neural Processing Unit Memory Bank Allocation for Data Dependency Conflicts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural processing units (NPUs) face inefficiencies due to increased data size, computational load, and training data size, leading to idle states and increased computation time when handling multiple neural network models, particularly in edge computing devices, and conventional memory structures result in reduced processing speed and power consumption.
Innovation Solution
A data allocatable neural processing unit (NPU) with variable memory banks that allocate data blocks based on input-output dependencies, using graph representation and predefined criteria to optimize memory usage and reduce collisions, enabling efficient computation by sorting data locality and maintaining neural network data locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data blocks are stored in fixed memory banks, then memory structure is simple, but processing speed decreases and collisions increase
Solution Approach 1:
The patent implements dynamic memory bank allocation where data blocks are assigned to memory banks based on their dependency relationships rather than fixed positions. The memory bank assignment is determined by analyzing the graph structure of data dependencies and allocating blocks to different banks to minimize collisions, allowing the system to adapt to different computational workloads and data access patterns.
2Productivity
If memory banks are allocated without considering data dependencies, then allocation is simple, but computation time increases due to idle states
Solution Approach 1:
The patent performs preliminary analysis of data block dependencies by constructing a graph representation before allocating memory banks. This pre-computation of dependency relationships allows the system to optimize memory allocation in advance, ensuring that data blocks with strong dependencies are placed in memory banks that minimize access conflicts and idle states during actual computation.
Solution Approach 2:
The system uses feedback from the analyzed dependency relationships to guide memory bank allocation decisions. By examining the graph structure and identifying critical paths and dependency patterns, the allocation algorithm adjusts memory bank assignments to feedback information about potential collisions and idle states, thereby optimizing computation efficiency.
3Speed
If data blocks with dependencies are stored in the same memory bank, then memory utilization is high, but collisions increase and processing slows down
Solution Approach 1:
The patent segments data blocks into different groups based on their dependency relationships, represented as nodes in a graph. By analyzing the connectivity and dependency strength between nodes, the system partitions data blocks into segments that can be allocated to different memory banks, reducing collisions while maintaining efficient memory utilization through targeted placement of related blocks.
Data Source
AI summary
A method for controlling a data allocatable neural processing unit is provided. The method may comprise: determining a dependency determination between data blocks of a specific layer of at least one neural network model; allocating memory banks to store the data blocks based on the dependency determination; and storing each data block in the allocated memory bank.