Neural Network Tile Dependency and Memory Queue Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks are compute-intensive and generate significant intermediate result data, leading to high power consumption and limitations in their use in mobile devices and power-constrained environments.
Innovation Solution
A processor-implemented method that determines an operation sequence for a neural network based on dependency information of tiles constituting a feature map and layer information, generating commands to control feature map memory and operators, thereby optimizing memory access and reducing computational overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural networks are executed to process complex input data, then computational capability is improved, but power consumption increases
Solution Approach 1:
The feature map is divided into multiple tiles that are processed in parallel across different memory queues. Each tile can be independently handled by dedicated memory queues, enabling fine-grained parallelism and reducing the computational burden on any single processing unit while maintaining overall computational capability.
Solution Approach 2:
The patent introduces a memory queue dimension to organize data flow through the neural network layers. By arranging memory queues in a layered structure corresponding to neural network layers, data can be efficiently routed and processed through multiple dimensions of computation, improving computational throughput while managing power consumption through structured data flow.
2Quantity of substance
If external memory is used to store intermediate result data and weights, then data storage capacity is improved, but memory access overhead increases
Solution Approach 1:
Memory queues are segmented and organized by neural network layers, with each queue handling specific layer operations. This segmentation allows for optimized memory access patterns where data is retrieved and processed in structured batches, reducing random access overhead and improving memory access efficiency while maintaining large storage capacity.
Solution Approach 2:
The system performs preliminary organization of data into memory queues corresponding to neural network layer dependencies. By pre-arranging data flow and establishing memory access sequences before execution, the system reduces runtime memory access overhead and improves data retrieval efficiency.
3Productivity
If tiles are processed in parallel across memory queues, then computational efficiency is improved, but memory management complexity increases
Solution Approach 1:
Memory queues are designed with multi-functionality, serving multiple purposes: data storage, data routing, and computational task coordination. Each memory queue can handle different types of operations (convolution, activation, pooling) and serves as a universal interface between data storage and processing units, simplifying memory management despite parallel processing requirements.
Solution Approach 2:
Memory queues act as intermediaries between data storage and computational operations. They buffer data, manage flow control, and coordinate access between parallel processing units, reducing direct complexity in memory management by introducing an intermediate layer that abstracts the complexity of parallel data handling.
Data Source
AI summary
A processor-implemented method includes: determining an operation sequence of a neural network based on dependency information of a tile constituting a feature map of the neural network and layer information of the neural network; and generating a first command for controlling a feature map memory and a second command for controlling an operator based on the operation sequence, wherein the first command comprises information on a tile input to each of a plurality of memory queues constituting the feature map memory.