Neural Network Scheduling via Memory Benefit Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face increased latency and inefficient hardware utilization due to large network sizes and frequent data transfers, necessitating improved allocation of CNN operations to limited computing resources.
Innovation Solution
A processor-implemented scheduling method that generates operation sets based on loop structures, prioritizes them using a priority table determined by memory benefits, and schedules them to optimize memory utilization and reduce data transfer, involving the use of directed acyclic graphs and threshold values for ordering operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CNN operations are executed with frequent data transfers to handle large network sizes, then computation accuracy is maintained, but latency increases and hardware utilization becomes inefficient
Solution Approach 1:
The patent segments CNN operations into operation sets based on loop structures and data reuse patterns. By dividing the computation into discrete schedulable units that can be executed efficiently with minimal data transfer, the system maintains computation accuracy while reducing latency caused by frequent memory accesses.
Solution Approach 2:
The patent performs preliminary analysis of loop structures to identify data reuse opportunities before execution. By pre-calculating memory benefits and generating priority tables that predict data access patterns, the system prepares optimization strategies in advance, reducing runtime latency without compromising computation accuracy.
2Productivity
If more computing resources are allocated to handle large network sizes, then processing capability is improved, but hardware utilization efficiency decreases due to limited resources
Solution Approach 1:
The patent dynamically changes scheduling parameters based on memory benefits calculated from loop structures. By adjusting operation set priorities and execution timing according to data reuse characteristics, the system optimizes hardware utilization efficiency while maintaining processing capability for large network sizes.
Solution Approach 2:
The patent implements a feedback mechanism where memory benefits are calculated based on actual data reuse patterns, and this information feeds into priority table generation. This closed-loop approach ensures that scheduling decisions continuously adapt to resource availability, improving hardware utilization efficiency while maintaining processing capability.
3Quantity of substance
If data reuse is increased to reduce data transfer size, then memory efficiency is improved, but operation scheduling complexity increases
Solution Approach 1:
The patent segments operations into sets based on loop structures, which naturally group operations that share data. This segmentation reduces scheduling complexity by creating manageable units with predictable data reuse patterns, while still achieving reduced data transfer size through intelligent grouping.
Solution Approach 2:
The patent calculates memory benefits as a quantitative parameter to guide scheduling decisions. By transforming the complex problem of data reuse optimization into a parameter-driven priority system, the patent reduces scheduling complexity while maintaining effective data reuse to minimize data transfer size.
Data Source
AI summary
A method and apparatus with scheduling a neural network (NN), which relate to extracting and scheduling priorities of operation sets, are provided. A scheduler may be configured to receive a loop structure corresponding to a NN model, generate a plurality of operation sets based on the loop structure, generate a priority table for the operation sets based on memory benefits of the operation sets, and schedule the operation sets based on the priority table.