Interoperable Neural Network Scheduler for Cross-Platform Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network (NN) tools and frameworks lack an intelligent mechanism to account for various computational requirements, leading to inefficiencies as optimizations for one platform are often incompatible or non-optimal for another, requiring manual repetition of optimization techniques across different execution environments.
Innovation Solution
A scheduler module that accepts NN models in an interoperable format, optimizes them, and decides where to execute operations based on annotated information about hardware capabilities, using weighting factors and feedback loops to optimize execution across multiple hardware platforms, reducing development time and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual optimization techniques are applied to each platform separately, then optimization effectiveness is improved, but development time and complexity increase
Solution Approach 1:
The system segments the optimization process by separating platform-specific optimization details from the high-level optimization strategy. The scheduler module divides the NN operations into discrete tasks that can be independently analyzed and scheduled on different hardware platforms, allowing optimization to be applied once and then automatically adapted to multiple platforms.
Solution Approach 2:
The scheduler module acts as an intermediary between the NN framework and the hardware platforms. It receives optimization requirements in an interoperable format, analyzes hardware capabilities, and automatically generates platform-specific execution schedules without requiring manual re-optimization for each platform.
2Reliability
If platform-specific optimizations are implemented, then performance on each platform is improved, but adaptability across platforms deteriorates
Solution Approach 1:
The scheduler module provides universal functionality by accepting NN models in an interoperable format that can be executed on multiple different hardware platforms. It maintains a library of hardware capability descriptors and can automatically adapt the execution schedule to match the specific capabilities of any target platform, making a single optimization implementation applicable across diverse hardware architectures.
3Productivity
If comprehensive hardware optimization is performed, then execution efficiency is improved, but system complexity increases
Solution Approach 1:
The system performs preliminary analysis of hardware capabilities and NN operation requirements before execution. The scheduler module pre-analyzes the hardware descriptors and operation characteristics to determine optimal scheduling strategies in advance, rather than performing complex optimizations during runtime execution, thereby reducing execution time while maintaining high efficiency.
Data Source
AI summary
A Neural Network (NN) scheduler and techniques to implement features of different possible NN schedulers are disclosed. In a first example, an NN scheduler that accepts NN models in an interoperable format and performs optimizations on this interoperable format as part of converting it to a run-time format is provided. In a second example, an NN scheduler analyzes operations and annotations associated with those operations to determine scheduling options based on hardware availability, data availability, hardware efficiency, processor affinity, etc. In a third example, an NN scheduler that may be integrated with a feed-back loop to recognize actual run-time attributes may be used to “learn” and adapt to change its future scheduling behavior. Each of these examples may be integrated individually, or together, to provide an NN scheduler that optimizes and adapts processing functions for an NN model either prior to processing or for just-in-time determination of operation scheduling.