Graph Neural Network Policy for Neural Network Execution Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for optimizing the execution of neural networks are device-specific, require constant tuning with expert knowledge, and are not transferable across different neural networks or devices, leading to inefficiencies and high computational costs.
Innovation Solution
A system that generates an execution optimization policy for neural networks by representing them as computational graphs, allowing for the optimization of multiple execution tasks simultaneously and being invariant to the underlying graph topology, thus enabling efficient execution across a wide range of devices and model architectures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If hand-tuned heuristics are used to optimize neural network execution, then optimization can be achieved for specific devices, but the system cannot be generalized to other devices or architectures
Solution Approach 1:
The patent uses graph neural networks to learn execution policies from training graphs and applies these learned policies to unseen computational graphs. Instead of hand-tuning heuristics for each device, the system learns generalizable patterns from training data and copies this knowledge to new devices and architectures, achieving adaptability without proportional increases in system complexity
Solution Approach 2:
The system changes the approach from fixed hand-tuned parameters to learned parameters through graph neural networks. The execution policies are represented as learnable parameters that are optimized during training and can be adapted to different devices by adjusting these parameters, rather than redesigning the entire optimization system for each device
2Productivity
If existing systems are re-trained for each individual neural network, then optimal policies can be generated for that specific network, but the trained parameters are not transferrable to other neural networks
Solution Approach 1:
The graph neural network is designed to be universal across different neural network architectures and computational graphs. The same trained model can process various types of graphs (different neural network architectures) and generate appropriate execution policies without re-training, making the system multi-functional and transferable across different productivity scenarios
Solution Approach 2:
The system performs preliminary training on a diverse set of computational graphs during the offline phase. This preliminary action prepares the graph neural network to handle various architectures, so when a new neural network needs optimization, the policy can be generated immediately without time-consuming re-training, thus reducing loss of time while maintaining productivity
3Manufacturing precision
If existing techniques generate decisions for a single node per iteration, then detailed optimization can be achieved, but the time required to generate execution optimization policies becomes computationally expensive
Solution Approach 1:
The computational graph is segmented into individual nodes that can be processed independently by the graph neural network. Each node's execution policy is determined based on its local features and neighborhood information, allowing parallel processing of multiple nodes simultaneously. This segmentation enables detailed optimization of each node while reducing overall policy generation time through parallel computation
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for optimizing the execution of the operations of a neural network. One of the methods includes obtaining data representing a graph characterizing a plurality of operations of a neural network, wherein each node of the graph characterizes an operation of the neural network and each edge of the graph characterizes data dependency between the operations; processing the data representing the graph using a graph embedding neural network to generate an embedding of the graph; and processing the embedding of the graph using a policy neural network to generate a task output, wherein the task output comprises, for each of the plurality of operations of the neural network, a respective decision for a particular optimization task.