Learned Graph Compiler Optimization for Unseen ML Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing compiler optimization techniques for machine learning frameworks suffer from sub-optimal configurations due to the use of hand-tuned heuristics, especially for unseen model architectures, and fail to leverage joint optimizations across tasks, leading to inefficiencies in computing resources and run time.
Innovation Solution
A compiler optimization network using a deep reinforcement learning model with an inductive graph-embedding network and segmented recurrent attention layers is trained jointly over a set of computation graphs to learn generalizable policies for device assignment, scheduling, and operation fusion, enabling faster convergence and improved performance on unseen graphs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If hand-tuned heuristics are used for compiler optimizations, then the implementation is simple and fast to deploy, but the optimization quality is sub-optimal especially for unseen model architectures
Solution Approach 1:
The patent replaces hand-tuned heuristic rules (mechanical system) with a machine learning model that learns optimization policies from data. The ML model processes computation graphs and generates optimized code assignments, replacing the manual rule-based approach with a data-driven intelligent system that adapts to different model architectures.
Solution Approach 2:
The patent changes the parameters of the compiler optimization system by using learned policies from machine learning models instead of fixed hand-tuned heuristics. The ML model learns optimal parameter configurations (such as graph partitioning strategies, operation fusion rules, and resource allocation parameters) from training data, enabling adaptive optimization for unseen architectures.
2Manufacturing precision
If compiler optimizations are solved in isolation using individual heuristics, then each optimization task is simple to implement, but joint optimizations across tasks are missed leading to sub-optimal configurations
Solution Approach 1:
The patent merges multiple individual optimization tasks (graph partitioning, operation fusion, resource allocation, scheduling) into a unified machine learning model. The model processes the computation graph and simultaneously determines optimal assignments across all these dimensions, capturing interdependencies between tasks that isolated heuristics cannot address.
Solution Approach 2:
The patent creates a universal optimization system where a single machine learning model performs multiple optimization functions simultaneously. The model can handle different optimization tasks (partitioning, fusion, allocation, scheduling) within a unified framework, making the system multi-functional and capable of joint optimization across all compiler tasks.
3Productivity
If traditional compilers are used for machine learning frameworks, then compatibility with existing code is maintained, but run time efficiency for training and inference is dramatically impacted
Solution Approach 1:
The patent replaces traditional compiler mechanisms with machine learning-based optimization. Instead of relying on conventional compilation pipelines and hand-crafted heuristics, the system uses trained ML models to analyze computation graphs and generate optimized execution plans, achieving dramatically improved runtime efficiency for ML workloads.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for compiler optimizations using a compiler optimization network. One of the methods includes receiving an input program, wherein the input program defines a graph of operation modules, wherein each node in the graph is a respective operation module, and each edge between nodes in the graph represents one operation module receiving the output generated by another operation module. The input program is processed by a compiler optimization network comprising a graph-embedding network that is configured to encode operation features and operation dependencies of the operation modules of the input program into a graph embedding representation and a policy network that is configured to generate an optimization action for each of one or more nodes encoded in the graph embedding representation. The compiler optimization network generates an output optimization plan comprising one or more optimization actions for the input program.