Program Directive Generation Using Neural-Guided Beam Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing programming languages face challenges in automatically finding the fastest schedule for program execution on target hardware due to a prohibitively large search space and inaccurate cost model estimations, leading to inefficient scheduling.
Innovation Solution
A hybrid approach combining beam search with a neural network-based cost model and random search is used to select beam nodes in a tree structure, adjusting the ratio of neural network usage to random search based on search completion percentage to efficiently determine a schedule for program execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exhaustive search is used to find the fastest schedule, then scheduling accuracy is improved, but the search space becomes prohibitively large and computation time increases
Solution Approach 1:
The patent segments the exhaustive search process into two phases: (1) a restricted search space evaluated by a neural network cost model to identify promising candidate schedules, and (2) a focused verification phase on selected candidates. This segmentation allows the system to maintain high scheduling accuracy while avoiding the computational burden of evaluating all possible schedules in the complete search space.
Solution Approach 2:
The patent performs preliminary filtering of the search space using a neural network-based cost model before conducting detailed schedule evaluation. By pre-identifying promising candidate schedules through cost estimation, the system narrows down the search space to only the most relevant candidates, thereby maintaining accuracy while reducing computational complexity.
2Loss of time
If a restricted search space is used, then computation time is reduced, but scheduling accuracy deteriorates due to missing optimal schedules
Solution Approach 1:
The patent implements a feedback mechanism where the neural network cost model is trained on actual schedule performance data and continuously refined. The cost model uses feedback from verified schedule outcomes to improve its predictions, enabling it to more accurately identify promising candidates in the restricted search space, thereby reducing the risk of missing optimal schedules while maintaining fast computation.
Solution Approach 2:
The patent dynamically adjusts the restricted search space parameters (such as schedule template configurations and operation ordering constraints) based on problem characteristics and neural network predictions. By adaptively changing search space parameters rather than using a fixed restriction, the system maintains computational efficiency while improving the likelihood of capturing optimal schedules.
3Productivity
If traditional cost models are used for schedule evaluation, then computation is faster, but estimation accuracy is insufficient leading to poor scheduling decisions
Solution Approach 1:
The patent replaces traditional mechanical cost modeling approaches (based on hand-crafted heuristics and simplified assumptions) with a neural network-based system. The neural network learns complex cost relationships from training data, providing more accurate estimates while maintaining computational efficiency. This substitution enables the system to achieve both high scheduling speed and improved estimation accuracy simultaneously.
Solution Approach 2:
The patent performs preliminary training of the neural network cost model on extensive schedule data before deployment. This preliminary action pre-computes and stores learned cost patterns, enabling the model to provide accurate estimations during actual scheduling without requiring complex real-time calculations, thus achieving both speed and accuracy.
Data Source
AI summary
A computing system generates directives of a program. Beam nodes are selected one level at a time from multiple nodes in the tree structure. Each node represents a subset of operations in the program. A first number of the beam nodes are selected at a given level of the tree structure. The selection of the first number of the beam nodes uses a cost model that is based on a neural network. A second number of the beam nodes are selected using a random search. The ratio of the first number to the second number is determined based on a search completion percentage at the given level. A path is identified that passes through respective beam nodes at multiple levels of the tree structure. The path represents a schedule for executing the program on a target machine. Then the directives corresponding to the schedule are generated.