Method and system for generating indicator of program

Abstract

The invention provides a method and a system for generating an indicator of a program. The method for generating the indicator of the program provided by the invention comprises the following steps of: selecting beam nodes in a plurality of nodes in a tree structure layer by layer, wherein each node represents a part of operations in the program; in a given layer of the tree structure, a first number of beam nodes is selected using a neural network-based cost model, and a second number of beam nodes is selected using a random search, where a ratio of the first number to the second number is determined based on a search completion percentage of the given layer, identifying a path through each beam node of a plurality of layers of the tree structure, wherein the path represents a schedule of executing the program on a target machine; and generating an indicator corresponding to the schedule.

Description

Technical Field

[0001] Embodiments of the present invention relate to the automatic generation of indicators for computer programs; more particularly, to determining indicators using random search and neural networks. Background Art

[0002] High-level programming languages, such as Halide, allow programmers to separate a program's algorithm from the program's scheduling. The algorithm defines what the programmer wants to compute, such as a sequence of functions. The schedule defines how the program should be executed. Given a program's algorithm, the search space of all possible schedules can be so large that finding the fastest executing schedule on a target machine becomes infeasible. Previous research has attempted to solve the scheduling problem by searching in overly restricted search spaces or relying on inaccurate cost model estimates. Therefore, there is a need to improve the automatic scheduling of programs for fast execution on target hardware. Summary of the invention

[0003] The present invention provides a method and system for generating an indicator of a program.

[0004] In an optional embodiment, the present invention provides a method for generating an indicator of a program, which may include: selecting beam nodes from multiple nodes in a tree structure layer by layer, wherein each node represents a portion of operations in the program; at a given layer of the tree structure, selecting a first number of beam nodes using a neural network-based cost model, and selecting a second number of beam nodes using random search, wherein a ratio of the first number to the second number is determined based on a search completion percentage of the given layer, identifying a path through each beam node of multiple layers of the tree structure, wherein the path represents a schedule for executing the program on a target machine; and generating an indicator corresponding to the schedule.

[0005] In another optional embodiment, the present invention provides a system for generating an indicator of a program, which may include: one or more processors; and a memory for storing the program, wherein the one or more processors operate to: select beam nodes from multiple nodes in a tree structure layer by layer, wherein each node represents a portion of operations in the program; at a given layer of the tree structure, select a first number of beam nodes using a neural network-based cost model, and select a second number of beam nodes using random search, wherein a ratio of the first number to the second number is determined based on a search completion percentage of the given layer, identify a path through each beam node of multiple layers of the tree structure, wherein the path represents a schedule for executing the program on a target machine; and generate an indicator corresponding to the schedule. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 A diagram showing a hybrid approach to beam searching according to one embodiment.

[0007] Figure 2 An example diagram showing adjustments to the usage and beam size of a cost model is shown according to one embodiment.

[0008] Figure 3 is a block diagram of a cost model according to one embodiment.

[0009] Figure 4 is a block diagram of a system 400 for generating an indicator of a program according to one embodiment.

[0010] Figure 5 is a flow chart of a method 500 for generating an indicator of a program according to one embodiment. DETAILED DESCRIPTION

[0011] In the following description, many specific details are provided. However, it should be understood that embodiments of the present invention can be implemented without these specific details. In other cases, well-known circuits, structures and techniques are not shown in detail to avoid obscuring the understanding of the present invention. However, those skilled in the art will understand that the present invention can be implemented without these specific details. Through the description herein, a technician with ordinary skill will be able to implement appropriate functions without undue experimentation.

[0012] Embodiments of the present invention provide a directive generator that automatically generates directives for a computer program (referred to as a "program"). These directives describe how to schedule a program for execution and can guide a compiler to optimize program execution on a target machine. A target machine can be any processing device having one or more processors and memory. Although the term "compiler" is used throughout the description herein, it should be understood that the techniques described herein are also applicable to interpreters. A program is divided into multiple blocks, such as functions, code blocks (e.g., instruction sequences), and the like. In one embodiment, a directive generator generates one or more directives for each block. Based on the directives, the compiler generates executable code that is optimized to utilize hardware resources provided by the target machine, such as multiple processing units, memory bandwidth, and the like. When generating directives for each block, the directive generator selects from a plurality of directive candidates, each of which causes the program to be executed according to a different schedule. The schedule indicates how the operation of the program is performed; for example, how to allocate the computing resources of the target machine (such as processors and memory) to perform the operation. The schedule that gives the target machine the fastest estimated run time is selected. The schedule can be mapped to the directive for each block. The indicator for each block provides guidance to the compiler as to how to compile the instructions in the block; for example, as to how computing resources are to be used by the instructions.

[0013] In one embodiment, the indicator generator performs a search in a directed acyclic graph (DAG) (e.g., a tree structure) to determine a schedule that has the fastest estimated run time on the target machine. The nodes of the tree represent operations of the program to be executed. In the following description, the term "schedule" is used to refer to the search results. It is understood that the schedule corresponds to an indicator of the program, which is used to guide the compiler to generate compiled code.

[0014] In one embodiment, beam search is used to limit the search space. In one embodiment, a cost model is used to predict the number of processing cycles required for the target machine to execute the program. The cost model is based on a trained neural network. Using the cost model can avoid using the target machine to actually run the program. In one embodiment, a hybrid approach is used to perform beam search. The hybrid approach uses a combination of random search and a cost model.

[0015] Figure 1 A diagram of a hybrid method for beam search is shown according to one embodiment. A tree structure (e.g., tree 100) represents the operation of a program, wherein each node represents a program block (i.e., a set of operations). Different paths from the root to the leaves represent different schedules for program execution. The tree 100 is expanded and searched starting from the root, one layer at a time. At each layer of the tree 100, a local evaluation is performed to prune a predetermined number of nodes (nodes represented by dashed circles). Then, the remaining nodes of the layer (referred to as candidate nodes) are evaluated to select B nodes as beam nodes (represented by solid black nodes). The number of beam nodes at each layer is referred to as beam width. Only beam nodes are expanded in the next layer for further search. The scheduling decision is an evaluation performed at each layer, aiming to select beam nodes. In one embodiment, the search for beam nodes is based on a combination of a cost model 110 and a random search 120. In one embodiment, different layers of the tree may have different beam widths. The search result is a path formed by beam nodes from the root to the leaf node, wherein there is a beam node on the path on each tree layer. Each tree layer represents a stage of searching for beam nodes.

[0016] At the beginning of the search, the accuracy of the predicted run times of the cost model 110 is low. This is because the cost model 110 is trained using a program with a complete schedule, but in the early stages of the search, the cost model 110 is queried when the program is partially scheduled. Therefore, in the early stages of the search, the random search 120 is as good as the cost model 110 in terms of accuracy and can be used to generate all or most of the beam nodes in the tree layer. The random search 120 requires minimal computational cost and is therefore much faster than the search of the cost model 110. As the search progresses, the accuracy of the cost model improves as more and more schedules are determined. As the accuracy of the cost model improves, more and more nodes can be evaluated by the cost model to generate a larger portion of the beam nodes.

[0017] Figure 1 The example shown in shows that the tree 100 includes a root and four layers from top to bottom: L1, L2, L3 and L4. To simplify the explanation, this example shows a beam search with two constant beam widths. In addition, the tree 100 contains much fewer nodes and layers than the tree usually used to generate indicators.

[0018] In this example, random search 120 is used to search for beam nodes from candidate nodes in the dashed box, and cost model 110 is used to search for beam nodes from candidate nodes in the dotted box. At the L1 layer, only random search 120 is used to identify two beam nodes. At the L4 layer, only cost model 110 is used to identify two beam nodes. At intermediate layers (e.g., L2 and L3 layers), a combination of cost model 110 and random search 120 is used. The usage ratio of the cost model to random search can be determined by analyzing the cost model 110 on the target machine before the search. More specifically, this ratio can be determined based on the accuracy of the cost model 110 at different stages of the search. The higher the accuracy of the cost model, the higher the usage rate of the cost model.

[0019] The complete schedule of the program is represented by a path from the root to the beam nodes at the bottom level of the tree. This schedule is formed by a series of scheduling decisions, where each scheduling decision at each tree level is made for a selected beam node on the path. Each scheduling decision corresponds to a set of indicators for the selected beam node. The indicators for all beam nodes on the path are sent to the compiler to guide the compiler in generating code.

[0020] Figure 2An example graph showing adjustments to cost model usage and beam size is shown according to one embodiment. It should be understood that the numbers shown in the example are illustrative and not limiting. Cost model usage is represented by solid white bars and random search usage is represented by diagonal lines. The right side of the graph shows that the ratio of cost model usage to random search usage increases as the search proceeds toward completion. In addition, the thick black line (i.e., line 210) represents the beam size, which decreases as the search proceeds toward completion. The search completion percentage can be measured by the percentage of programs that have been evaluated and scheduled. The search completion percentage may correspond to the ratio of the number of tree layers evaluated to the total number of tree layers. In this example, the first 30% of the search uses only random search. This 30% of the search may correspond to multiple layers of the search tree.

[0021] It is worth noting that higher usage of random search corresponds to larger beam sizes. Larger beam sizes are used at the beginning of the search (when the search space is relatively manageable by the computer running the search). Large beam sizes at the beginning of the search are intended to include as many candidate nodes as possible to minimize the probability that random search fails to select a good candidate node. As the search progresses, the beam size decreases, which corresponds to higher accuracy and higher usage of the cost model. The beam size is reduced in the later stages of the search to avoid exploding the computational resources required for the search.

[0022] In one embodiment, the candidate nodes for each tree layer are divided into two non-overlapping groups. The first group is evaluated by the cost model 110 to select a set of Ci beam nodes, and the second group is randomly sampled to select a set of Ri beam nodes, where Ci+Ri=Bi, Bi is the number of beam nodes in the i-th tree layer. In addition, the ratio of the first group size to the second group size is equal to Ci / Ri, or substantially equal to Ci / Ri (within a predetermined tolerance range), where the size of the group is the number of candidate nodes in the group. As the search proceeds, the ratio of Ci to Ri increases. More specifically, as the search completion percentage increases from 0 to 100, the ratio of Ci to Ri increases.

[0023] In one embodiment, the values ​​of Ci and Ri can be determined by performing a performance analysis on the target machine. The performance analysis evaluates the accuracy of the cost model when used to estimate the runtime of executing the partial scheduler on the target machine. The estimated runtime generated by the cost model is compared to the actual runtime of running the full scheduler on the target machine. In addition, the performance analysis evaluates the accuracy of the cost model for the partial scheduler on the target machine at different schedule completion percentages.

[0024] Figure 3 is a block diagram of a cost model according to one embodiment. The cost model may be Figure 1An example of a cost model 110 in FIG. The cost model 110 uses a neural network to predict the running time of a program on a target machine. The running time can be represented by the number of processing cycles of the target machine executing the program according to a given schedule. As previously described, the schedule can be provided to the compiler as an indicator. For example, the compiler can receive a set of indicators for each code block of the program. The neural network is trained using a series of programs (called training programs) and corresponding indicators at the input end and the number of processing cycles on the target machine at the output end. The weights and parameters of the neural network are determined during the training process.

[0025] exist Figure 1 In the example search process in , the input of the cost model 110 includes a scheduling feature 310 and an algorithm feature 320. The algorithm feature 320 includes the operations and functions of the program to be scheduled. The scheduling feature 310 captures resource usage behavior, such as the use of processors and memory. The scheduling feature 310 is input to a logarithmic layer 311, a fully connected (FC) layer 312, and a ReLU layer 313 to generate a low-dimensional vector called a scheduling embedding 330. The algorithm feature 320 is input to a FC layer 321 to generate a low-dimensional vector called an algorithm embedding 340. The stacking layer 350 stacks the scheduling embedding 330 and the algorithm embedding 340 to produce a stacked embedding 360, which is input to three FC layers 370, 380, and 390 to output a scalar value, namely the predicted number of cycles. In one embodiment, the FC layer 370 increases the size of the stacked embedding 360, while the FC layers 380 and 390 reduce the size to 1. In one example, if the size of stacked embedding 360 is 32, FC layer 370 may increase the size from 32 to 128, FC layer 380 reduces the size from 128 to 64, and FC layer 390 further reduces the size from 64 to 1, the number of cycles predicted.

[0026] Figure 4 4 is a block diagram of a system 400 for generating an indicator of a program according to one embodiment. The system 400 includes processing hardware 410, a memory 420, and a network interface 430. It should be understood that the system 400 is simplified for illustration and additional hardware and software components are not shown. Non-limiting examples of the processing hardware 410 may include one or more processors, including but not limited to a central processing unit (CPU), a graphics processing unit (GPU), an image processing unit, a deep learning accelerator (DLA), etc. One or more processors, processing units, and / or devices in the processing hardware 410 may perform the above-described search process.

[0027] The memory 420 may store a computer program 450, a cost model 460 (which may be Figure 1 and Figure 34) and an indicator generator 470. Computer program 450 may be written in a programming language (non-limiting examples include Halide, C, C++, etc.). Indicator generator 470 may be executed by processing hardware 410, such as one or more processors. Indicator generator 470 uses a combination of cost model 460 and random search to determine the schedule of computer program 450 and generates corresponding indicators. In one embodiment, memory 420 may further include a compiler and / or interpreter to receive indicators generated by indicator generator 470. Memory 420 may include dynamic random access memory (DRAM) devices, flash memory devices, and / or other volatile or non-volatile storage devices. In one embodiment, indicator generator 470 uses a reference Figure 5 The described method 500 generates an indicator.

[0028] Figure 5 is a flow chart of a method 500 for generating an indicator of a program according to one embodiment. In one embodiment, the method 500 may be Figure 4 The system 400 in is executed.

[0029] Method 500 begins at step 510, in which the system selects (one layer at a time) beam nodes from a plurality of nodes in a tree structure layer by layer. Each node represents a portion of an operation in a program. In step 520, the system selects a first number of beam nodes at a given layer of the tree structure using a neural network-based cost model and a second number of beam nodes using a random search. The ratio of the first number to the second number is determined based on a search completion percentage for the given layer. In step 530, the system identifies a path through each beam node of the plurality of layers of the tree structure. The path represents a schedule for executing the program on a target machine. In step 540, the system generates an indicator corresponding to the schedule.

[0030] In one embodiment, a random search may be used to select all beam nodes that are located at one or more layers directly below the root of the tree structure. As the percentage of search completion increases, the cost model may be used to select an increasing proportion of beam nodes. In one embodiment, the number of beam nodes decreases as the percentage of search completion increases. After the indicator is generated, the indicator may be sent to a compiler to generate compiled code for execution on a target machine.

[0031] In one embodiment, a performance analysis is performed on the cost model to obtain the accuracy of the cost model at different search completion percentages. A ratio of the first number of beam nodes to the second number of beam nodes is adjusted based on the accuracy of the cost model at a given search completion percentage. The accuracy is measured relative to the number of processing cycles predicted on the target machine.

[0032] The cost model is based on a neural network. In one embodiment, the neural network includes three consecutive fully connected layers and outputs a scalar value representing the predicted number of processing cycles.

[0033] In one embodiment, a random search is used to randomly select a first number of beam nodes from a first group of nodes at a given layer. A cost model is used to select a second number of beam nodes from a second group of nodes at the given layer. The second number of beam nodes is predicted by the cost model to have the fastest run time. The first group of nodes does not overlap with the second group of nodes. The first group of nodes has a first group size and the second group of nodes has a second group size. A ratio of the first group size to the second group size is equal to or substantially equal to a ratio of the first number of beam nodes to the second number of beam nodes.

[0034] Figure 5 The operation of the flowchart has been referred to Figure 3 and Figure 4 However, it should be understood that Figure 5 The operations of the flowchart can be performed by Figure 3 and Figure 4 The invention is performed in accordance with the embodiments of the present invention other than the embodiments of the present invention, and Figure 3 and Figure 4 Embodiments of the invention may perform operations different from those discussed with reference to the flowcharts. Figure 5 The flowcharts show a particular order of operations performed by certain embodiments, but it should be understood that this order is exemplary (for example, alternative embodiments may perform operations in a different order, combine certain operations, overlap certain operations, etc.).

[0035] Various functional components or blocks have been described. As will be appreciated by those skilled in the art, the functional blocks will preferably be implemented by circuits (whether special purpose or general purpose, operating under the control of one or more processors and coded instructions), which will typically include transistors configured to control the operation of the circuits according to the functions and operations described herein.

[0036] Although the present invention has been described in the form of some embodiments, those skilled in the art will recognize that the present invention is not limited to the described embodiments and can be modified and altered without departing from the spirit and scope of the appended claims. Therefore, the description should be regarded as illustrative rather than restrictive.

Claims

1. A method for generating an indicator of a program, characterized in that include: Selecting beam nodes from a plurality of nodes in a tree structure layer by layer, wherein each node represents a part of operations in the program; At a given layer of the tree structure, a first number of beam nodes is selected using a neural network-based cost model, and a second number of beam nodes is selected using a random search, wherein a ratio of the first number to the second number is determined based on a search completion percentage for the given layer, identifying a path through each beam node of a plurality of levels of the tree structure, wherein the path represents a schedule for executing the program on a target machine; as well as An indicator corresponding to the schedule is generated.

2. The method according to claim 1, characterized in that The random search is used to select all beam nodes located at one or more layers directly below the root of the tree structure.

3. The method according to claim 1, characterized in that As the search completion percentage increases, this cost model is used to select an increasing proportion of beam nodes.

4. The method according to claim 1, characterized in that Further including: Performing performance analysis on the cost model to obtain the accuracy of the cost model at different search completion percentages; as well as The ratio of the first amount to the second amount is adjusted according to the accuracy of the cost model at a given search completion percentage.

5. The method according to claim 4, characterized in that The accuracy is measured relative to the number of processing cycles predicted on the target machine.

6. The method according to claim 1, characterized in that The number of beam nodes decreases as the search completion percentage increases.

7. The method according to claim 1, characterized in that The neural network includes three consecutive fully connected layers that output a scalar value representing a predicted number of processing cycles.

8. The method according to claim 1, characterized in that Further including: randomly selecting the first number of beam nodes from the first set of nodes at the given layer; as well as The second number of beam nodes is selected at the given layer from a second set of nodes that do not overlap with the first set of nodes, wherein the second number of beam nodes is predicted by the cost model to have the fastest runtime.

9. The method according to claim 8, characterized in that The first group of nodes has a first group size, the second group of nodes has a second group size, and a ratio of the first group size to the second group size is equal to or substantially equal to a ratio of the first number to the second number.

10. The method according to claim 1, characterized in that This directive is sent to the compiler to generate compiled code that is executed on the target machine.

11. A system for generating an indicator of a program, characterized in that include: one or more processors; as well as a memory for storing the program, wherein the one or more processors operate to: Selecting beam nodes from a plurality of nodes in a tree structure layer by layer, wherein each node represents a part of operations in the program; At a given layer of the tree structure, a first number of beam nodes is selected using a neural network-based cost model, and a second number of beam nodes is selected using a random search, wherein a ratio of the first number to the second number is determined based on a search completion percentage for the given layer, identifying a path through each beam node of a plurality of levels of the tree structure, wherein the path represents a schedule for executing the program on a target machine; as well as An indicator corresponding to the schedule is generated.

12. The system according to claim 11, characterized in that The random search is used to select all beam nodes located at one or more layers directly below the root of the tree structure.

13. The system according to claim 11, characterized in that As the search completion percentage increases, this cost model is used to select an increasing proportion of beam nodes.

14. The system according to claim 11, characterized in that The one or more processors are further operable to: Performing performance analysis on the cost model to obtain the accuracy of the cost model at different search completion percentages; and The ratio of the first amount to the second amount is adjusted according to the accuracy of the cost model at a given search completion percentage.

15. The system of claim 14, wherein: The accuracy is measured relative to the number of processing cycles predicted on the target machine.

16. The system of claim 11, wherein: The number of beam nodes decreases as the search completion percentage increases.

17. The system of claim 11, wherein: The neural network includes three consecutive fully connected layers that output a scalar value representing a predicted number of processing cycles.

18. The system of claim 11, wherein: The one or more processors are further operable to: randomly selecting the first number of beam nodes from a first set of nodes at the given layer; and A second number of beam nodes is selected at the given layer from a second set of nodes that does not overlap with the first set of nodes, wherein the second number of beam nodes is predicted by the cost model to have the fastest runtime.

19. The system of claim 18, wherein: The first group of nodes has a first group size, the second group of nodes has a second group size, and a ratio of the first group size to the second group size is equal to or substantially equal to a ratio of the first number to the second number.

20. The system of claim 11, wherein: This directive is sent to the compiler to generate compiled code to be executed on the target machine.

Images