Convolutional neural network compiler based on depth-first scheduling technology

Abstract

The invention belongs to the technical field of artificial intelligence chips, and particularly relates to a convolutional neural network compiler based on a depth-first scheduling technology. The convolutional neural network compiler comprises a configuration module, a front-end processing module, a layer fusion group division module, a depth-first scheduling module, a cost model module and a code generation module. According to the compiler, an ONNX model is converted into an internal computational graph, a dynamic programming algorithm is adopted to divide layer fusion groups, and a novel depth-first scheduling technology is adopted for each layer fusion group to realize operator cutting and scheduling. According to the method, the compiling effect of the neural network can be effectively improved, redundant calculation is reduced, and more sufficient on-chip data reuse is realized, so that the reasoning speed of the convolutional neural network on a resource-limited neural network accelerator is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence chips, and in particular relates to a convolutional neural network compiler based on depth-first scheduling technology. Background Art

[0002] With the rapid development of neural network technology, it has been widely and deeply applied in various fields such as computer vision. In line with this trend, various neural network accelerator architectures for edge computing scenarios have emerged to meet the demand for high-performance inference in resource-constrained environments. However, due to storage bandwidth and capacity bottlenecks, the efficient deployment of neural networks on hardware accelerators still faces significant challenges and has become a research hotspot that has attracted widespread attention in academia and industry in recent years.

[0003] Depth-first scheduling is a common convolutional neural network compiler optimization technique. However, existing depth-first scheduling requires each layer to maintain the same operator splitting scale and cannot flexibly explore the operator splitting scale of each layer. This limits the compiler's optimization space, resulting in suboptimal compilation and deployment performance, and restricting the inference speed of convolutional neural networks on resource-constrained neural network accelerators. Summary of the Invention

[0004] In order to overcome the shortcomings of the existing technology, the present invention proposes a convolutional neural network compiler based on a novel depth-first scheduling technology, in order to achieve a convolutional neural network compiler with high performance and fast search speed. The compiler can be widely used in inference acceleration of resource-constrained neural network accelerators.

[0005] The convolutional neural network compiler based on depth-first scheduling technology provided by the present invention comprises a front-end processing module, a configuration module, a layer fusion group division module, a depth-first scheduling module, a cost model module and a code generation module; wherein:

[0006] The compiler front-end processing module is used to read the ONNX model file, traverse the operators of each layer of the ONNX model, map the convolutional neural network to the compiler's internal operator library according to the traversed operators, and build a neural network calculation graph inside the compiler;

[0007] The configuration module is configured to receive hardware configuration information, including data reuse upper limit, computing unit computing power, on-chip memory size, on-chip memory access bandwidth, and the balance coefficient between computing and memory access in the cost model. The data reuse upper limit is transmitted to the depth-first scheduling guidance module to guide the depth-first scheduling strategy search, and the other parameters are transmitted to the cost model module to evaluate the cost of the depth-first scheduling strategy;

[0008] The layer fusion group division module is used to receive the neural network calculation graph generated by the front-end processing module and divide the calculation graph into multiple layer fusion groups. This module first finds all candidate layer fusion groups that meet the constraints, and then

[0009] The fusion group attempts to schedule through the depth-first scheduling module, and evaluates the optimal performance of the fusion group of the layer obtained by scheduling. Finally, after obtaining the optimal performance of all candidate layer fusion groups, the dynamic programming algorithm is used to search for the optimal performance of the entire neural network.

[0010] Optimal layer fusion group division strategy.

[0011] The depth-first scheduling module is used to receive the candidate layer fusion groups generated by the layer fusion group division module during the calculation process, use the new depth-first scheduling technology to schedule the layer fusion group and cut it into multiple tiles, and calculate the scheduling results and their costs.

[0012] Return to the layer fusion group division module to guide the layer fusion group division. The new depth-first scheduling technology breaks the traditional depth

[0013] Priority scheduling technology constrains the data reuse rules within the layer fusion group, allowing each layer to be cut at different scales.

[0014] It is possible to flexibly select the tile shape, thereby expanding the optimization space. Specifically, for each adjacent

[0015] In layer pairs, the output data of the tile of the next layer is a combination or segmentation of the output data of multiple tiles of the previous layer.

[0016] The order of execution of each tile is still based on the depth-first principle. The depth-first scheduling module first determines each adjacent

[0017] The optimal scheduling strategy for the layer fusion group is obtained based on the data reuse relationship between layer pairs. Then, all optional tile shapes of the last layer of the layer fusion group are traversed, and the tile shapes and tile execution order of each layer are reversed based on the optimal scheduling strategy. These are then transmitted to the cost control module, which receives and records the returned cost. Finally, the tile shapes and tile execution order of each layer with the lowest cost are returned to the layer fusion group division module together with the cost.

[0018] The cost model module is used to receive the tile shapes of each layer of the fusion group and its

[0019] The execution order is to first verify whether the fusion group of this layer can meet the on-chip memory constraints in the current state.

[0020] The computation cost and memory access cost are calculated by calculating the power and memory access bandwidth in the current state.

[0021] The balance coefficient of memory access is weighted, and the weighted result is returned to the depth-first scheduling module as Cost.

[0022] The code generation module is used to receive the layer fusion group division strategy, the tile shape of each layer in each layer fusion group, and the

[0023] The execution order of each layer of Tile in the fusion group is converted into a machine that can be recognized by the neural network accelerator.

[0024] code to achieve end-to-end deployment of neural networks.

[0025] Compared with the prior art, the beneficial technical effects of the present invention are embodied in:

[0026] (1) The layer fusion group division module uses a dynamic programming algorithm to divide the layer fusion groups, which improves the search speed by avoiding repeated calculations while ensuring the optimal results.

[0027] (2) The depth-first scheduling module uses a new depth-first scheduling technology to perform operator cutting and scheduling. By allowing each layer in the layer fusion group to adopt different operator segmentation scales, it increases the optimization space, reduces redundant calculations, and improves deployment performance.

[0028] (3) The depth-first scheduling module improves the speed of compilation search by decoupling the search for scheduling strategies from the search for specific operator segmentation shapes. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Diagram of the convolutional neural network compiler structure based on the novel depth-first scheduling technology.

[0030] Figure 2 Search diagram for depth-first scheduling strategy. DETAILED DESCRIPTION

[0031] The present invention is further described below through specific examples with reference to the accompanying drawings.

[0032] The convolutional neural network compiler structure provided by the present invention is shown in Figure 1 As shown, it includes configuration module, front-end processing module, layer fusion group division module, depth-first scheduling module, cost model module and code generation module. The specific workflow is as follows:

[0033] First, the compiler's front-end processing module reads the onnx model file, traverses the operators at each layer of the onnx model, and maps the convolutional neural network to the compiler's internal operator library based on the traversed operators, thereby constructing a neural network computation graph within the compiler. The configuration module receives hardware configuration information, including the data reuse limit, computing unit power, on-chip memory size, on-chip memory access bandwidth, and the balance coefficient between computation and memory access in the cost model. The data reuse limit is used to guide the depth-first scheduling module in searching for a depth-first scheduling strategy, while other parameters are used to guide the cost model module in evaluating a depth-first scheduling strategy.

[0034] The layer fusion group partitioning module receives the neural network calculation graph generated by the front-end processing module and divides the calculation graph into multiple layer fusion groups. The module first finds all candidate layer fusion groups that meet the constraints. The constraints include: (1) The upper limit of operators in a single layer fusion group is 7. (2) Operators with multiple subsequent nodes can only serve as output nodes of the layer fusion group, and cannot serve as input or intermediate nodes of the layer fusion group. Then, all candidate layer fusion groups are scheduled through the depth-first scheduling module, and the optimal performance of the layer fusion group obtained by scheduling is evaluated to obtain the data structure LayerGroupInfo, which contains whether the layer fusion group can be executed in the accelerator, the shape of each layer tile in the layer fusion group, the execution order of each layer tile in the layer fusion group, and the performance that the layer fusion group can achieve under the optimal scheduling strategy. Define G as the set of all operators in the calculation graph, G = {P0, P1, P2...Pn}, where Pi represents each operator in the calculation graph, a total of n+1, P0 represents the operator containing the input node, and Pn represents the operator containing the output node. At the same time, define Gi to represent the subgraph from the input node to Pi, Gi = {P0, P1, P2, ..., Pi}. Under this definition, Gn = G. The LayerGroupInfo of all candidate layer fusion groups obtained in the previous step are integrated to create a data structure, LayerGroupDict[Pi] = {L0, L1, L2, ..., Lm}, which records the set of layer fusion groups with operator Pi as the output node, where L∈{L0, L1, L2, ..., Lm} represents each LayerGroupInfo with Pi as the output node, a total of m+1. Finally, after obtaining all candidate layer fusion group information LayerGroupDict, a dynamic programming algorithm is used to search for the optimal layer fusion group partitioning strategy for the entire neural network. A local optimal solution data structure, LocalOptimalDict[Pi] = {L0, L1, L2, ..., Lx}, is created to store the optimal layer fusion group partitioning solution for the subgraph Gi from the input node to Pi. The optimal solution consists of x layer fusion groups. This data structure is key to the dynamic programming algorithm. By saving the local optimal solution to a subgraph, it avoids repeated computation of the subproblems during the computation process. Once LocalOptimalDict[Pn] is calculated, the optimal solution for the layer fusion group combination for the entire network is obtained. The tile shape and execution order within each layer fusion group are also determined in the previous scheduling.

[0035] The depth-first scheduling module receives the candidate layer fusion groups generated by the layer fusion group division module during the calculation process, uses the new depth-first scheduling technology to cut and schedule the layer fusion groups, and returns the scheduling results and their costs to the layer fusion group division module to guide the layer fusion group division. The new depth-first scheduling technology breaks the constraints of the traditional depth-first scheduling on the data reuse rules within the layer fusion group, allowing each layer to be cut at different scales. Each layer can flexibly choose its tile shape, thereby expanding the optimization space. Specifically, for each adjacent layer pair in the layer fusion group, the input data shape of the tile of the next layer is no longer consistent with the output data shape of the tile of the previous layer. On the contrary, the tile shapes between adjacent layer pairs can have two relationships, which are represented by the data reuse type Reuse Type: (1) The output data of the tile of the next layer is a combination of the output data of multiple tiles of the previous layer. We call this relationship Combination Reuse. (2) The output data of the tile of the next layer is part of the input data of a tile of the previous layer. We call this relationship Split Reuse. Based on the Reuse Type, this technology uses the Reuse Ratio to record the ratio of tile shape combinations or splits between adjacent layers. For Combination Reuse, the Reuse Ratio can be n, where n∈{1,2,...,ReuseLimit}; for Split Reuse, the Reuse Ratio can be 1 / n, where n∈{1,2,...,ReuseLimit}. The Reuse Limit is the upper limit on data reuse configured through the configuration module, representing the maximum number of combinations or splits between adjacent layers. This technology continues to follow the depth-first principle when determining the execution order of each tile: when the input data of a tile in the subsequent layer in the depth direction is ready, the calculation of that tile is prioritized. If there are no tiles in the subsequent layer in the depth direction that can be calculated, the calculation of the remaining tiles in the current layer continues. The output data generated by all tile calculations is stored in on-chip memory until all subsequent tiles that use this data are calculated. This data is called reused data and is included in the memory usage calculation of each layer.

[0036] The depth-first scheduling module first determines the Reuse Type and Reuse Ratio between each adjacent layer pair by searching, and obtains the optimal scheduling strategy for the layer fusion group, such as Figure 2As shown. When searching for the optimal scheduling strategy, first count the feature map sizes of each layer in the layer fusion group, calculate the ratio of the feature map sizes between each pair of adjacent layers, and determine the Reuse Type between each adjacent layer pair. If the ratio of the feature map size of the current layer to the feature map size of the subsequent layer is greater than or equal to 1, the data reuse relationship representation of the adjacent layer pair is set to Combination Reuse; conversely, if the ratio of the feature map size of the current layer to the feature map size of the subsequent layer is less than 1, the data reuse relationship representation of the adjacent layer pair is set to SplitReuse. At the same time, construct a search space for the Reuse Ratio of each adjacent layer pair. For each adjacent layer pair, there will be ReuseLimit candidate Reuse Ratios, so the search space size of the entire layer fusion group scheduling strategy is Reuse Limit^Reuse Ratio. Subsequently, the depth-first scheduling module will traverse all candidate scheduling strategies (i.e., Reuse Ratios combinations) in this search space. For each traversed scheduling strategy, the feature map size of each layer in the layer fusion group will be relatively normalized based on the output feature map size of the last layer. The normalized input and output feature map size of each layer will be calculated according to the Reuse Ratio of each adjacent layer under the currently traversed scheduling strategy, and the normalized memory usage of each layer will be further calculated. The memory utilization of each layer is the memory usage of the layer divided by the maximum memory usage in the fusion group of the layer. The memory utilization of each layer is then weighted using the following formula to calculate the weighted memory utilization:

[0037] α i =kernel i -stride i ,

[0038]

[0039] Among them, FM_size i The memory usage of each layer, i is the memory utilization of each layer under the current scheduling strategy, weighted_utilization i is the weighted memory utilization of the fusion group of this layer in the current scheduling strategy, kernel i is the convolution kernel size of each convolution layer, stride i Calculate the convolution step size for each convolutional layer. By traversing all scheduling strategies, find the scheduling strategy with the highest weighted memory utilization and determine it as the optimal scheduling strategy for the current layer fusion group, which includes the reuse ratio and reuse type between adjacent layers.

[0040] After obtaining the optimal scheduling strategy, the depth-first scheduling module will calculate the specific tile shapes of each layer. First, a search space for the tile shape of the last layer is constructed. The last layer in the fusion group of this layer can adopt any tile shape that does not exceed the boundary of the operator shape, forming the search space for the tile shape of the last layer. Then, the search space is traversed. For the tile shape of the last layer traversed, the tile shape of each layer is calculated from bottom to top according to the optimal scheduling strategy of the current layer fusion group, and its execution order is scheduled. The tile shape and execution order of each layer are then transmitted to the cost model module. The cost model module will return the cost as the cost of the current tile shape. After traversing the search space, the tile shape of each layer with the lowest cost and its execution order will be the final result of the depth-first scheduling module and will be returned to the layer fusion group division module together with the lowest cost. If the cost returned under all scheduling strategies is positive infinity, the layer fusion group cannot perform depth-first scheduling on the current device, and this status will be returned to the layer fusion group division module.

[0041] The cost model module receives the tile shapes and execution order of each layer of the fusion group from the depth-first scheduling module. It first calculates the peak memory usage of the fusion group in the current state. If it exceeds the on-chip memory size, the returned cost is positive infinity. Then, based on the computing unit's computing power and the off-chip memory bandwidth, it calculates the computation and memory access costs in the current state. The computation cost is the computation amount of each tile divided by the computing unit's computing power, and the memory access cost is the number of bytes of the input data of the fusion group in the layer divided by the off-chip memory bandwidth. Finally, the computation cost and memory access cost are weighted using the computation and memory access balance coefficient, and the weighted result is returned as the cost to the depth-first scheduling module.

[0042] The code generation module receives the layer fusion group division strategy, the shape of each layer tile within each layer fusion group, and the execution order of each layer tile within each layer fusion group. It allocates the address of the calculation data of each tile in the on-chip memory in chronological order, and converts each operator slice in the calculation problem into machine code that can be recognized by the neural network accelerator to realize end-to-end deployment of the neural network.

Claims

1. A convolutional neural network compiler based on depth-first scheduling technology, characterized in that: It includes front-end processing module, configuration module, layer fusion group division module, depth-first scheduling module, cost model module and code generation module; among which: The compiler front-end processing module is used to read the ONNX model file, traverse the operators of each layer of the ONNX model, map the convolutional neural network to the compiler's internal operator library according to the traversed operators, and build a neural network calculation graph inside the compiler; The configuration module is used to receive hardware configuration information, including a data reuse upper limit, computing unit computing power, on-chip memory size, on-chip memory access bandwidth, and a balance coefficient between computing and memory access in a cost model; wherein the data reuse upper limit is transmitted to the depth-first scheduling guidance module for guiding the depth-first scheduling strategy search, and other parameters are transmitted to the cost model module for evaluating the cost of the depth-first scheduling strategy; The layer fusion group division module is used to receive the neural network calculation graph generated by the front-end processing module and divide the calculation graph into multiple layer fusion groups. Specifically, the module first finds all candidate layer fusion groups that meet the constraint conditions, then attempts to schedule all candidate layer fusion groups through the depth-first scheduling module, and evaluates the optimal performance of the layer fusion group obtained by scheduling. Finally, after obtaining the optimal performance of all candidate layer fusion groups, the module searches for the optimal layer fusion group division strategy for the entire neural network through a dynamic programming algorithm. The depth-first scheduling module is used to receive the candidate layer fusion groups generated by the layer fusion group division module during the calculation process, use the depth-first scheduling technology to cut and schedule the layer fusion group, and return the scheduling results and their costs to the layer fusion group division module to guide the layer fusion group division; the depth-first scheduling technology breaks the constraints of the traditional depth-first scheduling on the data reuse rules within the layer fusion group, allowing each layer to be cut at different scales, and each layer can flexibly choose its tile shape, thereby expanding the optimization space; specifically, for each adjacent layer pair in the layer fusion group, the output data of the tile of the latter layer is multiple of the previous layer. The output data of the layer tiles are combined or split; when determining the execution order of each tile, the depth-first principle is still followed; the depth-first scheduling module first determines the data reuse relationship between each adjacent layer pair by searching, and obtains the optimal scheduling strategy for the layer fusion group; then, all optional tile shapes of the last layer of the layer fusion group are traversed, and the tile shapes and tile execution order of each layer are reversed based on the optimal scheduling strategy, and transmitted to the cost control module, which receives and records the returned cost; finally, the tile shape and tile execution order of each layer with the lowest cost are returned to the layer fusion group division module together with the cost; The cost model module is used to receive the tile shapes and execution order of each layer of the fusion group of this layer from the depth-first scheduling module; first verify whether the fusion group of this layer can meet the on-chip memory constraints in the current state; then calculate the computing and memory access costs in the current state based on the computing unit computing power and memory access bandwidth; finally, the computing cost and memory access cost are weighted by the balance coefficient of computing and memory access, and the weighted result is returned as the cost to the depth-first scheduling module; The code generation module is used to receive the layer fusion group division strategy, the tile shape of each layer in each layer fusion group, and the execution order of each layer tile in each layer fusion group, and convert each tile in the calculation graph into machine code that can be recognized by the neural network accelerator to achieve end-to-end deployment of the neural network.

2. The convolutional neural network compiler based on depth-first scheduling technology according to claim 1, characterized in that In the layer fusion group division module, the first step is to find all candidate layer fusion groups that meet the constraints, and the constraints include: (1) the upper limit of operators in a single layer fusion group is 7; (2) operators with multiple subsequent nodes can only be used as output nodes of the layer fusion group, and cannot be used as input or intermediate nodes of the layer fusion group; then all candidate layer fusion groups are scheduled through the depth-first scheduling module, and the optimal performance of the layer fusion group obtained by scheduling is evaluated to obtain the data structure LayerGroupInfo, which contains whether the layer fusion group can be added. The execution in the accelerator, the shape of each layer of tiles in the fusion group of this layer, the execution order of each layer of tiles in the fusion group of this layer, and the performance that the fusion group of this layer can achieve under the optimal scheduling strategy; define G as the set of all operators in the computation graph, G = {P0, P1, P2...Pn}, where Pi represents each operator in the computation graph, a total of n+1, P0 represents the operator containing the input node, and Pn represents the operator containing the output node; at the same time, define Gi to represent the subgraph from the input node to Pi, Gi = {P0, P1, P2...Pi}, in this definition In the following example, Gn=G; the LayerGroupInfo of all candidate layer fusion groups obtained in the previous step are integrated to create a data structure LayerGroupDict[Pi]={L0,L1,L2…Lm} to record the set of layer fusion groups with operator Pi as output node, where L∈{L0,L1,L2…Lm} represents each LayerGroupInfo with Pi as output node, and there are m+1 of them in total; finally, after obtaining all candidate layer fusion group information LayerGroupDict, the layer fusion group partitioning strategy of the entire neural network is searched through the dynamic programming algorithm; a local optimal solution data structure LocalOptimalDict[Pi]={L0,L1,L2…Lx} is created to store the optimal solution of the layer fusion group partitioning of the subgraph Gi from the input node to Pi, and the optimal solution consists of x layer fusion groups in total; when LocalOptimalDict[Pn] is calculated, the optimal solution of the layer fusion group combination of the entire network is obtained, and the tile shape and execution order within each layer fusion group are also obtained in the previous scheduling.

3. The convolutional neural network compiler based on depth-first scheduling technology according to claim 2, characterized in that In the depth-first scheduling module, there are two relationships between tile shapes of adjacent layers, which are represented by the data reuse type ReuseType: (1) the output data of the tile of the next layer is a combination of the output data of multiple tiles of the previous layer, and this relationship is called Combination Reuse; (2) the output data of the tile of the next layer is a part of the input data of a tile of the previous layer, and this relationship is called Split Reuse; Based on Reuse Type, under different Reuse Types, the ratio of tile shape combination or split between adjacent layers is recorded by ReuseRatio. For Combination Reuse, Reuse Ratio is n, n∈{1,2,...,Reuse Limit}; for Split Reuse, Reuse Ratio is 1 / n, n∈{1,2,...,Reuse Limit}; Reuse Limit is the upper limit of data reuse configured by the configuration module, representing the upper limit of the number of combinations or splits between adjacent layers. When determining the execution order of each tile, the depth-first principle is still followed: when the input data of the tile of the subsequent layer in the depth direction is ready, the calculation of the tile is executed first; if there is no tile in the subsequent layer in the depth direction to execute the calculation, the calculation of other tiles in the current layer will continue. The output data generated by all tile calculations is stored in the on-chip memory until all next-layer tile calculations using this data are completed. This part of data is called reused data and is also counted when calculating the memory usage of each layer.

4. The convolutional neural network compiler based on depth-first scheduling technology according to claim 3, characterized in that In the depth-first scheduling module, the Reuse Type and Reuse Ratio between each adjacent layer pair are first determined by searching to obtain the optimal scheduling strategy for the layer fusion group; when searching for the optimal scheduling strategy, the feature map sizes of each layer in the layer fusion group are first counted, the ratio of the feature map sizes between each pair of adjacent layers is calculated, and the Reuse Type between each adjacent layer pair is determined; if the ratio of the feature map size of the current layer to the feature map size of the subsequent layer is greater than or equal to 1, the data reuse relationship representation of the adjacent layer pair is set to Combination Reuse; conversely, if the ratio of the feature map size of the current layer to the feature map size of the subsequent layer is less than 1, the data reuse relationship representation of the adjacent layer pair is set to Split Reuse; at the same time, a search space for the Reuse Ratio of each adjacent layer pair is constructed. For each adjacent layer pair, there will be Reuse Limit candidate Reuse Ratios, so the search space size of the entire layer fusion group scheduling strategy is Reuse Limit^Reuse Ratio; Then, all candidate scheduling strategies, i.e., Reuse Ratios combinations, in this search space are traversed. For each traversed scheduling strategy, the feature map sizes of each layer in the layer fusion group are relatively normalized based on the output feature map size of the last layer. Based on the Reuse Ratio of each adjacent layer pair under the currently traversed scheduling strategy, the normalized input and output feature map sizes of each layer are calculated, and the normalized memory usage of each layer is further calculated. The memory utilization of each layer is the memory usage of the layer divided by the maximum memory usage of the fusion group of the layer; Afterwards Use the following formula to weight the memory utilization of each layer and calculate the weighted memory utilization; α i =kernel i -strife i ; Among them, weighted_utilization i is the weighted memory utilization of the fusion group of this layer in the current scheduling strategy, utilization i is the memory utilization of each layer under the current scheduling strategy, FM_size i The memory size of each layer, kernel i is the convolution kernel size of each convolution layer, stride i Calculate the convolution step size for each convolution layer; by traversing all scheduling strategies, find the scheduling strategy with the highest weighted memory utilization, that is, determine this strategy as the optimal scheduling strategy for the current layer fusion group, which includes the reuse ratio and reuse type between adjacent layers.

5. The convolutional neural network compiler based on depth-first scheduling technology according to claim 4, characterized in that: In the depth-first scheduling module, after obtaining the optimal scheduling strategy, the specific tile shape of each layer is calculated. Specifically, the search space of the tile shape of the last layer is first constructed. The last layer in the fusion group of this layer adopts any tile shape that does not exceed the boundary of the operator shape to constitute the search space of the tile shape of the last layer; then the search space is traversed, and for the traversed last layer of tile shapes, the tile shapes of each layer are calculated from bottom to top according to the optimal scheduling strategy of the current layer fusion group, and their execution order is scheduled, and the tile shapes and execution order of each layer are transmitted to the cost model module, and the cost model module will return the cost as the cost of the current tile shape; after traversing the search space, the tile shapes of each layer with the lowest cost and their execution order will be used as the final result of the depth-first scheduling module, and will be returned to the layer fusion group division module together with the lowest cost; if the cost returned under all scheduling strategies is positive infinity, the layer fusion group cannot perform depth-first scheduling on the current device, and this status is returned to the layer fusion group division module.

Images