Method and system for evaluating a neural network cost model based on class guidance

Abstract

The application provides a kind of evaluation method and system of neural network cost model based on category guidance, it is related to data processing technical field, the feature of the tensor program to be evaluated is extracted from the perspective of syntax tree, and the tensor program feature is obtained;Tensor program features are processed by dimension reduction and clustering, to obtain the probability that the tensor program feature belongs to different categories;The feature of the tensor program to be evaluated is extracted from the perspective of scheduling primitive, to obtain intermediate features;Based on the probability that the tensor program feature belongs to different categories and the intermediate features, the performance score of the tensor program to be evaluated is obtained.From the perspective of syntax tree, extracting the features of the tensor program and extracting the features from the scheduling primitive can effectively utilize the complex relationship between subprograms, reduce the time of the AI compiler automatic tuning process, so that the neural network model can be deployed on the ground, and only needs less tuning time under the same tuning performance condition, improves the efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an evaluation method and system for category-guided neural network cost models. Background Technology

[0002] With the widespread application of deep learning, numerous neural network models have emerged. Among them, high-performance model structures are often used as backbone networks for general tasks. These neural network models have a significant impact on human life and work, and their runtime is often crucial for their successful deployment. This necessitates that the model fully utilize hardware computing power during runtime. Therefore, AI compiler tuning modules are needed to automatically tune neural networks. Typically, AI compiler automatic tuning modules use cost models to evaluate the performance of the tuned program.

[0003] Currently, the cost model in the AI ​​compiler's automatic tuning module has two main limitations: (a) the cost models are all trained online, which makes the automatic tuning process of tensor programs take a long time; (b) the cost models do not analyze the complex relationships between subroutines in depth and fail to make full use of this important information.

[0004] Therefore, there is an urgent need for a new evaluation method for neural network cost models to evaluate the optimized tensor program. Summary of the Invention

[0005] This application provides an evaluation method and system for a category-guided neural network cost model to address the aforementioned problems in existing evaluation methods for neural network cost models.

[0006] In a first aspect of this application, an evaluation method for a category-guided neural network cost model is provided. The cost model includes: a subgraph feature extraction module, a dimensionality reduction clustering module, a backbone network module, and a multi-head predictor module. The method includes: The tensor program features are obtained by extracting features of the tensor program to be evaluated from the perspective of the syntax tree through the subgraph feature extraction module. The tensor program features are subjected to dimensionality reduction and clustering processing by the dimensionality reduction and clustering module to obtain the probability that the tensor program features belong to different categories. The features of the tensor program to be evaluated are extracted from the perspective of scheduling primitives through the backbone network module to obtain intermediate features; The performance score of the tensor program to be evaluated is obtained by using the multi-head predictor module based on the probability that the tensor program features belong to different categories and the intermediate features.

[0007] In one optional embodiment of this application, the subgraph feature extraction module includes at least a tensor program feature extraction unit, which includes a first extraction unit, a second extraction unit, a third extraction unit, a fourth extraction unit, and a splicing unit. The subgraph feature extraction module extracts features of the tensor program to be evaluated from the perspective of the syntax tree, resulting in tensor program features, including: The first extraction unit extracts the features of the first subroutine from the dimension of calculating relevant features; The second extraction unit extracts the features of the second subroutine from the dimension of memory access-related features. The third extraction unit extracts the features of the third subroutine from the dimension of memory allocation features. The fourth extraction unit extracts the features of the fourth subroutine from the external scope feature dimension. The first subroutine feature, the second subroutine feature, the third subroutine feature, and the fourth subroutine feature are concatenated into the tensor program feature by the splicing unit.

[0008] In one optional embodiment of this application, the multi-head predictor module includes a plurality of single-head predictors, each single-head predictor including a decoder and a summing unit; Using a multi-head predictor module, based on the probabilities of the tensor program features belonging to different categories and the intermediate features, the performance score of the tensor program to be evaluated is obtained, including: After decoding the intermediate features using the decoder in each single-head predictor, the value of the i-th dimension is obtained; The summation unit in each single-head predictor sums the values ​​of the i-th dimension to obtain the cumulative value of the feature dimension; Based on the probabilities of different categories, the feature dimension accumulation values ​​output by each single-head predictor are weighted and summed to obtain the performance score of the tensor program to be evaluated.

[0009] In an optional embodiment of this application, the cumulative feature dimension output of each single-head predictor is defined as follows: , Where x is the tensor procedure to be evaluated. For the decoder in a single-head predictor. Represents the numerical value of the feature dimension; For the first in the decoder output The final output of the single-head predictor is the feature dimension. The accumulated value; Each decoder consists of 4 linear layers interspersed. The activation functions are composed of the following, and each decoder layer is defined as follows: , Where x is the tensor procedure to be evaluated. For the weights of the linear layer, For linear layer bias; The multi-head predictor module is defined as follows: , in, The number of heads in the predictor. Let be the probability that a tensor program feature belongs to a different category.

[0010] In one optional embodiment of this application, the subgraph feature extraction module includes a tensor procedure feature extraction unit and a subgraph feature aggregation unit, and the dimensionality reduction clustering module includes a dimensionality reduction module and a clustering module; The training process of the cost model is as follows: Obtain the tensor program sample set, the subgraph category corresponding to each sample tensor program in the tensor program sample set, and the true performance score of each sample tensor program; Based on the tensor program sample set, the tensor program feature extraction unit in the subgraph feature extraction module is used to extract the sample tensor program features, and the subgraph feature agglomeration unit in the subgraph feature extraction module is used to extract the subgraph features. The subgraph features are used to train the clustering modules in the dimensionality reduction clustering module to obtain the trained clustering modules; After the dimensionality reduction module in the dimensionality reduction clustering module is used to reduce the dimensionality of the sample tensor program features, the trained clustering module is used to obtain the probability that the dimensionality-reduced sample tensor program features belong to different categories. Input the sample tensor program from the sample tensor program dataset into the backbone network model to be trained to obtain the intermediate features of the samples; The probability of the intermediate features of the sample and the tensor procedural features of the sample after dimensionality reduction belonging to different categories is input into the multi-head predictor module to be trained to obtain the prediction performance score. Based on the predicted performance score and the actual performance score, the backbone network model to be trained and the multi-head predictor module to be trained are trained until a preset number of training rounds are completed to obtain the trained cost model.

[0011] In an optional embodiment of this application, the extraction of subgraph features using the subgraph feature agglomeration unit in the subgraph feature extraction module includes: Based on the subgraph category corresponding to each sample tensor program in the tensor program sample set, the features of multiple sample tensor programs are classified using subgraph feature aggregation units; The subgraph features are obtained by averaging the tensor procedural features of all samples of the same subgraph category.

[0012] In an optional embodiment of this application, the subgraph features are defined as follows: , in, The procedure for representing sample tensors, Representative sample tensor program Subgraph category to which it belongs Subgraph Category The number of sample tensors after generalization. This is the sample tensor program feature extraction unit.

[0013] A second aspect of this application proposes an evaluation system for a category-guided neural network cost model, the system comprising: The subgraph feature extraction module is used to extract features of the tensor program to be evaluated from the perspective of the syntax tree, and obtain the tensor program features. The dimensionality reduction and clustering module is used to perform dimensionality reduction and clustering on the tensor program features to obtain the probability that the tensor program features belong to different categories. The backbone network module is used to extract features of the tensor program to be evaluated from the perspective of scheduling primitives to obtain intermediate features; A multi-head predictor module is used to obtain the performance score of the tensor program to be evaluated based on the probability that the tensor program features belong to different categories and the intermediate features.

[0014] In one optional embodiment of this application, the subgraph feature extraction module includes at least a tensor procedure feature extraction unit, and the tensor procedure feature extraction unit further includes: The first extraction unit is used to extract the features of the first subroutine from the dimension of computation-related features; The second extraction unit is used to extract the features of the second subroutine from the memory access-related feature dimension; The third extraction unit is used to extract the features of the third subroutine from the memory allocation feature dimension; The fourth extraction unit is used to extract the features of the fourth subroutine from the external scope feature dimension; The splicing unit is used to splice the first subroutine feature, the second subroutine feature, the third subroutine feature and the fourth subroutine feature into the tensor program feature.

[0015] In one optional embodiment of this application, the multi-head predictor module includes multiple single-head predictors, each single-head predictor including a decoder and a summing unit; the multi-head prediction module further includes: A decoder is used to decode the intermediate features to obtain the value of the i-th dimension; The summation unit is used to sum the values ​​of the i-th dimension to obtain the cumulative value of the feature dimension; The weighted summation unit is used to perform a weighted summation of the feature dimension accumulation values ​​output by each single-head predictor according to the probability of different categories, so as to obtain the performance score of the tensor program to be evaluated.

[0016] In one optional embodiment of this application, the subgraph feature extraction module includes a tensor procedure feature extraction unit and a subgraph feature aggregation unit, and the dimensionality reduction clustering module includes a dimensionality reduction module and a clustering module; The system also includes a training module, which includes: The acquisition submodule is used to acquire the tensor program sample set, the subgraph category corresponding to each sample tensor program in the tensor program sample set, and the actual performance score of each sample tensor program. The feature extraction submodule is used to extract sample tensor program features based on the tensor program sample set using the tensor program feature extraction unit in the subgraph feature extraction module, and to extract subgraph features using the subgraph feature agglomeration unit in the subgraph feature extraction module. The first training submodule is used to train the clustering module in the dimensionality reduction clustering module using the subgraph features to obtain the trained clustering module; The probability acquisition submodule is used to reduce the dimensionality of the sample tensor program features using the dimensionality reduction module in the dimensionality reduction clustering module, and then uses the trained clustering module to obtain the probability that the dimensionality-reduced sample tensor program features belong to different categories. The intermediate feature acquisition submodule is used to input the sample tensor program from the sample tensor program dataset into the backbone network model to be trained to obtain the sample intermediate features; The prediction submodule is used to input the probability of the intermediate features of the sample and the tensor procedural features of the sample after dimensionality reduction into the multi-head predictor module to be trained, so as to obtain the prediction performance score. The second training submodule is used to train the backbone network model to be trained and the multi-head predictor module to be trained based on the predicted performance score and the actual performance score, until a preset number of training rounds are completed to obtain the trained cost model.

[0017] In an optional embodiment of this application, the feature extraction submodule further includes: The classification unit is used to classify the features of multiple sample tensor programs according to the subgraph category corresponding to each sample tensor program in the tensor program sample set, using the subgraph feature aggregation unit. The subgraph feature acquisition unit is used to take the average of the tensor program features of all samples of the same subgraph category to obtain the subgraph features.

[0018] In a third aspect of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the evaluation method of the category-guided neural network cost model as described in any of the first aspects above.

[0019] In a fourth aspect of this application, a computer-readable storage medium is provided, on which a computer program / instruction is stored, which, when executed by a processor, implements the evaluation method of the category-guided neural network cost model described in any one of the first aspects above.

[0020] This application offers the following advantages: Embodiments of this application provide an evaluation method and system for a category-guided neural network cost model. A subgraph feature extraction module extracts features of the tensor program to be evaluated from a syntax tree perspective, obtaining tensor program features. A dimensionality reduction and clustering module performs dimensionality reduction and clustering on the tensor program features to obtain the probability that the tensor program features belong to different categories. A backbone network module extracts features of the tensor program to be evaluated from a scheduling primitive perspective, obtaining intermediate features. A multi-head predictor module obtains the performance score of the tensor program to be evaluated based on the probability that the tensor program features belong to different categories and the intermediate features. The dimensionality reduction and clustering module effectively reduces the dimensionality of program features, obtaining category information for the tensor program to be evaluated, which can guide the multi-head predictor module in training and performance evaluation. Extracting tensor program features from a syntax tree perspective and feature extraction from a scheduling primitive perspective effectively utilizes the complex relationships between subroutines, reducing the time required for the AI ​​compiler's automatic tuning process. This allows the neural network model to be deployed with less tuning time while achieving the same post-tuning performance, thus improving efficiency. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the steps of an evaluation method for a category-guided neural network cost model proposed in an embodiment of this application. Figure 2 This is a schematic diagram of an evaluation method for a category-guided neural network cost model provided in an embodiment of this application; Figure 3 This is a schematic diagram of the architecture of the tensor procedure feature extraction unit in an evaluation method for a category-guided neural network cost model provided in an embodiment of this application; Figure 4 This is a schematic diagram of the architecture of the multi-head predictor module in an evaluation method for a category-guided neural network cost model provided in an embodiment of this application; Figure 5 This is a schematic diagram of the subgraph feature aggregation unit architecture of an evaluation method for a category-guided neural network cost model provided in an embodiment of this application; Figure 6 This is an architecture diagram of an evaluation system for a category-guided neural network cost model provided in an embodiment of this application; Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] With the widespread application of deep learning, numerous neural network models have emerged. Among them, high-performance model structures are often used as backbone networks for general tasks. These neural network models have a significant impact on human life and work, and their runtime is often crucial for their successful deployment. This necessitates that the model fully utilize hardware computing power during runtime. Therefore, AI compiler tuning modules are needed to automatically tune neural networks. Typically, AI compiler automatic tuning modules use cost models to evaluate the performance of the tuned program.

[0025] Currently, the cost model in the AI ​​compiler's automatic tuning module has two main limitations: (a) the cost models are all trained online, which makes the automatic tuning process of tensor programs take a long time; (b) the cost models do not analyze the complex relationships between subroutines in depth and fail to make full use of this important information.

[0026] Based on this, this application proposes a new evaluation method for neural network cost models to evaluate the optimized tensor program.

[0027] See Figure 1 , Figure 1 This is a flowchart illustrating the steps of an evaluation method for a category-guided neural network cost model proposed in this application. The method includes the following steps: Step 101: Extract the features of the tensor program to be evaluated from the perspective of the syntax tree using the subgraph feature extraction module to obtain the tensor program features; Step 102: Using the dimensionality reduction and clustering module, the tensor program features are subjected to dimensionality reduction and clustering to obtain the probability that the tensor program features belong to different categories; Step 103: Extract the features of the tensor program to be evaluated from the perspective of scheduling primitives through the backbone network module to obtain intermediate features; Step 104: Using the multi-head predictor module, obtain the performance score of the tensor program to be evaluated based on the probability that the tensor program features belong to different categories and the intermediate features.

[0028] To clearly illustrate the evaluation method for the category-guided neural network cost model proposed in this application, please refer to [reference needed]. Figure 2 , Figure 2 This is a schematic diagram of an evaluation method for a category-guided neural network cost model provided in an embodiment of this application. The following will be combined with... Figure 2 The above assessment methods are explained.

[0029] In specific implementation step 101, in this embodiment, a tensor program refers to a program that represents a deep learning program as composed of tensors and operators, and this tensor program is often represented in the form of a computation graph. The features of the tensor program to be evaluated are extracted from the syntax tree perspective using the subgraph feature extraction module, thus obtaining the tensor program features. A syntax tree (or parse tree) is a very important data structure that represents the syntactic structure of the source code. In this embodiment, the extraction of tensor program features from the syntax tree perspective is implemented from four dimensions: computation-related feature dimension, memory access-related feature dimension, memory allocation feature dimension, and external scope feature dimension.

[0030] In an optional embodiment of this application, the subgraph feature extraction module includes at least a tensor program feature extraction unit. This unit extracts features from the four dimensions of the syntax tree and then combines the features from each dimension to obtain the tensor program features. The tensor program feature extraction unit includes a first extraction unit, a second extraction unit, a third extraction unit, a fourth extraction unit, and a concatenation unit. See also... Figure 3 , Figure 3This is a schematic diagram of the tensor program feature extraction unit in an evaluation method for a category-guided neural network cost model provided in this application embodiment. Specifically, the subgraph feature extraction module extracts features of the tensor program to be evaluated from the perspective of the syntax tree, obtaining tensor program features, including: extracting first subroutine features from the computation-related feature dimension through a first extraction unit; extracting second subroutine features from the memory access-related feature dimension through a second extraction unit; extracting third subroutine features from the memory allocation feature dimension through a third extraction unit; extracting fourth subroutine features from the external scope feature dimension through a fourth extraction unit; and concatenating the first, second, third, and fourth subroutine features into the aforementioned tensor program features through a concatenation unit. By extracting subroutine features from each dimension from the perspective of the syntax tree, the complex relationships between subroutine features can be analyzed in depth, and the tensor program features obtained can be used to effectively evaluate the tensor program to be evaluated.

[0031] In specific implementation step 102, the dimensionality reduction and clustering module includes a dimensionality reduction module and a clustering module. The dimensionality reduction module performs dimensionality reduction processing on the tensor program features to obtain dimensionality-reduced tensor program features. Then, the clustering module performs clustering processing on the dimensionality-reduced tensor program features to obtain the probability that the tensor program features belong to different categories. The different categories mentioned above refer to the categories obtained by clustering through the clustering module. Assuming that the clustering module clusters into seven categories, the probability that the tensor program features belong to different categories is the probability that the tensor program features belong to each of the above seven categories. The dimensionality reduction method described above can be any one of Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), Multidimensional Scaling (MDS), or Locally Linear Embedding (LLE). Similarly, the clustering method can be any one of Gaussian Mixture Model (GMM) or Fuzzy C-means (FCM) clustering algorithm. In this embodiment, PCA is preferred as the dimensionality reduction method, and GMM is preferred as the clustering method. The dimensionality reduction and clustering module is trained; the specific training process is detailed in the following description of the cost model's training process.

[0032] In specific implementation step 103, based on the scheduling primitive perspective, the features of the tensor program to be evaluated are extracted through the backbone network module to obtain intermediate features. The backbone network module is preferably a neural network model, including four linear layers, an attention layer, and three sequentially connected linear layers. After the tensor program to be evaluated is input into the backbone network model, features are extracted, and then sequentially passed through the four linear layers, the attention layer, and the three sequentially connected linear layers to obtain intermediate features. A subgraph contains multiple different tensor programs, and a subgraph and a primitive perspective can determine a tensor program. The scheduling primitive perspective is used for feature extraction. The parameters of the primitive are vectorized, iteratively expanded, etc., and then feature extraction is performed on the processed parameters to obtain the aforementioned intermediate features.

[0033] In specific implementation step 104, the multi-head predictor module processes the probabilities of the tensor program features belonging to different categories and the intermediate features, and performs a weighted summation of the intermediate features using the probabilities of different categories as weights to obtain the performance score of the tensor program to be evaluated. The performance score of the tensor program to be evaluated is used as the basis for assessing its quality. The multi-head predictor module is trained; the specific training process is detailed in the following description of the cost model training process.

[0034] In one optional embodiment of this application, the multi-head predictor module includes multiple single-head predictors, each single-head predictor including a decoder and a summing unit. See also... Figure 4 , Figure 4 This is a schematic diagram of the architecture of a multi-head predictor module in an evaluation method for a category-guided neural network cost model provided in this application embodiment. The multi-head predictor module obtains the performance score of the tensor program to be evaluated based on the probability of the tensor program features belonging to different categories and the intermediate features. Specifically, this includes: decoding the intermediate features using the decoder in each single-head predictor to obtain the i-th dimension value; summing the i-th dimension value using the summation unit in each single-head predictor to obtain a feature dimension accumulation value; and then weighting and summing the feature dimension accumulation values ​​output by each single-head predictor according to the probability of different categories to obtain the performance score of the tensor program to be evaluated.

[0035] In an optional embodiment of this application, the cumulative feature dimension output of each single-head predictor is defined as follows: , Where x is the tensor procedure to be evaluated. For the decoder in a single-head predictor. Represents the numerical value of the feature dimension; For the first in the decoder output The final output of the single-head predictor is the feature dimension. The accumulated value.

[0036] Each decoder consists of 4 linear layers interspersed. The activation functions are composed of the following, and each decoder layer is defined as follows: , Where x is the tensor procedure to be evaluated. For the weights of the linear layer, This is for linear layer bias.

[0037] The multi-head predictor module is defined as follows: , in, The number of heads in the predictor. Let be the probability that a tensor program feature belongs to a different category.

[0038] In one optional embodiment of this application, the cost model includes a subgraph feature extraction module, a dimensionality reduction clustering module, a backbone network module, and a multi-head predictor module; the subgraph feature extraction module includes a tensor procedure feature extraction unit and a subgraph feature aggregation unit, and the dimensionality reduction clustering module includes a dimensionality reduction module and a clustering module. The training process of the cost model is as follows: First, obtain the tensor procedure sample set, the subgraph category corresponding to each sample tensor procedure in the tensor procedure sample set, and the true performance score of each sample tensor procedure; based on the tensor procedure sample set, extract sample tensor procedure features using the tensor procedure feature extraction unit in the subgraph feature extraction module, and extract subgraph features using the subgraph feature aggregation unit in the subgraph feature extraction module; train the clustering module in the dimensionality reduction clustering module using the above subgraph features to obtain a trained clustering module; train the sample tensor procedure using the dimensionality reduction module in the dimensionality reduction clustering module. After feature dimensionality reduction, a pre-trained clustering module is used to obtain the probability that the dimensionality-reduced sample tensor features belong to different categories. The sample tensors from the aforementioned sample tensor dataset are input into the backbone network model to be trained to obtain intermediate sample features. The probabilities of the intermediate sample features and the dimensionality-reduced sample tensor features belonging to different categories are input into the multi-head predictor module to be trained to obtain a prediction performance score. Based on the prediction performance score and the actual performance score, the backbone network model and the multi-head predictor module to be trained are trained until a preset number of training rounds are obtained, resulting in a trained cost model. In this embodiment, the preset number of rounds is 100 rounds.

[0039] In one optional embodiment of this application, see [reference] Figure 5 , Figure 5This is a schematic diagram of the subgraph feature agglomeration unit architecture of an evaluation method for a category-guided neural network cost model provided in this application embodiment. The subgraph feature agglomeration unit in the subgraph feature extraction module extracts subgraph features, specifically including: classifying the features of multiple sample tensor programs according to the subgraph category corresponding to each sample tensor program in the tensor program sample set, and obtaining sample tensor program features under different subgraph categories; then, taking the average of all sample tensor program features within the same subgraph category to obtain the aforementioned subgraph features.

[0040] In an optional embodiment of this application, the above-mentioned subgraph features are defined as follows: , in, The procedure for representing sample tensors, Representative sample tensor program Subgraph category to which it belongs Subgraph Category The number of sample tensors after generalization. This is the sample tensor program feature extraction unit.

[0041] This application provides an evaluation method for a category-guided neural network cost model. The method involves: a subgraph feature extraction module extracting features from the syntax tree perspective of the tensor program to be evaluated, obtaining tensor program features; a dimensionality reduction and clustering module performing dimensionality reduction and clustering on the tensor program features to obtain the probability that each feature belongs to a different category; a backbone network module extracting features from the scheduling primitives perspective of the tensor program to be evaluated, obtaining intermediate features; and a multi-head predictor module obtaining the performance score of the tensor program to be evaluated based on the probability of each feature belonging to a different category and the intermediate features. The dimensionality reduction and clustering module effectively reduces the dimensionality of program features, obtaining category information for the tensor program to be evaluated, which can guide the training and performance evaluation of the multi-head predictor module. Extracting tensor program features from the syntax tree perspective and feature extraction from the scheduling primitives perspective effectively utilizes the complex relationships between subroutines, reducing the time required for the AI ​​compiler's automatic tuning process. This results in less tuning time required when deploying the neural network model to achieve the same performance after tuning, thus improving efficiency.

[0042] In a second aspect of this application, an evaluation system for a category-guided neural network cost model is proposed, see [link to relevant documentation]. Figure 6 , Figure 6 This is an architecture diagram of an evaluation system for a category-guided neural network cost model provided in an embodiment of this application. The system includes: Subgraph feature extraction module 601 is used to extract features of the tensor program to be evaluated from the perspective of syntax tree to obtain tensor program features; The dimensionality reduction and clustering module 602 is used to perform dimensionality reduction and clustering on the tensor program features to obtain the probability that the tensor program features belong to different categories; The backbone network module 603 is used to extract features of the tensor program to be evaluated from the perspective of scheduling primitives to obtain intermediate features; The multi-head predictor module 604 is used to obtain the performance score of the tensor program to be evaluated based on the probability that the tensor program features belong to different categories and the intermediate features.

[0043] The subgraph feature extraction module includes at least a tensor program feature extraction unit, and the tensor program feature extraction unit further includes: The first extraction unit is used to extract the features of the first subroutine from the dimension of computation-related features; The second extraction unit is used to extract the features of the second subroutine from the memory access-related feature dimension; The third extraction unit is used to extract the features of the third subroutine from the memory allocation feature dimension; The fourth extraction unit is used to extract the features of the fourth subroutine from the external scope feature dimension; The splicing unit is used to splice the first subroutine feature, the second subroutine feature, the third subroutine feature and the fourth subroutine feature into the tensor program feature.

[0044] The multi-head predictor module includes multiple single-head predictors, each single-head predictor including a decoder and a summing unit; the multi-head prediction module also includes: A decoder is used to decode the intermediate features to obtain the value of the i-th dimension; The summation unit is used to sum the values ​​of the i-th dimension to obtain the cumulative value of the feature dimension; The weighted summation unit is used to perform a weighted summation of the feature dimension accumulation values ​​output by each single-head predictor according to the probability of different categories, so as to obtain the performance score of the tensor program to be evaluated.

[0045] The feature dimension accumulation value output by each single-head predictor in the summation unit is defined as follows: , Where x is the tensor procedure to be evaluated. For the decoder in a single-head predictor. Represents the numerical value of the feature dimension; For the first in the decoder output The final output of the single-head predictor is the feature dimension. The accumulated value; The decoder consists of four linear layers interspersed throughout. The activation functions are composed of the following, and each decoder layer is defined as follows: , Where x is the tensor procedure to be evaluated. For the weights of the linear layer, For linear layer bias; The weighted summation unit is defined as follows: , in, The number of heads in the predictor. Let be the probability that a tensor program feature belongs to a different category.

[0046] The subgraph feature extraction module includes a tensor program feature extraction unit and a subgraph feature aggregation unit, and the dimensionality reduction clustering module includes a dimensionality reduction module and a clustering module. The system also includes a training module, which includes: The acquisition submodule is used to acquire the tensor program sample set, the subgraph category corresponding to each sample tensor program in the tensor program sample set, and the actual performance score of each sample tensor program. The feature extraction submodule is used to extract sample tensor program features based on the tensor program sample set using the tensor program feature extraction unit in the subgraph feature extraction module, and to extract subgraph features using the subgraph feature agglomeration unit in the subgraph feature extraction module. The first training submodule is used to train the clustering module in the dimensionality reduction clustering module using the subgraph features to obtain the trained clustering module; The probability acquisition submodule is used to reduce the dimensionality of the sample tensor program features using the dimensionality reduction module in the dimensionality reduction clustering module, and then uses the trained clustering module to obtain the probability that the dimensionality-reduced sample tensor program features belong to different categories. The intermediate feature acquisition submodule is used to input the sample tensor program from the sample tensor program dataset into the backbone network model to be trained to obtain the sample intermediate features; The prediction submodule is used to input the probability of the intermediate features of the sample and the tensor procedural features of the sample after dimensionality reduction into the multi-head predictor module to be trained, so as to obtain the prediction performance score. The second training submodule is used to train the backbone network model to be trained and the multi-head predictor module to be trained based on the predicted performance score and the actual performance score, until a preset number of training rounds are completed to obtain the trained cost model.

[0047] The feature extraction submodule further includes: The classification unit is used to classify the features of multiple sample tensor programs according to the subgraph category corresponding to each sample tensor program in the tensor program sample set, using the subgraph feature aggregation unit. The subgraph feature acquisition unit is used to take the average of the tensor program features of all samples of the same subgraph category to obtain the subgraph features.

[0048] Based on the same concept, this application discloses an electronic device in a third aspect. Figure 7 A schematic diagram of an electronic device disclosed in an embodiment of this application is shown, such as... Figure 7 As shown, the electronic device 100 includes a memory 110 and a processor 120. The memory of the electronic device is not less than 12G, and the main frequency of the processor is not less than 2.4GHz. The memory 110 and the processor 120 are connected via a bus communication. The memory 110 stores a computer program that can run on the processor 120 to implement an evaluation method for a category-guided neural network cost model disclosed in the embodiments of this application.

[0049] Based on the same concept, this application discloses a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements an evaluation method for a category-guided neural network cost model disclosed in this application.

[0050] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0051] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices, and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0052] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0053] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0054] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0055] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0056] The above provides a detailed description of the evaluation method and system for a category-guided neural network cost model provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An evaluation method for a category-guided neural network cost model, characterized in that, The cost model includes: a subgraph feature extraction module, a dimensionality reduction clustering module, a backbone network module, and a multi-head predictor module; the method includes: The tensor program features are obtained by extracting features of the tensor program to be evaluated from the perspective of the syntax tree through the subgraph feature extraction module. The tensor program features are subjected to dimensionality reduction and clustering processing by the dimensionality reduction and clustering module to obtain the probability that the tensor program features belong to different categories. The features of the tensor program to be evaluated are extracted from the perspective of scheduling primitives through the backbone network module to obtain intermediate features; The performance score of the tensor program to be evaluated is obtained by using the multi-head predictor module based on the probability that the tensor program features belong to different categories and the intermediate features. The subgraph feature extraction module includes at least a tensor program feature extraction unit, which includes a first extraction unit, a second extraction unit, a third extraction unit, a fourth extraction unit, and a splicing unit. The subgraph feature extraction module extracts features of the tensor program to be evaluated from the perspective of the syntax tree, resulting in tensor program features, including: The first extraction unit extracts the features of the first subroutine from the dimension of calculating relevant features; The second extraction unit extracts the features of the second subroutine from the dimension of memory access-related features. The third extraction unit extracts the features of the third subroutine from the dimension of memory allocation features. The fourth extraction unit extracts the features of the fourth subroutine from the external scope feature dimension. The first subroutine feature, the second subroutine feature, the third subroutine feature, and the fourth subroutine feature are concatenated into the tensor program feature by the splicing unit; The multi-head predictor module includes multiple single-head predictors, each of which includes a decoder and a summing unit. Using a multi-head predictor module, based on the probabilities of the tensor program features belonging to different categories and the intermediate features, the performance score of the tensor program to be evaluated is obtained, including: After decoding the intermediate features using the decoder in each single-head predictor, the value of the i-th dimension is obtained; The summation unit in each single-head predictor sums the values ​​of the i-th dimension to obtain the cumulative value of the feature dimension; Based on the probabilities of different categories, the feature dimension accumulation values ​​output by each single-head predictor are weighted and summed to obtain the performance score of the tensor program to be evaluated.

2. The evaluation method for the category-guided neural network cost model according to claim 1, characterized in that, The cumulative feature dimension output by each single-head predictor is defined as follows: ， Where x is the tensor procedure to be evaluated. For the decoder in a single-head predictor. Represents the numerical value of the feature dimension; For the first in the decoder output The final output of the single-head predictor is the feature dimension. The accumulated value; Each decoder consists of 4 linear layers interspersed. The activation functions are composed of the following, and each decoder layer is defined as follows: ， Where x is the tensor procedure to be evaluated. For the weights of the linear layer, For linear layer bias; The multi-head predictor module is defined as follows: ， in, The number of heads in the predictor. Let be the probability that a tensor program feature belongs to a different category.

3. The evaluation method for the category-guided neural network cost model according to claim 1, characterized in that, The subgraph feature extraction module includes a tensor program feature extraction unit and a subgraph feature aggregation unit, and the dimensionality reduction clustering module includes a dimensionality reduction module and a clustering module; The training process of the cost model is as follows: Obtain the tensor program sample set, the subgraph category corresponding to each sample tensor program in the tensor program sample set, and the true performance score of each sample tensor program; Based on the tensor program sample set, the tensor program feature extraction unit in the subgraph feature extraction module is used to extract the sample tensor program features, and the subgraph feature agglomeration unit in the subgraph feature extraction module is used to extract the subgraph features. The subgraph features are used to train the clustering modules in the dimensionality reduction clustering module to obtain the trained clustering modules; After the dimensionality reduction module in the dimensionality reduction clustering module is used to reduce the dimensionality of the sample tensor program features, the trained clustering module is used to obtain the probability that the dimensionality-reduced sample tensor program features belong to different categories. Input the sample tensor program from the sample tensor program dataset into the backbone network model to be trained to obtain the intermediate features of the samples; The probability of the intermediate features of the sample and the tensor procedural features of the sample after dimensionality reduction belonging to different categories is input into the multi-head predictor module to be trained to obtain the prediction performance score. Based on the predicted performance score and the actual performance score, the backbone network model to be trained and the multi-head predictor module to be trained are trained until a preset number of training rounds are completed to obtain the trained cost model.

4. The evaluation method for the category-guided neural network cost model according to claim 3, characterized in that, The extraction of subgraph features using the subgraph feature agglomeration unit in the subgraph feature extraction module includes: Based on the subgraph category corresponding to each sample tensor program in the tensor program sample set, the features of multiple sample tensor programs are classified using subgraph feature aggregation units; The subgraph features are obtained by averaging the tensor procedural features of all samples of the same subgraph category.

5. The evaluation method for the category-guided neural network cost model according to claim 4, characterized in that, The subgraph features are defined as follows: ， in, The procedure for representing sample tensors, Representative sample tensor program Subgraph category to which it belongs Subgraph Category The number of sample tensors after generalization. This is the sample tensor program feature extraction unit.

6. An evaluation system for a category-guided neural network cost model, characterized in that, The system includes: The subgraph feature extraction module is used to extract features of the tensor program to be evaluated from the perspective of the syntax tree, and obtain the tensor program features. The dimensionality reduction and clustering module is used to perform dimensionality reduction and clustering on the tensor program features to obtain the probability that the tensor program features belong to different categories. The backbone network module is used to extract features of the tensor program to be evaluated from the perspective of scheduling primitives to obtain intermediate features; A multi-head predictor module is used to obtain the performance score of the tensor program to be evaluated based on the probability that the tensor program features belong to different categories and the intermediate features; The subgraph feature extraction module includes at least a tensor program feature extraction unit, which includes a first extraction unit, a second extraction unit, a third extraction unit, a fourth extraction unit, and a splicing unit. The subgraph feature extraction module extracts features of the tensor program to be evaluated from the perspective of the syntax tree, resulting in tensor program features, including: The first extraction unit extracts the features of the first subroutine from the dimension of calculating relevant features; The second extraction unit extracts the features of the second subroutine from the dimension of memory access-related features. The third extraction unit extracts the features of the third subroutine from the dimension of memory allocation features. The fourth extraction unit extracts the features of the fourth subroutine from the external scope feature dimension. The first subroutine feature, the second subroutine feature, the third subroutine feature, and the fourth subroutine feature are concatenated into the tensor program feature by the splicing unit; The multi-head predictor module includes multiple single-head predictors, each of which includes a decoder and a summing unit. Using a multi-head predictor module, based on the probabilities of the tensor program features belonging to different categories and the intermediate features, the performance score of the tensor program to be evaluated is obtained, including: After decoding the intermediate features using the decoder in each single-head predictor, the value of the i-th dimension is obtained; The summation unit in each single-head predictor sums the values ​​of the i-th dimension to obtain the cumulative value of the feature dimension; Based on the probabilities of different categories, the feature dimension accumulation values ​​output by each single-head predictor are weighted and summed to obtain the performance score of the tensor program to be evaluated.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the evaluation method for the category-guided neural network cost model according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program / instruction that, when executed by a processor, implements the evaluation method of the category-guided neural network cost model according to any one of claims 1-5.

Images