Model compiling method and device, computer device and computer readable storage medium

Abstract

The application relates to a model compilation method and device, computer equipment and a computer readable storage medium. The method comprises the following steps: obtaining a plurality of tensor programs; the plurality of tensor programs are a plurality of intermediate programs generated by a deep learning compiler according to different compilation optimization strategies for a to-be-compiled model; for each tensor program, a loop optimization statement in the tensor program is subjected to feature extraction to obtain a loop optimization feature; the loop optimization feature corresponding to the loop optimization statement and operator category information are subjected to encoding processing by an attention encoder to obtain feature encoding data; the cost of the tensor program is predicted according to the feature encoding data to obtain a cost prediction result; a target tensor program is determined from the plurality of tensor programs based on the cost prediction result corresponding to each tensor program; and the target tensor program is used to indicate that a machine language is generated by the deep learning compiler for the to-be-compiled model. The method can improve the efficiency of model compilation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a model compiling method and device, computer equipment and a computer readable storage medium. BACKGROUND

[0002] With the continuous development of deep learning, an increasing number of deep learning models have become the mainstream configuration of automated machine learning systems. It will be a very tedious and error-prone thing to manually adapt various hardware to the operators in the deep learning model. To address the above problems, model compiling technology has become a widely concerned technical direction.

[0003] In the traditional compiling method, a cost prediction function based on a statistical method is often used to explore the optimal compiling mode in a pre-defined solution space through a search algorithm. However, the cost prediction function does not have good cost prediction performance, resulting in the need to explore repeatedly, which can lead to low efficiency of the entire compiling process. SUMMARY

[0004] Therefore, it is necessary to provide a model compiling method, device, computer equipment, computer readable storage medium and computer program product to improve the efficiency of model compiling.

[0005] In a first aspect, the present application provides a model compiling method, comprising:

[0006] obtaining a plurality of tensor programs; the plurality of tensor programs are a plurality of intermediate programs generated by a deep learning compiler according to different compiling optimization strategies for a to-be-compiled model;

[0007] For each tensor program, a loop optimization statement in the tensor program is feature extracted to obtain loop optimization features; the loop optimization statement is a program statement with loop optimization characteristics;

[0008] The loop optimization features corresponding to the loop optimization statement and the operator category information are encoded by an attention encoder to obtain feature encoding data;

[0009] The cost of the tensor program is predicted according to the feature encoding data to obtain a cost prediction result;

[0010] Based on the cost prediction result corresponding to each tensor program, a target tensor program is determined from the plurality of tensor programs; the target tensor program is used to indicate that a machine language is generated by the deep learning compiler for the to-be-compiled model.

[0011] In a second aspect, the present application further provides a model compiling device, comprising:

[0012] The acquisition module is configured to acquire a plurality of tensor programs, wherein the plurality of tensor programs are a plurality of intermediate programs generated by a deep learning compiler according to different compilation optimization strategies for a to-be-compiled model respectively;

[0013] The extraction module is configured to perform feature extraction on the loop optimization statements in the tensor program to obtain loop optimization features for each tensor program, wherein the loop optimization statements are program statements with loop optimization features;

[0014] The cost prediction module is configured to perform encoding processing on the loop optimization features and the operator category information corresponding to the loop optimization statements by using an attention encoder to obtain feature encoding data, and to perform cost prediction on the tensor program according to the feature encoding data to obtain a cost prediction result.

[0015] The compilation module is configured to determine a target tensor program from the plurality of tensor programs based on the cost prediction result corresponding to each tensor program, wherein the target tensor program is used to indicate a machine language generated by the deep learning compiler for the to-be-compiled model.

[0016] In a third aspect, the present application further provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps in the model compilation method when executing the computer program.

[0017] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps in the model compilation method.

[0018] In a fifth aspect, the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in the model compilation method.

[0019] The aforementioned model compilation method, apparatus, computer equipment, storage medium, and computer program product generate multiple tensor programs for the model to be compiled using a deep learning compiler according to different compilation optimization strategies. Features are extracted from loop optimization statements in each tensor program to obtain loop optimization features. These features reflect the corresponding compilation optimization strategy for the tensor program, and the cost prediction results based on these features reflect the quality of the optimization strategy. Furthermore, an attention encoder selectively focuses on more important information from the loop optimization features and operator category information corresponding to the loop optimization statements, encoding this information to obtain feature-encoded data. Compared to simply making statistical predictions based on data distribution, cost prediction based on feature-encoded data is more accurate. Accurate cost prediction results precisely indicate a better tensor program. Subsequently, the target tensor program is determined from multiple tensor programs based on the cost prediction results, eliminating the need for repeated searches and improving the efficiency of model compilation. Attached Figure Description

[0020] Figure 1 A schematic flowchart illustrating a model compilation method provided in an embodiment of this application;

[0021] Figure 2 A schematic diagram illustrating the merging of loop optimization features and operator category information provided in this application embodiment;

[0022] Figure 3 This application provides a schematic diagram of an encoding process performed using an attention encoder, as illustrated in an embodiment of the present application.

[0023] Figure 4 This is a schematic diagram of the structure of a cost prediction model provided in an embodiment of this application;

[0024] Figure 5 A schematic diagram of a migration cost prediction model provided in an embodiment of this application.

[0025] Figure 6 A structural block diagram of a model compilation device provided in an embodiment of this application;

[0026] Figure 7 An internal structural diagram of a computer device provided in an embodiment of this application;

[0027] Figure 8 An internal structural diagram of another computer device provided in an embodiment of this application;

[0028] Figure 9 This is an internal structural diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0029] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0030] In some embodiments, as shown in Figure 1 A model compilation method is provided, which is described by taking a computer device as an example, and includes the following steps:

[0031] S102, obtaining a plurality of tensor programs.

[0032] The plurality of tensor programs are a plurality of intermediate programs generated by a deep learning compiler according to different compilation optimization strategies for a to-be-compiled model.

[0033] Exemplarily, the computer device can perform syntax analysis on a source program of the to-be-compiled model through the deep learning compiler to obtain an intermediate program after syntax analysis. It can be understood that the intermediate program after syntax analysis is an intermediate language between the source program and machine language, which is equivalent to the internal representation of the source program. The computer device can perform equivalent transformation on the intermediate program after syntax analysis according to different compilation optimization strategies through the deep learning compiler to obtain a plurality of tensor programs. It can be understood that the so-called equivalent transformation refers to transformation without changing the running result of the intermediate program after syntax analysis. One compilation optimization strategy is one equivalent transformation.

[0034] In some embodiments, the computer device can include at least one of a terminal and a server. It can be understood that the model compilation method provided by the present application can be executed by the terminal or the server alone, or by a system including the terminal and the server.

[0035] In some embodiments, the terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices or portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle-mounted device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc.

[0036] In some embodiments, the server can be implemented by an independent server or a server cluster composed of multiple servers.

[0037] S104, for each tensor program, performing feature extraction on the loop optimization statements in the tensor program to obtain loop optimization features.

[0038] The loop optimization statement is a program statement with a loop optimization feature. The loop optimization refers to optimization performed on a loop statement. The loop optimization feature refers to a feature of loop optimization, such as loop unrolling, loop parallelization, loop sequence change, loop merging, tiling, loop reorder, and the like. The loop optimization feature can represent the loop optimization feature possessed by the loop optimization statement.

[0039] The computer device can determine the program statements in each loop body included in each tensor program, and obtain the loop optimization statement. The computer device can perform feature extraction on each loop optimization statement in each tensor program, and obtain the loop optimization feature corresponding to each loop optimization statement.

[0040] In S106, the attention encoder is used to encode the loop optimization feature corresponding to the loop optimization statement and the operator category information, and obtain feature encoding data.

[0041] The operator category refers to the category of the operator. Different network layers in the to-be-compiled model can include different operators, such as a convolution layer operator, a fully connected layer operator, or an activation layer operator. The operator category information is used to indicate the category of the operator corresponding to the loop optimization statement. It can be understood that the intermediate program includes program statements for each operator; the program statement for each operator is equivalent to the expression of the operator, and is used to express the operator.

[0042] The computer device can determine the operator category information and the loop optimization feature corresponding to each loop optimization statement, and obtain the matched operator category information and the loop optimization feature. The computer device can use the matched operator category information and the loop optimization feature as the input of the cost prediction model, and use the attention encoder in the cost prediction model to encode the matched operator category information and the loop optimization feature, and obtain the feature encoding data.

[0043] In S108, the cost prediction result of the tensor program is obtained by performing cost prediction on the tensor program according to the feature encoding data.

[0044] The cost prediction result is used to indicate the program performance corresponding to the tensor program. It can be understood that the tensor program is actually equivalent to a pseudo code, which is used to indicate the compilation of the to-be-compiled model into machine language. The above program performance is actually the program performance of the machine language compiled based on the tensor program.

[0045] The computer device can perform feature mapping processing on the feature encoding data to implement cost prediction on the tensor program, and obtain the cost prediction result.

[0046] In some embodiments, the computer device can perform feature mapping processing on the feature encoding data through the fully connected layer in the cost prediction model.

[0047] In some embodiments, the cost prediction result corresponding to each tensor program is used to indicate the speed of running the machine language compiled based on the tensor program on the processor or the length of the running time. It can be understood that the cost prediction of the tensor program is actually a performance prediction of the machine language compiled based on the tensor program.

[0048] S110, determining a target tensor program from the plurality of tensor programs based on the cost prediction result corresponding to each tensor program.

[0049] The target tensor program is used to indicate that the machine language is generated by the deep learning compiler for the to-be-compiled model. The program performance corresponding to the target tensor program is better than the program performance corresponding to the tensor programs other than the target tensor program.

[0050] Exemplarily, the cost prediction result can include a cost prediction score. The size of the cost prediction score is positively correlated with the pros and cons of the program performance corresponding to the tensor program, that is, the larger the cost prediction score of the tensor program, the better the corresponding program performance. The computer device can determine the tensor program with the highest cost prediction score from the plurality of tensor programs to obtain the target tensor program. The computer device can convert the target tensor program into machine language through the deep learning compiler.

[0051] In the above model compilation method, the deep learning compiler generates a plurality of tensor programs according to different compilation optimization strategies for the to-be-compiled model, extracts features of the loop optimization statements in each tensor program to obtain loop optimization features, the loop optimization features can reflect the compilation optimization strategy corresponding to the corresponding tensor program, and the cost prediction result obtained based on the loop optimization features can reflect the pros and cons of the compilation optimization strategy. Further, through the attention encoder, the more important information in the loop optimization features and the operator category information corresponding to the loop optimization statements is selectively focused based on the attention mechanism for encoding processing to obtain feature encoding data. Compared with the way of simply statistically predicting according to the data distribution, the cost prediction result obtained according to the feature encoding data is more accurate, and the accurate cost prediction result can accurately indicate the better tensor program. Subsequently, the target tensor program is determined from the plurality of tensor programs according to the cost prediction result, without the need for repeated multiple searches, thereby improving the efficiency of model compilation.

[0052] In some embodiments, the feature extraction of the loop optimization statements in the tensor program to obtain the loop optimization features comprises:

[0053] determining a loop body in the tensor program; each loop body includes at least one program statement;

[0054] For each loop body, a loop optimization statement corresponding to the loop body is determined from at least one program statement included in the loop body;

[0055] Feature extraction is performed on the loop optimization statement to obtain a loop optimization feature corresponding to the loop body.

[0056] The loop structure in the tensor program is composed of a loop body and a loop condition.

[0057] For example, the computer device can determine each loop body in the tensor program, and determine a loop optimization statement corresponding to each loop body from the innermost program statement in each loop body. The computer device can perform feature extraction on each loop optimization statement, map the loop optimization characteristics possessed by the loop optimization statement into a loop optimization vector, and obtain a loop optimization feature corresponding to each loop optimization statement. It can be understood that feature extraction on the loop optimization statement is equivalent to vector modeling of the loop optimization statement, which is actually mapping the features of the loop optimization statement to a vector system.

[0058] In some embodiments, the dimension of the loop optimization feature can be a preset feature dimension. The computer device can perform feature extraction on the loop optimization statement to obtain a loop optimization feature of the feature dimension. For example, the feature dimension can be 1*164.

[0059] In this embodiment, the loop body in the tensor program is determined, for each loop body, a loop optimization statement corresponding to the loop body is determined from at least one program statement included in the loop body, feature extraction is performed on the loop optimization statement to obtain a loop optimization feature corresponding to the loop body, and the loop body often determines the running speed of the entire operator, which is a calculation bottleneck in the compilation optimization process. Cost prediction based on the loop optimization feature helps the optimization of the entire operator and can effectively improve the model compilation efficiency.

[0060] In some embodiments, the loop optimization feature corresponding to the loop optimization statement and the operator category information are encoded by an attention encoder to obtain feature encoding data, including:

[0061] For each loop optimization statement, the category of the operator corresponding to the loop optimization statement is determined to obtain operator category information, and the loop optimization feature corresponding to the loop optimization statement and the operator category information are combined to obtain comprehensive features.

[0062] The comprehensive features of each loop optimization statement are encoded by the attention encoder to obtain feature encoding data.

[0063] Exemplarily, the computer device can determine the category of the operator corresponding to each loop optimization statement to obtain operator category information corresponding to the loop optimization statement. The computer device can splice the loop optimization features corresponding to each loop optimization statement and the operator category information to obtain comprehensive features corresponding to each loop optimization statement. The computer device can perform encoding processing on the comprehensive features of each loop optimization statement by using multiple attention encoders in the cost prediction model to obtain feature encoding data corresponding to the tensor program.

[0064] In some embodiments, as Figure 2 An illustrative diagram of splicing loop optimization features and operator category information is provided. Figure 2 The program statements in the for loop structure shown in the above table are used to express operators, including a loop optimization statement 1 (Computation statement 1) and a loop optimization statement 2 (Computation statement 2). The computer device can perform feature extraction on the loop optimization statement 1 to obtain a vector of loop optimization features with a length of 164 (1*164 dimensions). The computer device can determine a vector of operator category information with a length of 10 (1*10 dimensions) corresponding to the loop optimization statement. It can be understood that the data dimensions of the loop optimization features and the operator category information can be preset and are not limited to the data dimensions with a length of 164 in the present embodiment and the data dimensions with a length of 10. By splicing the vector of the loop optimization features and the vector of the operator category information, a vector of comprehensive features with a length of 174 (1*174 dimensions) is obtained. It should be noted that the length of the operator category information is not less than the total number of operator categories included in any model to avoid the omission of operator categories.

[0065] In the present embodiment, for each loop optimization statement, the category of the operator corresponding to the loop optimization statement is determined to obtain operator category information, and the loop optimization features corresponding to the loop optimization statement and the operator category information are spliced to obtain comprehensive features. The comprehensive features of each loop optimization statement are encoded by using an attention encoder to obtain feature encoding data, and the cost prediction is performed by combining the loop optimization features and the corresponding operator category information, so that different operators in the model to be compiled are fully considered, and the accuracy of the cost prediction is improved.

[0066] In some embodiments, the comprehensive features of each loop optimization statement are encoded by using an attention encoder to obtain feature encoding data, including:

[0067] By using an attention encoder, the comprehensive features of each loop optimization statement are encoded based on the dependency relationship between the multiple loop optimization statements to obtain feature encoding data.

[0068] The dependency relationship between the loop optimization statements is related to the loop body in which the loop optimization statements are located, and the dependency relationship can include an indirect dependency relationship or a direct dependency relationship. If the loop bodies are directly dependent, the corresponding loop optimization statements have a direct dependency relationship. If the loop bodies are indirectly dependent, the corresponding loop optimization statements have an indirect dependency relationship.

[0069] For example, the computer device performs self-attention calculation on the comprehensive features of the loop optimization statements by the attention encoder to learn the dependency relationship between the loop optimization statements. It can be understood that the higher the dependency degree between the loop optimization statement and other loop optimization statements except the loop optimization statement, the greater the attention weight corresponding to the comprehensive feature of the loop optimization statement. The computer device can assign attention weights to the comprehensive features according to the dependency degree between the loop optimization statements by the attention encoder, so as to encode the comprehensive features of each loop optimization statement and obtain the feature encoding data corresponding to the tensor program.

[0070] In some embodiments, as Figure 3 An illustrative diagram of encoding processing by the attention encoder is shown. The attention encoder can be an attention encoder in a transformer. The tensor program includes loop optimization statement 1, loop optimization statement 2, loop optimization statement 3, and loop optimization statement 4. The computer device can obtain comprehensive features by merging the loop optimization features and operator category information corresponding to each loop optimization statement. V1 represents the comprehensive feature of loop optimization statement 1, V2 represents the comprehensive feature of loop optimization statement 2, V3 represents the comprehensive feature of loop optimization statement 3, and V4 represents the comprehensive feature of loop optimization statement 4. The computer device can learn the dependency relationship between V1, V2, V3, and V4 by the attention encoder to obtain key data K, query data Q, and value data V. The computer device can calculate the correlation between the key data and the query data by the attention encoder to obtain the weight corresponding to the value data, and perform weighted processing on the value data using the weight corresponding to the value data to obtain the feature encoding data. The computer device can obtain the cost prediction score by feature mapping the feature encoding data.

[0071] In this embodiment, the comprehensive features of each loop optimization statement are encoded by the attention encoder based on the dependency relationship between the loop optimization statements to obtain the feature encoding data, rather than simply analyzing the loop optimization statements. The dependency relationship between the loop optimization statements is fully considered for encoding, and the cost prediction result obtained based on the feature encoding data is also more accurate.

[0072] In some embodiments, the comprehensive features of each loop optimization statement are encoded by an attention encoder to obtain feature encoding data, including:

[0073] The comprehensive features of each loop optimization statement are determined as inputs of the cost prediction model, and the comprehensive features of each loop optimization statement are encoded by multiple attention encoders in the cost prediction model to obtain feature encoding data; wherein the output of a previous attention encoder is used as the input of a next attention encoder.

[0074] For example, the computer device can determine the comprehensive features of each loop optimization statement as inputs of the cost prediction model, encode the comprehensive features of each loop optimization statement by a first attention encoder in the cost prediction model, use the output of the first attention encoder as the input of a next attention encoder, and encode by the next attention encoder to obtain feature encoding data output by the next attention encoder. The structures of the attention encoders are the same.

[0075] In this embodiment, the comprehensive features of each loop optimization statement are determined as inputs of the cost prediction model, and the comprehensive features of each loop optimization statement are encoded by multiple attention encoders in the cost prediction model to obtain feature encoding data, and accurate cost prediction results can be obtained based on the feature encoding data.

[0076] In some embodiments, the attention encoder is arranged in the cost prediction model; the cost prediction model further includes a multi-layer perceptron; and the cost prediction of the tensor program based on the feature encoding data obtains a cost prediction result, including:

[0077] The feature encoding data is determined as the input of the multi-layer perceptron, and the feature encoding data is feature mapped by the multi-layer perceptron to obtain the cost prediction result corresponding to the tensor program.

[0078] For example, the cost prediction model includes an attention encoder and a fully connected layer. The fully connected layer can be a multi-layer perceptron. The computer device can perform feature linear mapping processing on the feature encoding data by the multi-layer perceptron in the cost prediction model to obtain the cost prediction result corresponding to the tensor program. It can be understood that the multi-layer perceptron plays a role of full connection.

[0079] In some embodiments, as Figure 4The structure diagram of the cost prediction model is shown. The cost prediction model includes a calculation layer and a regression layer. The calculation layer includes two attention encoders, and the two attention encoders have the same structure. The attention encoder includes a first normalization layer, an embedding layer, a multi-head attention layer, a second normalization layer, and a first multi-layer perceptron. It can be understood that the structure of the cost prediction model provided in this embodiment is very simple, and the implementation is simple, and the cost prediction model can be embedded into the deep learning compiler using the PyTorch framework without too complex programming work.

[0080] The computer device can normalize the input of the attention encoder through the first normalization layer, preliminarily encode the output of the first normalization layer through the embedding layer, perform self-attention calculation on the output of the embedding layer through the multi-head attention layer, normalize the superimposed output of the multi-head attention layer and the output of the embedding layer through the second normalization layer, and obtain the output of the attention encoder through the first multi-layer perceptron. The number of attention heads and the size of the multi-head attention layer can be pre-set, for example, the number of attention heads can be 4, and the size can be 512.

[0081] The regression layer includes a second multi-layer perceptron. The computer device can input the feature encoding data into the regression layer, perform feature linear mapping on the feature encoding data through the second multi-layer perceptron in the regression layer, and obtain the cost prediction result. The cost prediction model can be deployed on the processor included in the computer device, and the model parameters can be continuously adjusted through online training on the processor.

[0082] In this embodiment, the feature encoding data is determined as the input of the multi-layer perceptron, the feature encoding data is processed through the multi-layer perceptron to obtain the cost prediction result corresponding to the tensor program, and the target tensor program is determined from the plurality of tensor programs according to the accurate cost prediction result, without repeatedly searching multiple times, thereby improving the efficiency of model compilation.

[0083] In some embodiments, the cost prediction result is obtained by taking the loop optimization feature corresponding to the loop optimization statement and the operator category information as the input of the cost prediction model; and the method further includes:

[0084] An initial cost prediction model is obtained. The initial cost prediction model is obtained based on the labeled sample tensor program through offline training on the first processor.

[0085] The initial cost prediction model is migrated to the second processor, and the initial cost prediction model is trained online to obtain the cost prediction model.

[0086] The first processor and the second processor can be different processors. The label of the sample tensor program is obtained by executing the sample machine language by the first processor. The sample machine language is generated by a deep learning compiler based on the sample tensor program.

[0087] Exemplarily, the computer device can generate the sample machine language based on the sample tensor program by a deep learning compiler. The computer device obtains an execution speed or an execution time of executing the sample machine language by the first processor, and obtains the label of the sample tensor program. The computer device can perform offline training on the to-be-trained cost prediction model based on the labeled sample tensor program on the first processor to obtain an initial cost prediction model. The computer device can obtain the initial cost prediction model, migrate the initial cost model to the second processor, and perform online training on the initial cost prediction model on the second processor to obtain the cost prediction model.

[0088] It should be noted that the first processor is a source processor, that is, the initial cost prediction model is a model in a source domain. The second processor is a target processor, that is, the cost prediction model is a model in a target domain. By offline training the initial cost prediction model on the first processor and then migrating the initial cost prediction model to the second processor for online training on the second processor, the initial cost prediction model learns sufficient knowledge in a large enough source domain through offline training, and then the learned knowledge is converted into the cost prediction model in the target domain through as few iteration training times as possible in the target domain. It can be understood that the domain of the first processor is not less than the domain of the second processor, that is, the source domain is not less than the target domain.

[0089] In some embodiments, as Figure 5 A schematic diagram of migrating the cost prediction model is provided. The domain of the first processor is a source domain, and the domains of the second processor and the third processor are target domains. The first processor, the second processor, and the third processor can be graphics processors. The computer device can migrate the migration features in the initial cost prediction model in the source domain to the target domain in the second processor or the third processor.

[0090] In this embodiment, the initial cost prediction model is obtained; the initial cost prediction model is migrated to the second processor, and the initial cost prediction model is trained online to obtain the cost prediction model. Through offline training on the source processor and online training on the target processor, the migration of the cost prediction model is realized, so that the cost prediction model can adapt to multiple processors.

[0091] It should be understood that although each step in the flowchart involved in each of the above embodiments is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowchart involved in each of the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0092] Based on the same inventive concept, the embodiments of the present application also provide a model compiling device. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more model compiling device embodiments provided below can refer to the limitations of the model compiling method in the above text, which will not be repeated here.

[0093] In some embodiments, as shown in Figure 6 A model compiling device 600 is provided, comprising:

[0094] The acquisition module 602 is configured to acquire a plurality of tensor programs, wherein the plurality of tensor programs are a plurality of intermediate programs generated by a deep learning compiler according to different compilation optimization strategies for a to-be-compiled model.

[0095] The extraction module 604 is configured to, for each tensor program, extract features of a loop optimization statement in the tensor program to obtain loop optimization features, wherein the loop optimization statement is a program statement with loop optimization characteristics.

[0096] The cost prediction module 606 is configured to encode the loop optimization features corresponding to the loop optimization statement and the operator category information by using an attention encoder to obtain feature encoding data, and predict a cost of the tensor program according to the feature encoding data to obtain a cost prediction result.

[0097] The compiling module 608 is configured to determine a target tensor program from the plurality of tensor programs based on the cost prediction result corresponding to each tensor program, wherein the target tensor program is used to indicate that a machine language is generated for the to-be-compiled model by the deep learning compiler.

[0098] In some embodiments, in the aspect of extracting features of a loop optimization statement in a tensor program to obtain loop optimization features, the extraction module 604 is specifically configured to:

[0099] determining a loop body in the tensor program; each loop body includes at least one program statement;

[0100] For each loop body, determining a loop optimization statement corresponding to the loop body from the at least one program statement included in the loop body;

[0101] characteristic extraction is performed on the loop optimization statement to obtain the loop optimization characteristic corresponding to the loop body.

[0102] In some embodiments, in the aspect of encoding and processing the loop optimization characteristic corresponding to the loop optimization statement and the operator category information through the attention encoder to obtain the feature encoding data, the cost prediction module 606 is specifically configured to:

[0103] For each loop optimization statement, determining the category of the operator corresponding to the loop optimization statement to obtain the operator category information; and merging the loop optimization characteristic corresponding to the loop optimization statement and the operator category information to obtain the comprehensive characteristic;

[0104] encoding and processing the comprehensive characteristic of each loop optimization statement through the attention encoder to obtain the feature encoding data.

[0105] In some embodiments, in the aspect of encoding and processing the comprehensive characteristic of each loop optimization statement through the attention encoder to obtain the feature encoding data, the cost prediction module 606 is specifically configured to:

[0106] encoding and processing the comprehensive characteristic of each loop optimization statement based on the dependency relationship between the multiple loop optimization statements through the attention encoder to obtain the feature encoding data.

[0107] In some embodiments, in the aspect of encoding and processing the comprehensive characteristic of each loop optimization statement 5 through the attention encoder to obtain the feature encoding data, the cost prediction module 606 is specifically configured to:

[0108] determining the comprehensive characteristic of each loop optimization statement as the input of the cost prediction model, and encoding and processing the comprehensive characteristic of each loop optimization statement through multiple attention encoders in the cost prediction model to obtain the feature encoding data; wherein the output of the previous attention encoder is taken as the input of the next attention encoder.

[0109] In some embodiments, the attention encoder is arranged in the cost prediction model; the cost prediction model further includes a multi-layer perception; in the aspect of predicting the cost of the tensor program according to the feature encoding data to obtain the cost prediction result, the cost prediction module 606 is specifically configured to:

[0110] determining the feature encoding data as the input of the multi-layer perception, and performing feature mapping processing on the feature encoding data through the multi-layer perception to obtain the cost prediction result corresponding to the tensor program.

[0111] 5. In some embodiments, the cost prediction result is the output result obtained by taking the loop optimization features and operator category information corresponding to the loop optimization statement as input to the cost prediction model; the acquisition module 602 is also used to: acquire the initial cost prediction model; the initial cost prediction model is obtained by offline training on the first processor based on the labeled sample tensor program; the initial cost prediction model is transferred to the second processor and trained online to obtain the cost prediction model.

[0112] The modules in the aforementioned model compilation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0113] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as shown below. Figure 7 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores tensor programs. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external computer devices via a network connection. When the computer program is executed by the processor, it performs the steps in the model compilation method described above.

[0114] In some embodiments, a computer device is provided, which may be a computer device, and its internal structure diagram may be as follows: Figure 8As shown in the figure. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through the system bus, the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control ability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the external computer device in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to realize the steps in the above-mentioned model compilation method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0115] Those skilled in the art can understand that, Figure 7 or Figure 8 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0116] In some embodiments, a computer device is provided, which includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps in the above-mentioned method embodiments.

[0117] In some embodiments, as Figure 9 A block diagram of the internal structure of a computer readable storage medium is provided, which stores a computer program. The computer program is executed by the processor to realize the steps in the above-mentioned method embodiments.

[0118] In some embodiments, a computer program product is provided, which includes a computer program. The computer program is executed by the processor to realize the steps in the above-mentioned method embodiments.

[0119] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of the country and region.

[0120] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to a memory, database or other medium used in the embodiments provided by the present application can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (Read-Only Memory, ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive memory (ReRAM), a magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), a ferroelectric memory (Ferroelectric Random Access Memory, FRAM), a phase change memory (Phase Change Memory, PCM), a graphene memory, etc. The volatile memory can include a random access memory (Random Access Memory, RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0121] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present disclosure.

[0122] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the patent scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A model compilation method, characterized by, The method comprises the following steps: obtaining a plurality of tensor programs; the plurality of tensor programs are a plurality of intermediate programs generated by a deep learning compiler for a to-be-compiled model according to different compilation optimization strategies respectively; for each of the tensor programs, extracting features of a loop optimization statement in the tensor program to obtain loop optimization features; the loop optimization statement is a program statement with loop optimization characteristics, and is determined from the innermost program statement of each loop body of the tensor program; for each of the loop optimization statements, determining the category of the operator corresponding to the loop optimization statement to obtain operator category information; combining the loop optimization features and the operator category information corresponding to the loop optimization statement to obtain comprehensive features; encoding the comprehensive features of each of the loop optimization statements by using an attention encoder to obtain feature encoding data; predicting the cost of the tensor program according to the feature encoding data to obtain a cost prediction result; based on the cost prediction result corresponding to each of the tensor programs, determining a target tensor program from the plurality of tensor programs; the target tensor program is used to indicate that a machine language is generated by the deep learning compiler for the to-be-compiled model.

2. The method of claim 1, wherein, The method comprises the following steps: determining a loop body in the tensor program; each of the loop bodies comprises at least one program statement; for each of the loop bodies, determining the loop optimization statement corresponding to the loop body from the at least one program statement included in the loop body; extracting features of the loop optimization statement to obtain the loop optimization features corresponding to the loop body.

3. The method of claim 1, wherein, The method comprises the following steps: performing feature mapping processing on the feature encoding data by using a fully connected layer in a cost prediction model to obtain a cost prediction result; the cost prediction result is used to represent the speed of running or the length of running time of the machine language compiled based on the tensor program on a processor.

4. The method of claim 1, wherein, The method comprises the following steps: based on the dependency relationship between a plurality of the loop optimization statements, performing encoding processing on the comprehensive features of each of the loop optimization statements by using an attention encoder to obtain feature encoding data.

5. The method of claim 3, wherein, The method comprises the following steps: determining the comprehensive features of each of the loop optimization statements as inputs of a cost prediction model, performing encoding processing on the comprehensive features of each of the loop optimization statements by using a plurality of attention encoders in the cost prediction model to obtain feature encoding data; wherein the output of a previous attention encoder is used as the input of a next attention encoder.

6. The method of claim 1, wherein, The attention encoder is arranged in the cost prediction model; the cost prediction model further comprises a multi-layer perception; the cost prediction of the tensor program according to the feature encoding data comprises: The feature encoding data is determined as the input of the multi-layer perception, and the feature mapping processing of the feature encoding data is performed through the multi-layer perception to obtain the cost prediction result corresponding to the tensor program.

7. The method according to any one of claims 1 to 6, characterized in that, The cost prediction result is obtained by taking the loop optimization feature and the operator category information corresponding to the loop optimization statement as the input of the cost prediction model; the method further comprises: An initial cost prediction model is obtained; the initial cost prediction model is obtained by offline training of a labeled sample tensor program on a first processor; The initial cost prediction model is migrated to a second processor, and the initial cost prediction model is trained online to obtain the cost prediction model.

8. A model compiling apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire a plurality of tensor programs; the plurality of tensor programs are a plurality of intermediate programs generated by a deep learning compiler according to different compilation optimization strategies for a to-be-compiled model; An extraction module is configured to extract loop optimization features from loop optimization statements in each of the tensor programs; The loop optimization statement is a program statement with loop optimization characteristics, and is determined from each loop body of the tensor program; A cost prediction module is configured to determine the category of an operator corresponding to each loop optimization statement to obtain operator category information; The loop optimization feature and the operator category information corresponding to the loop optimization statement are combined to obtain comprehensive features; The comprehensive features of each loop optimization statement are encoded by an attention encoder to obtain feature encoding data; the cost prediction of the tensor program is performed according to the feature encoding data to obtain a cost prediction result; A compilation module is configured to determine a target tensor program from the plurality of tensor programs based on the cost prediction result corresponding to each of the tensor programs; the target tensor program is used to indicate that a machine language is generated by the deep learning compiler for the to-be-compiled model.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

11. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images