Graph optimization sequence generation for deep learning compiler, and training of sequence generation model

Abstract

Disclosed in the present application are a graph optimization sequence generation method and apparatus for a deep learning compiler, and a method and apparatus for training a sequence generation model. The graph optimization sequence generation method for a deep learning compiler comprises: in response to an optimization request, determining a graph intermediate representation corresponding to each operator included in a target model; using a plurality of optimization strategies and the graph intermediate representations as inputs, and inputting same into an encoder of a trained sequence generation model, so as to obtain hidden-layer features output by the encoder and optimization features respectively corresponding to the plurality of optimization strategies; inputting the hidden-layer features into a decoder of the sequence generation model, so as to obtain vectors respectively corresponding to a plurality of ranks in a target sequence, which vectors are output by the decoder, and then determining the target sequence on the basis of relevancy between the vectors and the plurality of optimization features, wherein the target sequence includes the plurality of optimization strategies and the execution order of the plurality of optimization strategies; and returning the target sequence.

Description

Graph-optimized sequence generation for deep learning compilers and training of sequence generation models Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a graph optimization sequence generation method and device for a deep learning compiler, a training method and device for a sequence generation model, an electronic device, and a storage medium. Background Art

[0002] At present, with the development of computer technology and the need for business integration, deep learning technology has been widely used in many fields such as machine vision and natural language processing. As a result, the number of businesses executed using models has increased. In this case, how to improve the efficiency of models in executing business has become one of the technical problems that need to be solved.

[0003] A commonly used method for improving the efficiency of a model's business execution is to identify the operators included in the target model and, through these operators, determine the graph intermediate representation (IR) corresponding to the target model. This graph IR is then optimized according to a number of pre-defined optimization strategies to obtain an optimized result. Finally, based on the optimized result, code executable by the target hardware is generated and deployed on the target hardware, which then executes the corresponding code to execute the business.

[0004] However, when optimizing a graph IR, even if the same optimization strategy is used, the performance of the resulting optimization results may vary if the execution order of the optimization strategies is different. Therefore, optimizing a graph IR based on a preset optimization strategy is an urgent problem that needs to be solved. Based on this, this application provides a method and apparatus for generating a graph optimization sequence for a deep learning compiler.

[0005] Summary of the Invention

[0006] This application provides a graph optimization sequence tuning method and device for a deep learning compiler to partially solve the above-mentioned problems existing in the prior art. This application adopts the following technical solutions.

[0007] In a first aspect, the present application provides a graph optimization sequence generation method for a deep learning compiler, the method comprising: in response to an optimization request, determining a graph intermediate representation corresponding to each operator contained in a target model; determining a plurality of preset optimization strategies, and inputting the plurality of optimization strategies and the graph intermediate representation as input into an encoder of a pre-trained sequence generation model, obtaining hidden features output by the encoder and optimization features corresponding to the plurality of optimization strategies respectively; inputting the hidden features into a decoder of the sequence generation model, obtaining vectors corresponding to a plurality of ranks in a target sequence output by the decoder, and then determining the target sequence based on the correlation between the vectors corresponding to the plurality of ranks and the plurality of optimization features, the target sequence including the plurality of optimization strategies and the execution order of the plurality of optimization strategies; and returning the target sequence.

[0008] Optionally, determining the target sequence includes: for each rank in the target sequence, determining the optimization strategies corresponding to each other rank before the rank in the target sequence as each allocated strategy, and taking the other optimization strategies except the each allocated strategy as each unallocated strategy; determining the correlation between the vector corresponding to the rank and the optimization features corresponding to the each unallocated strategy; determining the probability that the rank corresponds to the each unallocated strategy based on the obtained correlations; sampling the each unallocated strategy based on the probability that the rank corresponds to the each unallocated strategy, and taking the sampled unallocated strategy as the optimization strategy corresponding to the rank.

[0009] Optionally, the method further includes: setting the probability that the ranking corresponds to each of the allocated strategies to 0.

[0010] Optionally, obtaining vectors corresponding to multiple ranks in the target sequence output by the decoder includes: for the decoder corresponding to the first rank, obtaining the vector corresponding to the first rank based on the trained initial parameters and the hidden layer features; for the decoder corresponding to the i-th rank, obtaining the vector corresponding to the i-th rank based on the hidden layer features output by the decoder corresponding to the i-1-th rank and the optimization strategy corresponding to the i-1-th rank, wherein i is an integer greater than 1 and less than or equal to the number of ranks in the target sequence.

[0011] Optionally, the encoder and the decoder are recurrent neural network structures.

[0012] In a second aspect, the present application provides a method for training a sequence generation model, the method comprising: obtaining a sample model, and determining an intermediate representation of a sample graph corresponding to each operator contained in the sample model; determining multiple sample optimization strategies corresponding to the sample model; inputting the intermediate representation of the sample graph and the multiple sample optimization strategies into an encoder of a sequence generation model to be trained, and obtaining hidden layer features output by the encoder and sample features corresponding to the multiple sample optimization strategies respectively; obtaining a sample sequence through a decoder of the sequence generation model based on the hidden layer features, the multiple sample features and the multiple sample optimization strategies, wherein the multiple sample optimization strategies in the sample sequence are The correlation between the corresponding rankings is positively correlated with the probability that the decoder outputs the sample sequence; the sample model is optimized according to the sample sequence, and the time required to execute the usage task of the sample model according to the optimization result is determined as the sample period corresponding to the sample sequence; the operations of obtaining the encoder output, obtaining a sample sequence, and determining the sample period corresponding to the sample sequence are repeated to obtain multiple sample sequences of the same sample model and the sample period corresponding to each sample sequence; the sequence generation model is trained with the optimization goal that the sample period corresponding to each sample sequence of the same sample model is inversely proportional to the probability that the decoder outputs the sample sequence.

[0013] Optionally, the method further includes: determining, for each rank in each sample sequence, the correlation between the rank and the multiple sample optimization strategies; determining, based on the correlation between the rank and the multiple sample optimization strategies, the probability that the rank selects the corresponding sample optimization strategy; obtaining the probability that each rank in the sample sequence selects the corresponding sample optimization strategy; and taking the product of each of the probabilities as the output probability of the sample sequence, wherein the output probability is used to characterize the probability that the decoder outputs the sample sequence.

[0014] Optionally, obtaining a sample sequence includes: for each rank in the sample sequence, determining the sample optimization strategies corresponding to each other rank before the rank in the sample sequence as each allocated strategy, and taking other sample optimization strategies except the each allocated strategy as each unallocated strategy; determining the correlation between the vector corresponding to the rank and the sample features corresponding to each unallocated strategy; determining the probability that the rank corresponds to each unallocated strategy based on the obtained correlations; sampling each unallocated strategy based on the probability that the rank corresponds to each unallocated strategy, and taking the sampled unallocated strategy as the sample optimization strategy corresponding to the rank.

[0015] Optionally, the method also includes: for the decoder corresponding to the first rank, obtaining the vector corresponding to the first rank based on the initial parameters and the hidden layer features; for the decoder corresponding to the i-th rank, obtaining the vector corresponding to the i-th rank based on the hidden layer features output by the decoder corresponding to the i-1-th rank and the sample optimization strategy selected for the i-1-th rank, where i is an integer greater than 1 and less than or equal to the number of ranks in the sample sequence.

[0016] Optionally, training the sequence generation model includes multiple iterative trainings, using one sample model in one iterative training and using different sample models in different iterative trainings.

[0017] Optionally, the encoder and the decoder are recurrent neural network structures.

[0018] In a third aspect, the present application provides a graph optimization sequence generation device for a deep learning compiler, the device comprising: a response module for determining, in response to an optimization request, a graph intermediate representation corresponding to each operator contained in a target model; an encoding module for determining a plurality of preset optimization strategies, and taking the plurality of optimization strategies and the graph intermediate representation as input, inputting the plurality of optimization strategies and the graph intermediate representation into an encoder of a pre-trained sequence generation model, to obtain hidden features output by the encoder and optimization features corresponding to the plurality of optimization strategies respectively; a decoding module for inputting the hidden features into a decoder of the sequence generation model, to obtain vectors corresponding to a plurality of ranks in a target sequence output by the decoder, and then determining the target sequence based on the correlation between the vectors corresponding to the plurality of ranks and the plurality of optimization features, the target sequence including the plurality of optimization strategies and the execution order of the plurality of optimization strategies; and a return module for returning the target sequence.

[0019] In a fourth aspect, the present application provides a training device for a sequence generation model, the device comprising: an acquisition module for obtaining a sample model and determining an intermediate representation of a sample graph corresponding to each operator contained in the sample model; a determination module for determining a plurality of sample optimization strategies corresponding to the sample model; an encoding module for inputting the intermediate representation of the sample graph and the plurality of sample optimization strategies into an encoder of the sequence generation model to be trained, and obtaining hidden layer features output by the encoder and sample features corresponding to the plurality of sample optimization strategies respectively; a decoding module for obtaining a sample sequence through a decoder of the sequence generation model according to the hidden layer features, the plurality of sample features and the plurality of sample optimization strategies, wherein the correlation between the plurality of sample optimization strategies and their corresponding rankings in the sample sequence and the correlation between the plurality of sample optimization strategies and the corresponding rankings in the sample sequence and the correlation between the plurality of sample optimization strategies and the corresponding rankings in the sample sequence are The sample model is configured to generate a plurality of sample sequences of the same sample model and a sample period corresponding to each sample sequence; the ...

[0020] In a fifth aspect, the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the graph optimization sequence generation method of the deep learning compiler of the first aspect or the training method of the sequence generation model of the second aspect.

[0021] In a sixth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the graph optimization sequence generation method of the deep learning compiler of the first aspect or the training method of the sequence generation model of the second aspect is implemented.

[0022] At least one of the above-mentioned technical solutions adopted in the present application can achieve the following beneficial effects: in the graph optimization sequence generation method of the deep learning compiler provided in the present application, the hidden layer features and the optimization features corresponding to each optimization strategy are determined according to the graph intermediate representation of the target model and multiple optimization strategies through a pre-trained sequence generation model, and then the vectors corresponding to each ranking are determined through the decoder of the sequence generation model, and then the optimization strategies corresponding to each ranking are determined based on the correlation between each vector and each optimization feature, so as to determine the target sequence, and then optimize the target model based on the target sequence, thereby improving the running time of the compiled model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0024] FIG1 is a schematic flow chart of a method for generating a graph optimization sequence for a deep learning compiler provided in an embodiment of the present application;

[0025] FIG2 is a schematic diagram of the structure of a sequence generation model provided in an embodiment of the present application;

[0026] FIG3A is a schematic diagram of a training process of a sequence generation model provided in this application;

[0027] FIG3B is a schematic diagram of a training process of another sequence generation model provided in an embodiment of the present application;

[0028] FIG4A is a schematic diagram of a graph optimization sequence generation device of a deep learning compiler provided in an embodiment of the present application;

[0029] FIG4B is a schematic diagram of a training device for a sequence generation model provided in an embodiment of the present application;

[0030] FIG5 is a schematic diagram of an electronic device corresponding to FIG1 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0032] In practical applications, deep learning models can be deployed on multiple hardware resources. Specifically, the deep learning model can be divided into several operators, which are then executed by multiple hardware resources. The server can obtain the computational graph corresponding to the deep learning model, identify the operators contained in the computational graph, and determine the hardware resources that match each operator.

[0033] The server can determine the high-level IR corresponding to the deep learning model through the deep learning compiler running on it, and perform graph optimization on the high-level IR to obtain the optimized IR (also called low-level IR). The optimized IR can be used directly for code generation (such as generating a processor executable program) or after operator optimization before code generation. The graph optimization mentioned here can refer to the graph optimization strategies performed by conventional deep learning compilers, which can include constant folding, dead code elimination, common subexpression elimination, etc.

[0034] The computational graph mentioned above can refer to a directed acyclic graph that represents the logical relationships between the operators included in a deep learning model. The operators mentioned here can refer to the basic computational units that constitute a deep learning model. For example, different activation functions can be used as different operators; different convolution operations can be used as different operators; and operations such as summation, product, and modulo can also be used as separate operators. A deep learning model can be composed of multiple operators. Each operator can correspond to a matching hardware resource. For example, if there are hardware resources 1, hardware resources 2, and hardware resources 3, operator 1 may be supported by hardware resources 1, hardware resources 2, and hardware resources 3, while operator 2 may only be supported by hardware resources 1 and hardware resources 2.

[0035] The above process of determining the graph IR, optimizing the graph IR, obtaining the optimization results, and finally generating the code for execution by the target hardware based on the optimization results is the process of the deep learning compiler performing the compilation task.

[0036] The following is a detailed description of the technical solutions provided by each embodiment of the present application in conjunction with the accompanying drawings. Figure 1 is a flow chart of a method for generating a graph optimization sequence for a deep learning compiler provided by the present application. The method includes the following steps.

[0037] S100: In response to an optimization request, determining a graph intermediate representation corresponding to each operator included in a target model.

[0038] In an embodiment of the present application, a graph optimization sequence generation method for a deep learning compiler is provided, wherein the sequence generation model involved may be obtained through pre-training, and the execution process of the graph optimization sequence generation method for the deep learning compiler may be performed by a server for executing the model training process, or an electronic device such as a server for executing the model optimization method, or a deep learning compiler for compiling the target model into executable code. The electronic device for executing the training process of the sequence generation model and the server for executing the graph optimization sequence generation method for the deep learning compiler may be the same server or different servers, and this application does not limit this. For the sake of ease of description, the graph optimization sequence generation method for the deep learning compiler provided in this application will be described in detail below, taking the server as the execution subject.

[0039] In an embodiment of the present application, the server may receive an optimization request. In some examples, the optimization request may include a model identifier of a target model for optimization. In some examples, the optimization request may also include a model structure and model parameters of the target model for optimization. In some examples, the optimization request may also include a graph intermediate representation (i.e., graph IR) corresponding to the target model for optimization.

[0040] The server can parse the received optimization request to determine the target model for optimization, and then determine the graph IR of the target model based on the deep learning framework and operators included in the target model. Alternatively, the server can parse the received optimization request and directly determine the graph IR corresponding to the target model.

[0041] The server may also directly receive an optimization task, which similarly carries a model identifier of a target model to be optimized, a model structure and model parameters of the target model to be optimized, and a graph intermediate representation corresponding to the target model to be optimized. The specific graph intermediate representation of the target model may be determined as needed and is not limited in this application.

[0042] It should be noted that the graph optimization sequence generation method for a deep learning compiler provided in this application can be specifically applied to the step of optimizing the computational graph when a deep learning compiler executes a compilation task. In other words, the generated target sequence can be used to execute the computational graph optimization step in the compilation task.

[0043] S102: Determine multiple preset optimization strategies, and use the multiple optimization strategies and the intermediate representation of the graph as input to an encoder of a pre-trained sequence generation model to obtain hidden layer features output by the encoder and optimization features corresponding to the multiple optimization strategies.

[0044] In one or more embodiments provided in the present application, as mentioned above, the graph optimization sequence generation method of the deep learning compiler in the present application is intended to determine the execution order corresponding to each optimization strategy. For this purpose, the technical means adopted in the present application include: determining the optimization features corresponding to each optimization strategy on the encoder side, determining the vectors corresponding to each ranking (corresponding to the decoding order of the decoder) based on the input hidden layer features on the decoder side, and then determining the optimization strategies corresponding to each ranking based on the correlation between the vectors corresponding to each ranking and each optimization feature. Based on this, the server can first determine each optimization feature.

[0045] For example, the server may determine the optimization strategies corresponding to the target model, and may determine a specified number of optimization strategies from various optimization strategies pre-stored in the server itself as the optimization strategies corresponding to the target model according to a preset specified number.

[0046] Alternatively, the optimization strategy may be carried in the optimization request, and the server may parse the optimization request to determine the optimization strategies carried in the optimization request as the optimization strategies corresponding to the target model.

[0047] Alternatively, the optimization strategy corresponding to the target model may also be all optimization strategies supported by the deep learning compiler.

[0048] Afterwards, the server can use the optimization strategies corresponding to the target model as input to the encoder of the pre-trained sequence generation model to obtain the optimization features corresponding to each optimization strategy output by the encoder. In addition, the encoder can also obtain hidden layer features.

[0049] Exemplarily, the sequence generation model can adopt a seq2seq structure, comprising an encoder and a decoder. Both the encoder and decoder can be implemented using a recurrent neural network (RNN). The RNN consists of an input layer, a hidden layer, an output layer, and a delay unit (also known as a memory unit). The aforementioned optimized features are the features output by the output layer, while the hidden layer features are the features output by the hidden layer. The hidden layer features have a memory function, which can record previous information and thus capture dependencies in the sequence.

[0050] The server can take each optimization strategy as input and input them into the encoder in sequence to obtain the optimization features corresponding to each optimization strategy output by the encoder.

[0051] S104: Input the hidden layer features into the decoder of the sequence generation model to obtain vectors corresponding to multiple ranks in the target sequence output by the decoder, and then determine the target sequence based on the correlation between the vectors corresponding to the multiple ranks and the multiple optimization features. The target sequence includes the multiple optimization strategies and the execution order of the multiple optimization strategies.

[0052] In one or more embodiments provided in the present application, as described above, the graph optimization sequence generation method of the deep learning compiler needs to determine the vectors corresponding to each ranking, and then determine the optimization strategies corresponding to each ranking based on the correlation between the vectors corresponding to each ranking and each optimization feature, so as to generate the target sequence.

[0053] Specifically, for each rank in the target sequence (e.g., each rank i in rank 1 to rank n), the server can use the hidden features obtained by the encoder as input to the decoder of the sequence generation model. For the decoder corresponding to rank 1 (which can be understood as the first decoder or the first layer of neurons in the decoder), the server can use the trained initial parameters to generate the hidden features. <s>And the hidden layer features input by the encoder, the vector corresponding to rank 1 is obtained. For the decoder corresponding to rank i, the vector corresponding to rank i can be obtained based on the hidden layer features output by the decoder corresponding to rank i-1 and the strategy selected by rank i-1, and the vector corresponding to the rank i output by the decoder (which can be recorded as q i ). Then, according to the vector q i And each optimized feature (can be recorded as {enc1,enc2,…,enc n }), the server can determine the correlation between the ranking feature and each optimization feature (such as the correlation between ranking i and optimization feature 1, the correlation between ranking i and optimization feature 2, ..., the correlation between ranking i and optimization feature n). Then the server can determine the optimization strategy with the highest correlation with the ranking from each optimization strategy based on each correlation, as the optimization strategy corresponding to the ranking. Finally, according to the optimization strategies corresponding to each ranking, the server can determine the target sequence. Among them, the target sequence includes each optimization strategy and the execution order of each optimization strategy (such as ranking 1 corresponds to optimization strategy 2, ranking 2 corresponds to optimization strategy 1, ranking 3 corresponds to optimization strategy 3, and the corresponding optimization strategies are executed according to rankings 1 to 3). And for each of the preset multiple optimization strategies, the optimization strategy is executed and only executed once in the target sequence.

[0054] It should be noted that, the decoder obtains the vector corresponding to the ranking output by the decoder based on the hidden layer features and the output of the previous decoding step, specifically including: when decoding is performed for the first time, the decoder obtains the vector corresponding to the first ranking according to the initial parameters and the hidden layer features obtained from the encoding, and determines the optimization strategy corresponding to the first ranking (also referred to as the first selected optimization strategy) according to the vector corresponding to the first ranking and the optimization features; when decoding is performed for the second time, the decoder obtains the vector corresponding to the second ranking according to the first selected optimization strategy and the hidden layer features obtained from the previous decoder, and determines the optimization strategy corresponding to the second ranking (also referred to as the second selected optimization strategy) according to the vector corresponding to the second ranking and the optimization features except the first selected optimization strategy; and so on, the vectors corresponding to each ranking are obtained and the optimization strategies corresponding to each ranking are finally determined.

[0055] Based on the correlations, the decoder selects the optimization strategy with the highest correlation to the ranking from among the optimization strategies. This is done by sampling the optimization strategies based on the correlation probability. Therefore, the higher the correlation between the ranking and a particular optimization strategy, the more likely it is to be selected. The pre-trained sequence generation model ensures that the correlation between each optimization strategy obtained by the decoder and the corresponding ranking is as high as possible.

[0056] Furthermore, the decoder may adopt a recurrent neural network structure. The server inputs the hidden layer features corresponding to each optimization strategy into the decoder, and obtains the ranking features corresponding to each ranking outputted by the decoder in sequence.

[0057] Based on the same idea, the present application provides a structural diagram of a sequence generation model, as shown in Figure 2. In the figure, the server can input each optimization strategy into the encoder of the sequence generation model in sequence, and obtain the optimization features corresponding to each strategy output by the encoder. The figure takes the three optimization strategies of strategy 1, strategy 2 and strategy 3 as an example for explanation, and it should be noted that the number of strategies included in the above optimization strategy and the input order of each strategy are only examples. The number of preset optimization strategies and the input order corresponding to each optimization strategy can be set as needed, and this application does not impose any restrictions on this. It should be noted that for the sequence generation model trained by the method provided in the present disclosure, the order of the optimization strategies input into the sequence generation model does not affect the result of the target sequence finally generated. The input order in Figure 2 is strategy 1, strategy 2 and strategy 3. The target sequence obtained by inputting in this order is the same as the target sequence obtained by inputting in other input orders (such as strategy 2, strategy 3 and strategy 1).

[0058] Then, the server can input the hidden layer features into the decoder to obtain the vectors corresponding to each rank in the target sequence, such as vectors 1 to 3 in FIG2 .

[0059] Finally, based on the vector corresponding to the ranking and each optimized feature (such as optimized features 1 to 3 in Figure 2), the correlation between the vector corresponding to the ranking and each optimized feature is determined. In some examples, for each optimized feature, the correlation between the optimized feature and the vector corresponding to the ranking can be the Euclidean distance, cosine distance, etc. between the optimized feature and the ranked vector. The specific method for measuring the correlation between the optimized feature and the ranked vector can be set as needed, and this application does not impose any restrictions on this.

[0060] In Figure 2, the arrows on the left pointing from the strategy to the encoder are used to represent the input of the strategy into the encoder. The arrows between the encoders are used to represent that for each strategy, the hidden features corresponding to the strategy are determined based on the input strategy itself and the hidden features of one or more strategies input to the encoder before the strategy. The arrows from the encoder to the decoder are used to represent that after the encoder encodes each strategy, the hidden features obtained based on each strategy are input into the decoder. The optimized features on the top of a certain encoder (or understood as a certain neuron of the encoder) represent the optimized features output by the output layer of the encoder. The arrows between the decoders are used to represent that for each decoder (or understood as each neuron of the decoder), the hidden layer of the decoder obtains the hidden features based on the input and outputs the obtained hidden features to the next decoder. The vector on the top of the decoder represents the vector output by the output layer of the decoder. For the first decoder, it is based on the hidden features output by the encoder and the initial parameters <s>, the corresponding hidden features and vector 1 are obtained. Correlation can then be calculated between vector 1 and optimized features 1-3, as shown in the branch from vector 1 to optimized features 1-3 in the upper portion of Figure 2. Exemplarily, the calculation result shows that vector 1 has the highest correlation with optimized feature 2 (i.e., strategy 2), as indicated by the arrow pointing to optimized feature 2 in the figure. The second decoder takes the hidden features output by the first decoder and the previously selected strategy 2 as input, obtaining the corresponding hidden features and vector 2. Correlation can then be calculated between vector 2 and optimized features 1-3. Since optimized feature 2 has already been selected by the first decoder, the "⊥" on the branch from vector 2 to optimized feature 2 indicates that optimized feature 2 is not included in the calculation (or the calculated correlation is irrelevant). Exemplarily, the calculation result shows that vector 2 has the highest correlation with optimized feature 1 (i.e., strategy 1). Similarly, the third decoder outputs vector 3, which corresponds to strategy 3. This results in the target sequence, where each rank in the target sequence represents the execution order of the strategy corresponding to that rank. For the example shown in FIG2 , the target sequence is a sequence ranked in the order of optimization strategy 2, optimization strategy 1, and optimization strategy 3.

[0061] S106: Return the target sequence.

[0062] In one or more embodiments provided herein, the graph optimization sequence generation method of the deep learning compiler aims to determine an optimization strategy that maximizes the execution efficiency of the target model. Therefore, after determining the target sequence, the server can return the target sequence based on the optimization request.

[0063] The server may determine the target sequence and return the target sequence as a response / result to the optimization request.

[0064] After receiving the target sequence, the server used to optimize the target model can optimize the graph IR of the target model according to the target sequence to obtain the optimization result, and compile the execution code executable by the target hardware according to the optimization result, so that the target hardware can build the model by executing the execution code, so that the model can be used to perform corresponding tasks later.

[0065] As shown in Figure 1, the graph optimization sequence generation method of the deep learning compiler uses a pre-trained sequence generation model to determine the optimization features and hidden features corresponding to each optimization strategy based on the intermediate representation of the graph of the target model. Then, the decoder of the sequence generation model is used to determine the vectors corresponding to each ranking. Based on the correlation between the vectors corresponding to each ranking and the optimization features, the optimization strategy corresponding to each ranking is determined to determine the target sequence. This application can determine the vector corresponding to the current ranking based on the hidden strategy and the optimization strategy selected at the previous level, so as to determine the optimization strategy that matches each ranking, thereby determining the order of the optimization strategies contained in the target sequence, and then optimizing the target model based on the target sequence to improve the running time of the compiled model.

[0066] Furthermore, Figure 3A is a schematic diagram of a training process of a sequence generation model provided in this application. As shown in Figure 3A, the sequence generation model in this application can be trained in the following manner.

[0067] In step S300 , the server may first obtain a sample model and determine a sample graph intermediate representation corresponding to each operator included in the sample model.

[0068] When training a sequence generation model, multiple sample models can be used to improve its generalization. The optimization strategies corresponding to different sample models may be the same or different, and this disclosure does not limit this. For simplicity, the following explanation uses a single sample model to train a sequence generation model as an example.

[0069] In step S302 , the server may determine each sample optimization strategy corresponding to the sample model.

[0070] In step S304, the server inputs the sample graph intermediate representation and each sample optimization strategy into the encoder of the sequence generation model to be trained, obtaining the hidden features output by the encoder and the sample features corresponding to each sample optimization strategy. Similar to the optimization strategy of the target model, the sample optimization strategy of the sample model can be determined by the server based on its own stored optimization strategies, or can be included in the model training request corresponding to the sequence generation model, or can be all optimization strategies supported by the deep learning compiler.

[0071] Step S306: Obtain a sample sequence through the decoder of the sequence generation model according to the hidden layer features, the features of each sample and the optimization strategy of each sample.

[0072] The method for the sequence generation model to obtain the sample sequence is similar to the method for obtaining the target sequence described above, and will not be repeated here.

[0073] In step S308, the server may optimize the sample model according to the sample sequence and determine the time required for the optimization result to execute a task using the sample model, as the sample period corresponding to the sample sequence. For example, each sample sequence may represent an optimization result, from which executable code corresponding to the sample model may be generated (for simplicity, the impact of possible operator optimization is not considered here). This code may then be used to execute the task corresponding to the sample model, thereby recording the time required to execute the task as the sample period corresponding to the sample sequence. In short, the sample period corresponding to the sample sequence may represent the runtime of the sample model.

[0074] Since the training samples (sample models) in this application do not have corresponding annotations, the server can use reinforcement learning to train the sequence generation model.

[0075] In step S310, the server may repeat steps S304-S308 above to obtain multiple sample sequences (e.g., N sample sequences) of the same sample model and the sample time periods corresponding to each sample sequence. The server may repeatedly input the optimization features corresponding to each sample optimization strategy of the sample model into the decoder of the sequence generation model to obtain multiple sample sequences output by the decoder. The server may also repeatedly input each optimization strategy and the intermediate representation of the sample graph of the sample model as input into the sequence generation model to obtain each sample sequence output by the sequence generation model. The specific method for determining each sample sequence can be set as needed and is not limited in this application.

[0076] For each sample sequence, the decoder outputs the optimization strategy corresponding to each rank based on probability, and this probability is positively correlated with the correlation between that rank and each optimization strategy. In other words, even if the input to the sequence generation model is identical, different sample sequences can be generated for the same sample model. For each rank in the sample sequence, the higher the correlation between the optimization strategy corresponding to that rank and that rank, the higher the probability that the decoder will output that optimization strategy.

[0077] Step S312 , training the sequence generation model with the sample period corresponding to each sample sequence of the same sample model being inversely proportional to the probability of the decoder outputting the sample sequence as an optimization goal.

[0078] Exemplarily, a policy-based reinforcement learning algorithm can be used to train the sequence generation model. For example, a policy gradient algorithm, such as the REINFORCE algorithm, can be used for training. The policy gradient algorithm updates the model parameters in a manner that updates after each episode (a round) (in this disclosure, a round refers to obtaining a sample sequence and its corresponding sample period). An evaluation index can be found first (in this disclosure, this evaluation index is related to the sample period, such as the inverse of the sample period), and then the stochastic gradient ascent method is used to update the parameters so that the evaluation index continues to rise. This evaluation index can also be called a reward.

[0079] For policy-based reinforcement learning methods, the probability of the entire trajectory is determined by outputting the probability of each action based on the state. For the method provided by this disclosure, the correlation / probability of the selected sample optimization strategy for each ranked sample is calculated based on the input graph IR and each optimization strategy. The correlation / output probability corresponding to the sample sequence is then determined. If the evaluation metric increases (e.g., the sample period decreases), the parameter update direction increases the probability of the corresponding sample sequence occurring in the current state.

[0080] Specifically, for each rank in each sample sequence, the correlation between the rank and the multiple sample optimization strategies can be determined; based on the correlation between the rank and the multiple sample optimization strategies, the probability of the rank selecting the corresponding sample optimization strategy can be determined; the probability of each rank in the sample sequence selecting the corresponding sample optimization strategy can be obtained; the product of the probabilities can be used as the output probability of the sample sequence, and the output probability is used to characterize the probability of the decoder outputting the sample sequence.

[0081] The model trained using this method can ensure that sample sequences with shorter execution periods have higher correlations with their corresponding sample sequences, while sample sequences with longer execution periods have lower correlations with their corresponding sample sequences. For each sample sequence, the encoder outputs a probability proportional to the correlation between each optimization strategy and its corresponding ranking within the sample sequence.

[0082] In other words, the training goal of the sequence generation model is to ensure that, after inputting each optimization strategy into the model, the model can output the fastest running sample sequence with the highest possible probability. In other words, the probability of a sample sequence with a short execution time is high, while the probability of a sample sequence with a long execution time is low. To this end, the server can obtain the corresponding loss value based on the probability of selecting each optimization strategy in a sample sequence (the action probability predicted by the neural network), the execution time (reward) of the sample sequence, and the actual ranking of the optimization strategies in the sample sequence (the actions actually executed), and train the sequence generation model based on the loss value.

[0083] Correspondingly, in step S104, the server may determine the sampling probability of each ranking for each optimization strategy, and then determine the optimization strategy corresponding to each ranking according to the sampling probability.

[0084] Specifically, for each rank, the server may use the correlation between each optimization strategy and the ranking feature corresponding to that rank as the probability of sampling each optimization strategy. Then, based on the probability of sampling each optimization strategy, the server may sample each unassigned strategy and use the sampled unassigned strategy as the optimization strategy corresponding to that rank.

[0085] In the present application, for each optimization strategy in the target sequence, the number of such optimization strategies in the target sequence is and is only one, that is, each optimization strategy appears only once in the target sequence. The server can determine, for each rank, the optimization strategy assigned to the rank from the optimization strategies that have not yet been assigned. Specifically, the server can determine, for each rank in the target sequence, the optimization strategies corresponding to the other ranks before the rank in the target sequence as the assigned strategies, and the strategies other than the assigned strategies as the unassigned strategies. Thus, the server can determine the correlation between the vector corresponding to the rank and the strategy features corresponding to the unassigned strategies, and determine the optimization strategy assigned to the rank based on the correlation.

[0086] The ranking feature corresponding to the i-th ranking in the target sequence is q i , the optimization features corresponding to each strategy output by the encoder are {enc1,enc2,…,enc n Taking the optimization feature and the ranking output vectors as row vectors or column vectors, and the dimensions of the two are equal, as an example, if a certain optimization strategy is not assigned to other rankings before the corresponding ranking in the target sequence (that is, the optimization strategy is an unassigned optimization strategy), then the correlation between the optimization feature j corresponding to the optimization strategy and the ranking i is u ij =q i ·enc j , otherwise take u ij =-∞ (that is, the correlation between the assigned optimization strategy and the ranking is -∞, so that the probability that the ranking corresponds to the assigned optimization strategy is 0), where 1≤j≤n.

[0087] For example, for each ranking, after determining the correlation between the ranking and each optimization strategy, the server may directly output the probability of each optimization strategy using the correlation corresponding to the optimization strategy as the ranking.

[0088] In some examples, the server may also construct a pre-correlation vector for each ranking based on the correlations between the ranking and each optimization strategy, and perform softmax normalization on the correlation vector to obtain a normalized result. Based on the normalized result, the server may re-determine the correlations for each optimization strategy. Specifically, for each optimization strategy, the server may directly use the normalized result of the correlation corresponding to the optimization strategy as the re-determined correlation for the optimization strategy.

[0089] For example, for the input sequence in Figure 2, assume the correlation vector for the second rank in the target sequence is [3, -∞, 1]. The correlation corresponding to the second optimization strategy is -∞ because it has already been assigned to the first rank in the target sequence. Then, normalizing this correlation vector yields the normalized value [0.88, 0, 0.12]. Based on this probability sampling, the output for this rank points to optimization strategy 1.

[0090] Based on the network structure in Figure 2, the probability of optimizing the arrangement can be modeled, namely where p(x j |x 0:j-1 ) represents the probability of the optimization strategy pointed to by the output at rank j. Compared with the above example, for the second output position, p(x2|x 0:1 )=0.88. For each sequence, the server can determine the product of the correlation between each ranking in the sequence and its corresponding optimization strategy, and output the probability of the sequence as the sequence generation model. In Figure 2, the connection structure above the encoder and decoder represents the server's u ij =q i ·enc j The network structure of the sequence generation model can be a pointer network. For example, after determining the correlation between each rank and each optimization strategy, the server samples each rank based on the correlation and generates a path from the decoder to the sampled result. For example, in Figure 2, the server can generate a path from the encoder corresponding to rank 2 to strategy 1.

[0091] After selecting the sequence generation model, initialize it and denote the model parameters as θ. Optimize the sequence generation model according to the reinforcement learning algorithm. The objective function J(θ) = E x～p(x) L(x), where L(x) represents the running time corresponding to the sample model optimization based on the sample sequence x. Using the REINFORCE algorithm, we can find the gradient of the objective function and approximate it with the Monte Carlo algorithm to obtain Where b is the baseline, which is used to reduce the variance of the estimate. For example, a sliding average of the running time under different optimization permutations can be taken. After obtaining the gradient, the model parameters θ are updated using the gradient descent method. It is used to characterize the gradient of the objective function when the model parameter is θ. N is the number of sample sequences output by the sequence generation model in one iteration. In one iteration, N sample sequences are obtained by sampling using the same sample model. L(x k ) is used to represent the sample period corresponding to the kth sample sequence in each sample sequence, p(x k ) is used to represent the output probability corresponding to the kth sample sequence in each sample sequence. Different sample models can be used in different iterations.

[0092] The training goal of the reinforcement learning can be: for each sample sequence, if the sample period corresponding to the sample sequence is short, then adjust the model parameters to make the output probability of the sample sequence higher; if the sample period corresponding to the sample sequence is long, then adjust the model parameters to make the output probability of the sample sequence lower.

[0093] Of course, the above reinforcement learning algorithm using REINFORCE is only an example, and the server can use other reinforcement learning algorithms to train the sequence generation model.

[0094] Based on the same idea, the present application provides a training process diagram of a sequence generation model, as shown in FIG3B .

[0095] 3B is a schematic diagram of the training process of another sequence generation model provided by the present application. The server can first initialize the model parameters of the sequence generation model, and determine the sample sequences corresponding to the sample model through the initialized sequence generation model. Determine the sample time period of each sample sequence, and the output probability corresponding to each sample sequence. Then, based on each sample optimization strategy in each sample sequence, the probability of each ranking in each sample sequence selecting the corresponding sample optimization strategy, and the sample time period of each sample sequence, adjust the model parameters of the sample model until the iteration termination condition is met. Among them, the iteration termination condition can be that the number of iterations of the sequence generation model reaches a preset number threshold. After the training is completed, the server can determine the parameters of the sequence generation model.

[0096] Figure 4A is a schematic diagram of a graph optimization sequence generation device of a deep learning compiler provided in this application, which includes the following modules.

[0097] The response module 200 is configured to determine the graph intermediate representation corresponding to each operator included in the target model in response to the optimization request.

[0098] The encoding module 202 is used to determine multiple preset optimization strategies, and input the multiple optimization strategies and the intermediate representation of the graph into the encoder of the pre-trained sequence generation model to obtain the hidden layer features output by the encoder and the optimization features corresponding to the multiple optimization strategies.

[0099] The decoding module 204 is used to input the hidden layer features into the decoder of the sequence generation model to obtain vectors corresponding to multiple rankings in the target sequence output by the decoder, and then determine the target sequence based on the correlation between the vectors corresponding to the multiple rankings and the multiple optimization features. The target sequence includes the multiple optimization strategies and the execution order of the multiple optimization strategies.

[0100] The returning module 206 is configured to return the target sequence.

[0101] Optionally, the decoding module 204 is further used to determine, for each rank in the target sequence, the optimization strategies corresponding to each other rank before the rank in the target sequence as each allocated strategy, and to use the other optimization strategies except the allocated strategies as each unallocated strategy; determine the correlation between the vector corresponding to the rank and the optimization features corresponding to the each unallocated strategy; determine the probability that the rank corresponds to the each unallocated strategy based on the obtained correlations; sample the each unallocated strategy based on the probability that the rank corresponds to the each unallocated strategy, and use the sampled unallocated strategy as the optimization strategy corresponding to the rank.

[0102] Optionally, the decoding module 204 is further configured to set the probability that the rank corresponds to each of the allocated strategies to 0.

[0103] Optionally, the decoding module 204 is further used to obtain, for the decoder corresponding to the first rank, the vector corresponding to the first rank based on the trained initial parameters and the hidden layer features; and for the decoder corresponding to the i-th rank, the vector corresponding to the i-th rank based on the hidden layer features output by the decoder corresponding to the i-1-th rank and the optimization strategy corresponding to the i-1-th rank, where i is an integer greater than 1 and less than or equal to the number of ranks in the target sequence.

[0104] Optionally, the encoder and the decoder are recurrent neural network structures.

[0105] FIG4B is a schematic diagram of a sequence generation model device provided in this application, which includes the following modules.

[0106] The obtaining module 400 is used to obtain a sample model and determine a sample graph intermediate representation corresponding to each operator included in the sample model.

[0107] The determination module 402 is configured to determine a plurality of sample optimization strategies corresponding to the sample model.

[0108] The encoding module 404 is used to input the intermediate representation of the sample graph and the multiple sample optimization strategies into the encoder of the sequence generation model to be trained, and obtain the hidden layer features output by the encoder and the sample features corresponding to the multiple sample optimization strategies respectively.

[0109] The decoding module 406 is used to obtain a sample sequence through the decoder of the sequence generation model based on the hidden layer features, the multiple sample features and the multiple sample optimization strategies, wherein the correlation between the multiple sample optimization strategies and their corresponding rankings in the sample sequence is positively correlated with the probability of the decoder outputting the sample sequence.

[0110] The execution module 408 is configured to optimize the sample model according to the sample sequence, and determine the time required to execute a task using the sample model according to the optimization result as the sample period corresponding to the sample sequence.

[0111] The repetition module 410 is configured to repeatedly input the sample graph intermediate representation and the multiple sample optimization strategies into the encoding module, the decoding module, and the execution module to obtain multiple sample sequences of the same sample model and a sample period corresponding to each sample sequence.

[0112] The training module 412 is configured to train the sequence generation model with the optimization goal of the sample period corresponding to each sample sequence of the same sample model being inversely proportional to the probability of the decoder outputting the sample sequence.

[0113] Optionally, the decoding module 406 is also used to determine, for each rank in each sample sequence, the correlation between the rank and the multiple sample optimization strategies; determine the probability of the rank selecting the corresponding sample optimization strategy based on the correlation between the rank and the multiple sample optimization strategies; obtain the probability of each rank in the sample sequence selecting the corresponding sample optimization strategy; and use the product of each of the probabilities as the output probability of the sample sequence, and the output probability is used to represent the probability of the decoder outputting the sample sequence.

[0114] Optionally, the decoding module 406 is also used to determine, for each rank in the sample sequence, the sample optimization strategies corresponding to each other rank before the rank in the sample sequence as each allocated strategy, and to use other sample optimization strategies except the allocated strategies as each unallocated strategy; determine the correlation between the vector corresponding to the rank and the sample features corresponding to the each unallocated strategy; determine the probability that the rank corresponds to the each unallocated strategy based on the obtained correlations; sample the each unallocated strategy based on the probability that the rank corresponds to the each unallocated strategy, and use the sampled unallocated strategy as the sample optimization strategy corresponding to the rank.

[0115] Optionally, the decoding module 406 is also used to obtain, for the decoder corresponding to the first rank, the vector corresponding to the first rank based on the initial parameters and the hidden layer features; and for the decoder corresponding to the i-th rank, the vector corresponding to the i-th rank based on the hidden layer features output by the decoder corresponding to the i-1-th rank and the sample optimization strategy selected for the i-1-th rank, where i is an integer greater than 1 and less than or equal to the number of ranks in the sample sequence.

[0116] Optionally, training the sequence generation model includes multiple iterative trainings, using one sample model in one iterative training and using different sample models in different iterative trainings.

[0117] Optionally, the encoder and the decoder are recurrent neural network structures.

[0118] The present application also provides a computer-readable storage medium storing a computer program that can be used to execute the graph-optimized sequence generation method for the deep learning compiler shown in FIG1 or the training method for the sequence generation model shown in FIG3A . The computer-readable storage medium can be a non-volatile storage medium.

[0119] The present application also provides a schematic structural diagram of an electronic device as shown in FIG5 . As shown in FIG5 , at the hardware level, the electronic device includes a processor, memory, and non-volatile memory, and of course may also include hardware required for other services, such as a network interface and an internal bus. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the graph optimization sequence generation method of the deep learning compiler shown in FIG1 or the training method of the sequence generation model shown in FIG3A . In addition to hardware implementation methods, the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0120] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system onto a PLD through their own programming, eliminating the need for chip manufacturers to design and manufacture dedicated integrated circuit chips. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly performed using "logic compiler" software. This is similar to the software compilers used during program development. Before compilation, the original code must be written in a specific programming language, called a hardware description language (HDL). There are many different HDLs, including ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also appreciate that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0121] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0122] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0123] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0124] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0125] The present application is described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.

[0126] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0127] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0128] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0129] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0130] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0131] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0132] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0134] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment.

[0135] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.< / s> < / s>

Claims

1. A graph optimization sequence generation method for a deep learning compiler, characterized in that: The method comprises: In response to the optimization request, determining a graph intermediate representation corresponding to each operator included in the target model; Determine a plurality of preset optimization strategies, and use the plurality of optimization strategies and the graph intermediate representation as inputs to an encoder of a pre-trained sequence generation model to obtain hidden layer features output by the encoder and optimization features corresponding to the plurality of optimization strategies respectively; Inputting the hidden layer features into the decoder of the sequence generation model, obtaining vectors corresponding to multiple ranks in the target sequence output by the decoder, and then determining the target sequence according to the correlation between the vectors corresponding to the multiple ranks and the multiple optimization features, wherein the target sequence includes the multiple optimization strategies and the execution order of the multiple optimization strategies; Returns the target sequence.

2. The method according to claim 1, characterized in that Determining the target sequence comprises: For each rank in the target sequence, Determine the optimization strategies corresponding to the other rankings before the ranking in the target sequence as the allocated strategies, and determine the other optimization strategies except the allocated strategies as the unallocated strategies; Determine the correlation between the vector corresponding to the ranking and the optimization features corresponding to each of the unassigned strategies; Determining the probability that the ranking corresponds to each of the unassigned strategies according to the obtained correlations; According to the probability that the ranking corresponds to the unallocated strategies, the unallocated strategies are sampled, and the sampled unallocated strategies are used as the optimization strategies corresponding to the ranking.

3. The method according to claim 2, characterized in that The method further comprises: The probability that the rank corresponds to each of the allocated strategies is set to 0.

4. The method according to claim 1, characterized in that The vectors corresponding to the multiple ranks in the target sequence output by the decoder are obtained, respectively, including: For the decoder corresponding to the first rank, based on the trained initial parameters and the hidden layer features, obtain a vector corresponding to the first rank; For the decoder corresponding to the i-th rank, based on the hidden layer features output by the decoder corresponding to the i-1-th rank and the optimization strategy corresponding to the i-1-th rank, a vector corresponding to the i-th rank is obtained, where i is an integer greater than 1 and less than or equal to the number of ranks in the target sequence.

5. The method according to claim 1, characterized in that The encoder and the decoder are recurrent neural network structures.

6. A method for training a sequence generation model, characterized in that: The method comprises: Obtaining a sample model, and determining a sample graph intermediate representation corresponding to each operator included in the sample model; Determining multiple sample optimization strategies corresponding to the sample model; Inputting the sample graph intermediate representation and the multiple sample optimization strategies into an encoder of a sequence generation model to be trained, and obtaining hidden layer features output by the encoder and sample features corresponding to the multiple sample optimization strategies respectively; According to the hidden layer features, the plurality of sample features and the plurality of sample optimization strategies, a sample sequence is obtained through a decoder of the sequence generation model, wherein the correlation between the plurality of sample optimization strategies and their corresponding rankings in the sample sequence is positively correlated with the probability that the decoder outputs the sample sequence; Optimizing the sample model according to the sample sequence, and determining the time required to execute the usage task of the sample model according to the optimization result as the sample period corresponding to the sample sequence; Repeat the operations of obtaining the encoder output, obtaining a sample sequence, and determining a sample period corresponding to the sample sequence to obtain multiple sample sequences of the same sample model and a sample period corresponding to each sample sequence; The sequence generation model is trained with the optimization goal that the sample period corresponding to each sample sequence of the same sample model is inversely proportional to the probability that the decoder outputs the sample sequence.

7. The method according to claim 6, characterized in that The method further comprises: For each ranking in each sample sequence, determining a correlation between the ranking and the plurality of sample optimization strategies; Determining the probability of selecting a corresponding sample optimization strategy for the ranking according to the correlation between the ranking and the plurality of sample optimization strategies; Obtain the probability of selecting the corresponding sample optimization strategy for each ranking in the sample sequence; The product of the probabilities is used as the output probability of the sample sequence, and the output probability is used to represent the probability that the decoder outputs the sample sequence.

8. The method according to claim 6, characterized in that The obtaining of a sample sequence comprises: For each rank in the sample sequence, Determine the sample optimization strategies corresponding to the other rankings before the ranking in the sample sequence as the allocated strategies, and determine the other sample optimization strategies except the allocated strategies as the unallocated strategies; Determine the correlation between the vector corresponding to the ranking and the sample features corresponding to each of the unassigned strategies; Determining the probability that the ranking corresponds to each of the unassigned strategies according to the obtained correlations; According to the probability that the ranking corresponds to the unallocated strategies, the unallocated strategies are sampled, and the sampled unallocated strategies are used as sample optimization strategies corresponding to the ranking.

9. The method according to claim 8, characterized in that The method further comprises: For a decoder corresponding to the first rank, obtaining a vector corresponding to the first rank based on initial parameters and the hidden layer features; For the decoder corresponding to the i-th rank, based on the hidden layer features output by the decoder corresponding to the i-1-th rank and the sample optimization strategy selected by the i-1-th rank, a vector corresponding to the i-th rank is obtained, where i is an integer greater than 1 and less than or equal to the number of ranks in the sample sequence.

10. The method according to claim 6, characterized in that Training the sequence generation model includes multiple iteration trainings, using one sample model in one iteration training and using different sample models in different iteration trainings.

11. The method according to claim 6, characterized in that The encoder and the decoder are recurrent neural network structures.

12. A graph optimization sequence generation device for a deep learning compiler, characterized in that: The device comprises: A response module, used to determine the graph intermediate representation corresponding to each operator included in the target model in response to the optimization request; An encoding module, used to determine a plurality of preset optimization strategies, and input the plurality of optimization strategies and the graph intermediate representation as input into an encoder of a pre-trained sequence generation model, to obtain hidden layer features output by the encoder and optimization features corresponding to the plurality of optimization strategies respectively; A decoding module, used for inputting the hidden layer features into a decoder of the sequence generation model, obtaining vectors corresponding to a plurality of ranks in a target sequence output by the decoder, and then determining the target sequence according to the correlation between the vectors corresponding to the plurality of ranks and the plurality of optimization features, wherein the target sequence includes the plurality of optimization strategies and the execution order of the plurality of optimization strategies; The return module is used to return the target sequence.

13. A training device for a sequence generation model, characterized in that: The device comprises: An acquisition module, used to obtain a sample model and determine a sample graph intermediate representation corresponding to each operator included in the sample model; A determination module, used to determine a plurality of sample optimization strategies corresponding to the sample model; An encoding module, used for inputting the sample graph intermediate representation and the multiple sample optimization strategies into an encoder of a sequence generation model to be trained, to obtain hidden layer features output by the encoder and sample features corresponding to the multiple sample optimization strategies respectively; A decoding module, configured to obtain a sample sequence through a decoder of the sequence generation model according to the hidden layer features, the plurality of sample features and the plurality of sample optimization strategies, wherein the correlation between the plurality of sample optimization strategies and their corresponding rankings in the sample sequence is positively correlated with the probability of the decoder outputting the sample sequence; An execution module is used to optimize the sample model according to the sample sequence and determine the execution result according to the optimization result. The time required to perform the task of using the sample model once is used as the sample period corresponding to the sample sequence; A repeating module, used for repeatedly inputting the sample graph intermediate representation and the multiple sample optimization strategies into the encoding module, the decoding module and the execution module to obtain multiple sample sequences of the same sample model and a sample period corresponding to each sample sequence; The training module is used to train the sequence generation model by taking the sample period corresponding to each sample sequence of the same sample model and the probability that the decoder outputs the sample sequence as an optimization target to be inversely proportional.

14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 5 or 6 to 11 is implemented.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 5 or 6 to 11 is implemented.

Images