Method for compiling neural network models, method and device for training optimization models

Abstract

To provide a method for compiling a neural network model, and a method and an apparatus for training an optimization model.SOLUTION: A method includes the steps of: obtaining a to-be-compiled neural network model; determining an intermediate command sequence based on the to-be-compiled neural network model; processing the intermediate command sequence by using a pre-trained command sequence optimization model, to obtain a target optimization parameter; determining an optimization command sequence based on the target optimization parameter; and converting the optimization command sequence into an executable command sequence, to obtain a target command sequence that is executable by a neural network processor.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to artificial intelligence technology, and in particular to a method for compiling a neural network model, and a method and apparatus for training an optimization model. [Background technology]

[0002] With the rapid development of artificial intelligence, deep neural network models (also called "models") are widely applied in many fields such as computer vision, natural language processing, speech recognition, and autonomous driving. After the network structure of a model is determined, the actual execution efficiency of the model mainly depends on the hardware and compiler. In order to improve the execution efficiency and processing power of the model, a compilation optimization technique is proposed and applied to the compilation process of the model, which can improve the execution efficiency and processing power of the model without increasing the hardware cost. In the compilation process of the related technology, generally, the model is first converted into an intermediate representation command sequence (also called an "intermediate command sequence" or "computation graph"), and then a rule for identifying the optimizable part of the model is artificially created, and a heuristic search algorithm is designed for the optimizable part, so as to traverse various optimization methods (optimization command sequence) to search for the optimal optimization method. However, the method of determining the optimal optimization method based on the heuristic search algorithm has a long traversal time, which results in a long compilation time and low compilation efficiency. Summary of the Invention [Problem to be solved by the invention]

[0003] In order to solve the above technical problems such as long compilation time and low efficiency, the embodiments of the present disclosure provide a neural network model compiling method and an optimization model training method and apparatus, thereby shortening the compilation time and improving the compilation efficiency. [Means for solving the problem]

[0004] A method for compiling a neural network model according to a first aspect of the present disclosure includes the steps of: obtaining a neural network model to be compiled; determining an intermediate command sequence corresponding to the neural network model to be compiled based on the neural network model to be compiled; processing the intermediate command sequence using a command sequence optimization model obtained by training in advance to obtain target optimization parameters corresponding to the intermediate command sequence; determining an optimized command sequence corresponding to the intermediate command sequence based on the target optimization parameters; and converting the optimized command sequence into an executable command sequence to obtain a target command sequence executable by a neural network processor corresponding to the neural network model to be compiled.

[0005] A method for training a command sequence optimization model according to a second aspect of the present disclosure includes the steps of obtaining training intermediate command sequences corresponding to at least one neural network model and optimally optimized parameter tags corresponding to each of the training intermediate command sequences, and training a pre-constructed command sequence optimization network based on each of the training intermediate command sequences and the optimally optimized parameter tags corresponding to each of the training intermediate command sequences to obtain a trained command sequence optimization model.

[0006] A neural network model compiling device according to a third aspect of the present disclosure includes a first acquisition module for acquiring a neural network model to be compiled, a first processing module for determining an intermediate command sequence corresponding to the neural network model to be compiled based on the neural network model to be compiled, a second processing module for processing the intermediate command sequence using a command sequence optimization model acquired by prior training to acquire target optimization parameters corresponding to the intermediate command sequence, a third processing module for determining an optimized command sequence corresponding to the intermediate command sequence based on the target optimization parameters, and a fourth processing module for converting the optimized command sequence into an executable command sequence to acquire a target command sequence executable by a neural network processor corresponding to the neural network model to be compiled.

[0007] A command sequence optimization model training apparatus according to a fourth aspect of the present disclosure includes a second acquisition module for acquiring training intermediate command sequences corresponding to at least one neural network model and optimal optimization parameter tags corresponding to each of the training intermediate command sequences, and a fifth processing module for training a pre-constructed command sequence optimization network based on each of the training intermediate command sequences and the optimal optimization parameter tags corresponding to each of the training intermediate command sequences to acquire a trained command sequence optimization model.

[0008] A computer-readable storage medium according to a fifth aspect of the present disclosure stores a computer program, the computer program executing the method for compiling a neural network model described in any of the above embodiments of the present disclosure, or the method for training a command sequence optimization model described in any of the above embodiments of the present disclosure.

[0009] An electronic device according to a sixth aspect of the present disclosure comprises a processor and a memory for storing commands executable by the processor, and the processor reads the executable commands from the memory and executes them to realize the neural network model compilation method described in any of the above embodiments of the present disclosure, or executes the commands to realize the command sequence optimization model training method described in any of the above embodiments of the present disclosure.

[0010] A seventh aspect of the present disclosure provides a computer program product that, when commands in the computer program product are executed by a processor, performs the method for compiling a neural network model as described in any of the above embodiments of the present disclosure, or performs the method for training a command sequence optimization model as described in any of the above embodiments of the present disclosure. Effect of the Invention

[0011] According to the neural network model compiling method and optimization model training method and device according to the above embodiments of the present disclosure, in the neural network model compiling process, after obtaining an intermediate command sequence corresponding to the neural network model to be compiled, the optimal optimization parameters can be quickly obtained through the command sequence optimization model obtained by training in advance, and the optimal optimization command sequence corresponding to the intermediate command sequence can be determined. When the optimized command sequence is converted into an executable command sequence, the neural network processor corresponding to the neural network model to be compiled obtains an executable target command sequence to complete the compilation of the neural network model, and the optimal optimization command sequence can be determined through the command sequence optimization model. Therefore, compared with the artificial creation of rules or the design of a heuristic search algorithm, the compilation time can be greatly shortened, and the compilation efficiency can be effectively improved. [Brief description of the drawings]

[0012] [Figure 1] 1 is an exemplary application scenario of a neural network model compilation method according to the present disclosure. [Diagram 2] 1 is a flowchart of a method for compiling a neural network model according to an exemplary embodiment of the present disclosure. [Diagram 3] FIG. 2 is a schematic diagram of a computational subgraph decomposition principle according to an exemplary embodiment of the present disclosure. [Figure 4] 11 is a flowchart of a method for compiling a neural network model according to another exemplary embodiment of the present disclosure. [Diagram 5] FIG. 2 is a schematic diagram of a network structure of a command sequence optimization model according to an exemplary embodiment of the present disclosure. [Figure 6] 10 is a flowchart of a method for compiling a neural network model according to a further exemplary embodiment of the present disclosure. [Figure 7] FIG. 13 is a schematic diagram of a network structure of a command sequence optimization model according to another exemplary embodiment of the present disclosure. [Figure 8] 1 is a flowchart of a method for training a command sequence optimization model according to an exemplary embodiment of the present disclosure. [Figure 9] 13 is a flowchart of a method for training a command sequence optimization model according to another exemplary embodiment of the present disclosure. [Figure 10] 13 is a flowchart of a method for training a command sequence optimization model according to a further exemplary embodiment of the present disclosure. [Figure 11] FIG. 13 is a schematic diagram of a network structure of a command sequence optimization model according to a further exemplary embodiment of the present disclosure. [Figure 12] FIG. 2 is a structural schematic diagram of a neural network model compilation device according to an exemplary embodiment of the present disclosure; [Figure 13] FIG. 13 is a structural schematic diagram of a neural network model compiling device according to another exemplary embodiment of the present disclosure; [Figure 14] FIG. 13 is a structural schematic diagram of a neural network model compilation device according to a further exemplary embodiment of the present disclosure; [Figure 15] FIG. 2 is a structural schematic diagram of a training device for a command sequence optimization model according to an exemplary embodiment of the present disclosure; [Figure 16] FIG. 13 is a structural schematic diagram of a training device for a command sequence optimization model according to another exemplary embodiment of the present disclosure. [Figure 17] FIG. 13 is a structural schematic diagram of a training device for a command sequence optimization model according to a further exemplary embodiment of the present disclosure; [Figure 18] FIG. 1 is a structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] In order to interpret the present disclosure, exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present disclosure, and are not all of the embodiments, and the present disclosure is not limited by the exemplary embodiments.

[0014] Unless otherwise specified, the relative arrangement of components and steps, formulas, and numerical values ​​described in these examples do not limit the scope of the present disclosure.

[0015] Summary of the Disclosure In the process of realizing the present disclosure, the inventors have found that with the rapid development of artificial intelligence, deep neural network models (also called "models") are widely applied to many fields such as computer vision, natural language processing, speech recognition, and automatic driving. After the network structure of the model is determined, the actual execution efficiency of the model mainly depends on the hardware and the compiler. In order to improve the execution efficiency and processing power of the model, a compilation optimization technique is proposed and applied to the compilation process of the model, and the compilation optimization technique can improve the execution efficiency and processing power of the model without increasing the hardware cost. In the compilation process of the related technology, generally, the model is first converted into an intermediate representation command sequence (also called an "intermediate command sequence" or "computation graph"), and then a rule for identifying an optimizable part of the model is artificially created, and a heuristic search algorithm is designed for the optimizable part, so as to traverse various optimization methods (optimization command sequence) to search for the optimal optimization method. The method of determining the optimal optimization method based on the heuristic search algorithm has a long traversal time, which results in a long compilation time and low compilation efficiency.

[0016] Illustrative Overview FIG. 1 is an exemplary application scenario of the neural network model compilation method according to the present disclosure.

[0017] In many scenes, such as computer vision, natural language processing, voice recognition, and autonomous driving, it is necessary to obtain intermediate command sequences corresponding to each of a plurality of neural network models as training intermediate command sequences, determine parameters of an optimal optimization method corresponding to each of the intermediate command sequences as optimal optimization parameter tags corresponding to each of the training intermediate command sequences, execute a training method for the corresponding command sequence optimization model by a training device for a command sequence optimization model of the present disclosure, train a command sequence optimization network previously constructed based on each training intermediate command sequence and the corresponding optimal optimization parameter tag, obtain a trained command sequence optimization model, place the command sequence optimization model in a compilation device for a neural network model of the present disclosure, and compile the neural network model to be compiled for the scene. In some cases, the method for compiling a neural network model disclosed herein (performed by the device for compiling a neural network model disclosed herein) can be used to determine an intermediate command sequence corresponding to a neural network model to be compiled, process the intermediate command sequence using a pre-arranged command sequence optimization model to obtain a target optimization parameter corresponding to the intermediate command sequence, the target optimization parameter meaning an optimization parameter corresponding to an optimal optimization method of the intermediate command sequence predicted by the command sequence optimization model, and further determine an optimized command sequence corresponding to the intermediate command sequence based on the target optimization parameter, convert the optimized command sequence into a command sequence executable by a neural network processor to obtain a target command sequence executable by a neural network processor corresponding to the neural network model to be compiled. The target command sequence is deployed to a corresponding terminal device for processing data to be processed to obtain a corresponding data processing result.The terminal device may be a device that needs to perform corresponding data processing based on the target command sequence of the neural network model to be compiled for any scene, such as an in-vehicle computing platform for an autonomous driving scene, a device used for voice recognition processing for a voice recognition scene, etc. The data to be processed may be determined based on the specific task type of the neural network model to be compiled, such as if the neural network model to be compiled is a target detection model for an autonomous driving scene, the data to be processed may be an image collected by a camera in a vehicle or a preprocessed image, and if the neural network model to be compiled is a voice recognition model, the data to be processed may be voice data to be recognized, and may be specifically set according to actual needs. The present disclosure can determine an optimal optimization command sequence through a trained command sequence optimization model, greatly shorten the compilation time, and effectively improve the compilation efficiency.

[0018] The method for compiling a neural network model according to the embodiment of the present disclosure can be applied to any scene that requires the compilation of a neural network model, and is not limited to the above scenes.

[0019] Exemplary Methods 2 is a flowchart of a method for compiling a neural network model according to an exemplary embodiment of the present disclosure. This embodiment can be specifically applied to electronic devices such as servers and terminals. As shown in FIG. 2, the method includes steps 201 to 205.

[0020] In step 201, a neural network model to be compiled is obtained.

[0021] Here, the neural network model to be compiled can be a neural network model of any task type and any network structure, and can be specifically set according to actual needs. The task type can include, for example, object detection, semantic segmentation, image classification, voice recognition, text recognition, etc., and the network structure can include, for example, convolutional neural network and its series network structure, Transformer and its series network structure, RNN (Recurrent Neural Network) and its series network structure, MLP (Multilayer Perceptron) and its series network structure, etc.

[0022] In step 202, based on the neural network model to be compiled, an intermediate command sequence corresponding to the neural network model to be compiled is determined.

[0023] Here, the intermediate command sequence is an intermediate representation between a programming language and an executable command, also called an initial computation graph, and the initial computation graph may include at least one computation subgraph, and each computation subgraph may include at least one operator node and a connection relationship between each operator node, that is, the intermediate command sequence represents an initial computation order relationship between intermediate commands corresponding to each operator in the neural network model to be compiled, and the operators may include computation operators such as a convolution computation operator, a matrix multiplication operator, a corresponding element addition operator, a corresponding element multiplication operator, and a data load operator, a data storage operator, etc., required for the computation of various operators. The data load operator is for loading data used in the corresponding computation such as tensors, weights, etc., and the data storage operator is for storing the computation result.

[0024] In some optional embodiments, the manner of obtaining the intermediate command sequence can be set according to actual needs, for example, an initial compilation can be performed on the neural network model to be compiled based on any executable compiler, and the neural network model to be compiled is converted from a programming language into an intermediate command sequence of an intermediate representation, but is not specifically limited thereto.

[0025] In step 203, the intermediate command sequence is processed using the command sequence optimization model obtained through pre-training to obtain target optimization parameters corresponding to the intermediate command sequence.

[0026] Here, the specific network structure of the command sequence optimization model can be set according to actual needs, for example, a network structure based on a recurrent neural network, a network structure based on a convolutional neural network, etc., but is not limited to these. The command sequence optimization model can be acquired by training based on the training intermediate command sequences of a plurality of neural network models acquired in advance and the corresponding optimal optimization parameter tags. The specific content of the target optimization parameters can be set according to actual needs, specifically, based on the optimizable methods of the intermediate command sequences of various neural network models, the optimization methods having commonality can be expressed by parameters, for example, the target optimization parameters can include parameters for determining an operator execution mode (also called operator execution mode parameters), parameters for determining a memory usage policy (also called memory management parameters), parameters for determining a tensor decomposition method (also called tensor decomposition parameters), etc.

[0027] The operator execution mode parameter represents the execution mode when the hardware executes the operator, and the execution mode when the hardware executes the operator may include at least one type, for example, a default execution mode and a sharing slice execution mode, and may include more other modes, but is not limited thereto. The execution modes included in different operators may be the same or different, and are specifically determined according to the hardware situation. The operator execution mode parameter included in the target optimization parameters is a predicted optimal execution mode.

[0028] Regarding the memory management parameters, after the network structure of the neural network model is determined, the memory size of each layer required for inference and the time between memory claim and release can be determined, where the memory size is the size of the interlayer tensor, the memory claim time is before the tensor generator generates it, and the memory release time is after the tensor is consumed by all consumers (users), so that the memory of the neural network model inference scene can be clearly grasped, and model optimization can be realized by optimizing the memory allocation, reducing memory waste, and based on this, the memory allocation is displayed by the memory management parameters, and the optimal memory management parameters of various neural network models are learned to realize the prediction of the optimal memory management parameters of the neural network model to be compiled. The memory management parameters include, for example, an early release (Prefree) parameter, and the early release parameter can include two types of states, early release and non-early release. The early release state indicates that the output of the immediately preceding calculation subgraph is released from the buffer to the memory in advance when scheduling the calculation subgraph later, and the memory management parameters can also include other parameters related to memory management, and can be specifically set according to actual needs.

[0029] The tensor decomposition parameter represents the decomposition method and decomposition number of the input tensor of the calculation operator, and for different calculation operators, at least one decomposition method can be provided according to the decomposable principle of the tensor of the calculation operator, and each decomposition method can be provided with at least one decomposition number. Different combinations of decomposition methods and decomposition numbers are represented by different tensor decomposition parameters, for example, for a convolution operator, the decomposition method for a tensor of H*W can be decomposing the tensor into multiple H1*W subtensors according to height, or decomposing the tensor into multiple H*W1 subtensors according to width, or decomposing the tensor into multiple H1*W1 subtensors according to two dimensions (height and width), etc. Here, the decomposition method includes three types of decomposition methods of height, width, and two dimensions, and various decomposition methods can correspond to multiple decomposition numbers. In practical application, any decomposition number in any decomposition method is one tensor decomposition parameter, that is, corresponds to one decomposition type, and this type of decomposition type can be identified by, for example, coding. Different decomposition types correspond to different codes.

[0030] In some selectable embodiments, the target optimization parameters may include optimal optimization parameters corresponding to each optimizable computation subgraph in the intermediate command sequence. The optimizable computation subgraphs may be those that the command sequence optimization model has learned in a training process. Since the network structure of the neural network model is generally composed of some general-purpose operators (e.g., the above-mentioned convolution operator, matrix multiplication operator, corresponding element-by-element addition operator, etc.), it may be possible to determine whether the computation subgraphs including these operators are optimizable based on the computation characteristics of the operators. For example, the computation subgraphs including the convolution operator may be optimized by tensor decomposition, and the computation subgraphs including the element-by-element addition operator may be optimized by tensor decomposition.

[0031] In step 204, an optimized command sequence that corresponds to the intermediate command sequence is determined based on the target optimization parameters.

[0032] Here, after the target optimization parameter is determined, the intermediate command sequence can be optimized based on the target optimization parameter to obtain a corresponding optimized command sequence.

[0033] For example, based on a tensor decomposition parameter corresponding to any one of the computation subgraphs included in the target optimization parameters, the computation subgraph is decomposed into computation subgraphs the number of which corresponds to the tensor decomposition parameter.

[0034] Illustratively, FIG. 3 is a schematic diagram of a computation subgraph decomposition principle according to an exemplary embodiment of the present disclosure. The computation subgraph in this example includes three operator nodes, such as a data load operator node, an element-wise addition operator node, and a data storage operator node, and performs two kinds of decomposition according to two different tensor decomposition parameters, respectively: tensor decomposition parameter 1 decomposes the computation subgraph into two target computation subgraphs, i.e., decomposes the input tensor into two subtensors to perform calculations respectively; tensor decomposition parameter 2 decomposes the computation subgraph into three target computation subgraphs.

[0035] In step 205, the optimized command sequence is converted into an executable command sequence to obtain a target command sequence executable by a neural network processor corresponding to the neural network model being compiled.

[0036] Here, the executable command sequence means a binary command sequence that can be executed by hardware. The neural network processor is a processor for accelerating a neural network model to improve the model inference speed. The purpose of compiling a neural network model to be compiled is to obtain a target command sequence to be executed by the neural network processor, so that the neural network processor can infer the model and improve the inference speed. The conversion of the optimized command sequence into the executable command sequence can be realized in any feasible manner, for example, the conversion of the optimized command sequence into the executable command sequence is realized based on a compiler that is compatible with the hardware (i.e., the neural network processor) that executes the target command sequence. This disclosure is not limited thereto.

[0037] In the neural network model compiling method according to the present embodiment, in the process of compiling a neural network model, after obtaining an intermediate command sequence corresponding to the neural network model to be compiled, the optimal optimization parameters can be quickly obtained through the command sequence optimization model obtained by training in advance, and the optimal optimized command sequence corresponding to the intermediate command sequence can be determined. After the optimized command sequence is converted into an executable command sequence, the neural network processor corresponding to the neural network model to be compiled obtains a target command sequence that is executable, and the compilation of the neural network model can be completed. The optimal optimized command sequence is determined through the command sequence optimization model, and the compilation time can be greatly shortened, thereby effectively improving the compilation efficiency.

[0038] FIG. 4 is a flowchart of a method for compiling a neural network model according to another exemplary embodiment of the present disclosure.

[0039] In one alternative embodiment, step 203 may specifically include steps 2031a to 2033 as follows:

[0040] In step 2031a, feature extraction is performed on the intermediate command sequence using a first feature extraction network in the command sequence optimization model to obtain command sequence features.

[0041] Here, the first feature extraction network can adopt any feasible network structure, such as a feature extraction network based on a recurrent neural network, a feature extraction network based on a convolutional neural network, etc., and can be specifically configured according to actual needs.

[0042] In step 2032, the command sequence features are processed using a predictive head network in the command sequence optimization model to obtain a processing result.

[0043] Wherein, the processing result includes the optimization parameter prediction probability corresponding to the intermediate command sequence. The optimization parameter probability can include the optimization parameter prediction probability corresponding to each of the computation subgraphs in at least one optimizable computation subgraph, and each optimizable computation subgraph can include the optimization parameter prediction probability corresponding to each of various parameters, such as at least one state probability corresponding to the early release parameter in the memory management parameter, the probability of each decomposition type corresponding to the tensor decomposition parameter, the probability of various execution modes of the operator execution mode, etc., and the specific structure of the prediction head network can be set according to actual needs, for example, the prediction head network can be a head network with a fully connected layer.

[0044] In one alternative embodiment, the tensor decomposition parameters may include a decomposition number prediction value, which may be a decimal number, such as 2.4, 3.7, etc.

[0045] Illustratively, the early release parameter is a probability value between 0 and 1, with 1 representing an early release state and 0 representing a non-early release state, and the corresponding predicted probability being a probability value between 0 and 1.

[0046] In step 2033, target optimization parameters are determined based on the processing results and the preset mapping rules.

[0047] Here, the preset mapping rule can be set based on the specific type of the parameter, for example, for a parameter of binary classification, a probability threshold can be set, and the predicted probability of the parameter can be compared with the probability threshold, and if it is greater than the probability threshold, it can indicate that it belongs to one classification, and if it is less than the probability threshold, it can indicate that it belongs to the other classification; for multiple types of parameters, a parameter corresponding to the class with the highest probability can be selected as the optimal parameter; for the predicted value of the decomposition number, rounding can be performed based on a rounding rule, and the predicted value can be mapped to the decomposition number, etc. For example, for a certain operator, if the predicted value of the predicted tensor decomposition number is 2.4, it can be converted to an integer by rounding or other rounding rules, for example, 2.4 can be rounded or converted to a lower integer to convert it to 2.

[0048] Illustratively, the early release parameter may include a predicted probability of early release of the processing result, and if the predicted probability is greater than a corresponding probability threshold, it determines that the computation subgraph needs to early release the output of the previous subgraph.

[0049] In an alternative embodiment, Figure 5 is a schematic diagram of a network structure of a command sequence optimization model according to an exemplary embodiment of the present disclosure. In this example, the network structure of the command sequence optimization model includes a first feature extraction network and a predictive head network. The first feature extraction network is for performing feature extraction on the intermediate command sequence to obtain command sequence features, and the predictive head network is for predicting the command sequence features to obtain a processing result.

[0050] In this embodiment, the predicted probability of the target optimization parameter is obtained by the feature extraction network and the prediction head network, and the predicted optimal target optimization parameter is obtained by post-processing, thereby realizing the fast automatic prediction of the optimal optimization parameter of the intermediate command sequence, and effectively improving the compilation processing efficiency.

[0051] FIG. 6 is a flowchart of a method for compiling a neural network model according to a further exemplary embodiment of the present disclosure.

[0052] In one alternative embodiment, the method of the present disclosure further includes a step 301 of obtaining a current memory information sequence.

[0053] Here, the current memory information sequence can be obtained during pre-compilation to obtain the intermediate command sequence, and the current memory information sequence indicates a memory allocation policy and an allocation result during command scheduling of the intermediate command sequence.

[0054] The step 203 of processing the intermediate command sequence using the command sequence optimization model obtained through pre-training to obtain target optimization parameters corresponding to the intermediate command sequence further includes steps 2031b and 2031c.

[0055] In step 2031b, the second feature extraction network in the command sequence optimization model is used to perform feature extraction on the current memory information sequence to obtain memory information features.

[0056] Here, the specific network structure of the second feature extraction network can be set according to actual needs, for example, a feature extraction network based on a recurrent neural network, a feature extraction network based on a fully connected network, but is not specifically limited. For example, the second feature extraction network may adopt a two-layer recurrent neural network, or a fully connected network with two or more layers.

[0057] In step 2031c, the command sequence feature and the memory information feature are fused to obtain a fusion feature.

[0058] Here, the fusion method of the command sequence feature and the memory information feature can be set according to actual needs, for example, concatenation, addition, etc., and can be set according to actual needs. If the command sequence feature and the memory information feature are completely different in dimension, they are transformed to be the same in at least one dimension by a set method, and then fusion is performed.

[0059] The step 2032 of processing the command sequence features using the predictive head network in the command sequence optimization model to obtain processing results includes a step 20321 of processing the fused features using the predictive head network to obtain processing results.

[0060] The prediction head network can be explained by referring to the above description, so a detailed explanation will be omitted here.

[0061] This embodiment can further improve the accuracy of the prediction result by assisting the prediction of the optimal optimization parameters with the current memory information sequence.

[0062] In one alternative embodiment, the step 2031c of blending the command sequence feature and the memory information feature to obtain a fusion feature includes aggregating the command sequence feature according to a first aggregation rule to obtain a first aggregate feature, aggregating the memory information feature according to a second aggregation rule to obtain a second aggregate feature, making the second aggregate feature and the first aggregate feature the same in at least one dimension, and concatenating the first aggregate feature and the second aggregate feature to obtain a fusion feature.

[0063] Here, the first and second aggregation rules can be set according to actual needs, and the function of aggregation is to allow the command sequence feature and the memory information feature to be linked. The first and second aggregation rules can both include an aggregation direction and an aggregation method of the aggregation direction, where the aggregation direction can include row aggregation and column aggregation, and the aggregation method can include taking any of the average value, maximum value, minimum value, addition, etc. in the corresponding aggregation direction. Specifically, they can be set according to actual needs. Through aggregation, the command sequence feature and the memory information feature can be linked in at least one dimension.

[0064] Illustratively, the command sequence features are 16*2 features and are aggregated by columns into a 1*2 first aggregate feature, and the aggregation method can be any of the average, maximum, minimum, addition, etc. of the 16 elements of each column. The memory information features are 6*4 features and are aggregated by columns into a 1*4 second aggregate feature, and the aggregation method can be any of the above as well, and the first aggregate feature and the second aggregate feature can be concatenated in the second dimension to obtain a 1*6 fusion feature.

[0065] This embodiment can combine command sequence features and memory information features in at least one dimension through aggregation to achieve feature fusion.

[0066] In an alternative embodiment, Figure 7 is a schematic diagram of a network structure of a command sequence optimization model according to another exemplary embodiment of the present disclosure. In this example, the network structure of the command sequence optimization model includes a first feature extraction network, a second feature extraction network, a feature fusion network and a predictive head network, where the first feature extraction network is for performing feature extraction on an intermediate command sequence to obtain a command sequence feature, the second feature extraction network is for performing feature extraction on a current memory information sequence to obtain a memory information feature, the feature fusion network is for fusing the command sequence feature and the memory information feature to obtain a fusion feature, and the predictive head network is for predicting the fusion feature to obtain a processing result.

[0067] In one alternative embodiment, if the command sequence feature and the memory information feature can be concatenated in a preset dimension, the feature fusion network can be a Concat layer.

[0068] In an alternative embodiment, when the command sequence feature and the memory information feature cannot be directly fused, the feature fusion network can include a first aggregation network, a second aggregation network, and a concatenation layer, where the first aggregation network is for aggregating the command sequence feature to obtain a first aggregate feature, the second aggregation network is for aggregating the memory information feature to obtain a second aggregate feature, and the concatenation layer is for concatenating the first aggregate feature and the second aggregate feature to obtain a fusion feature.

[0069] In an alternative embodiment, the target optimization parameters include a memory management parameter, an operator execution mode parameter, and a tensor decomposition parameter, and determining 204 an optimized command sequence corresponding to the intermediate command sequence based on the target optimization parameters includes: The method includes a step 2041 of optimizing the intermediate command sequence based on the memory management parameter, the operator execution mode parameter and the tensor decomposition parameter included in the target optimization parameters to obtain an optimized command sequence.

[0070] Here, after obtaining the target optimization parameters, the corresponding commands in the intermediate command sequence that need to be optimized can be optimized based on the target optimization parameters, for example, optimizing commands related to memory management in the intermediate command sequence based on the memory management parameters included in the target optimization parameters, updating the execution modes of the corresponding operators in the intermediate command sequence based on the operator execution mode parameters, decomposing the command sequences corresponding to the computation subgraphs in the intermediate command sequence into multiple groups of command sequences based on the tensor decomposition parameters, etc. Thus, the obtained optimized command sequences are executed according to an optimal optimization method corresponding to the target optimization parameters, and the finally obtained target command sequences can be deployed to terminal devices and have optimal performance, for example, having an optimal memory usage policy, reducing memory waste, having optimal parallel execution capabilities of operators, improving calculation efficiency, etc.

[0071] Each of the above embodiments of the present disclosure may be implemented alone, or may be implemented in any combination manner in a non-conflicting situation, and may be specifically set according to actual needs, and is not limited by the present disclosure.

[0072] The neural network model compiling method according to the embodiment of the present disclosure can be executed by any suitable device (e.g., a terminal device, a server, etc.) having data processing capability. Alternatively, any of the neural network model compiling methods according to the embodiment of the present disclosure can be executed by a processor, for example, the processor can execute any of the neural network model compiling methods mentioned in the embodiment of the present disclosure by calling a corresponding command stored in a memory. Hereinafter, the description thereof will be omitted.

[0073] 8 is a flowchart of a method for training a command sequence optimization model according to an exemplary embodiment of the present disclosure. This embodiment can be specifically applied to electronic devices such as servers and terminals, and as shown in FIG. 8, the method includes steps 401 and 402.

[0074] In step 401, a training intermediate command sequence corresponding to each of at least one neural network model and an optimal optimized parameter tag corresponding to each training intermediate command sequence are obtained.

[0075] Here, the method of acquiring the training intermediate command sequence is similar to that of the intermediate command sequence described above, so the description is omitted here. The optimal optimization parameter tag corresponding to the training intermediate command sequence is an optimization parameter corresponding to the verified optimal optimization method. The optimal optimization parameter tag can be acquired by adopting any feasible manner, for example, for any training intermediate command sequence, at least one training optimization command sequence corresponding to the training intermediate command sequence is determined based on a preset algorithm, performance corresponding to each training optimization command sequence is determined by any feasible manner, and the training optimization command sequence with the optimal performance is determined by performance comparison, and the optimization parameter corresponding to the training optimization command sequence with the optimal performance is set as the optimal optimization parameter tag corresponding to the training intermediate command sequence. Here, the performance determination can be, for example, by converting each training optimization command sequence into an executable training optimization command sequence, executing the training optimization command sequence on an actual terminal device, and determining its performance, or the performance of the training target command sequence can be determined based on at least one of a simulator, a performance prediction algorithm, etc. The simulator is for simulating the execution process of the training optimization command sequence on the terminal device (or hardware), and can obtain at least one of performance indicators such as execution time, memory transfer times, hardware usage rate for performing each calculation, etc., and can be specifically set according to actual needs. The performance of the training optimization command sequence is determined based on the obtained performance indicator. The performance prediction algorithm includes a performance indicator calculation formula (e.g., execution time calculation formula) for different operators, and can calculate the performance indicator of the training optimization command sequence based on the operator status included in the training optimization command sequence.

[0076] In step 402, a pre-constructed command sequence optimization network is trained based on each training intermediate command sequence and the optimal optimization parameter tag corresponding to each training intermediate command sequence to obtain a trained command sequence optimization model.

[0077] Here, the specific structure of the command sequence optimization network can be referred to the above embodiment, so the description is omitted here. During training, for any iterative iteration, the command sequence optimization network updated by the previous iteration is taken as the current command sequence optimization network, each training intermediate command sequence is processed using the current command sequence optimization network to obtain a prediction processing result, the current command sequence optimization network is updated according to the prediction processing result and the corresponding optimal optimization parameter tag to obtain an updated command sequence optimization network, if the updated command sequence optimization network satisfies the training termination condition, the updated command sequence optimization network is taken as the trained command sequence optimization model, if the training termination condition is not met, proceed to the next iteration, the updated command sequence optimization model is taken as the current command sequence optimization model, and the steps of processing each training intermediate command sequence using the current command sequence optimization network to obtain a prediction processing result are repeatedly performed until the training termination condition is met, to obtain a trained command sequence optimization model.

[0078] FIG. 9 is a flowchart of a method for training a command sequence optimization model according to another exemplary embodiment of the present disclosure.

[0079] In one alternative embodiment, step 401 of obtaining training intermediate command sequences corresponding to each of at least one neural network model and optimal optimized parameter tags corresponding to each training intermediate command sequence includes steps 4011 to 4015.

[0080] In step 4011, based on the at least one neural network model, a training intermediate command sequence corresponding to each of the neural network models is determined.

[0081] In step 4012, for any training intermediate command sequence, at least one training optimization command sequence corresponding to the training intermediate command sequence and training optimization parameters corresponding to each training optimization command sequence are determined based on a preset heuristic search algorithm.

[0082] Here, the preset heuristic search algorithm may be set according to actual needs and is not limited in the present disclosure. For example, the preset heuristic search algorithm may be an algorithm for determining a training optimization command sequence realized based on at least one of algorithms such as simulated annealing, genetic algorithm, ant colony optimization, etc. The training optimization parameters may be determined based on a specific optimization situation for the training intermediate command sequence of the training optimization command sequence. For example, the training optimization command sequence may determine a tensor decomposition parameter when performing a certain tensor decomposition on the training intermediate command sequence, and may determine an operator execution mode parameter corresponding to the operator when updating the execution mode of an operator, etc.

[0083] In step 4013, a performance result corresponding to each of the training optimization command sequences is determined based on preset performance determination rules.

[0084] Here, the preset performance decision rule may include at least one of the above-mentioned actual hardware execution, simulator, and performance prediction algorithm, and the specific description is omitted here because it is sufficient to refer to the above content. The performance index type of the performance result can be set according to actual needs, for example, the performance result may include at least one of the performance indexes such as execution time, memory transfer count, and utilization rate of each operator of the hardware operation.

[0085] In step 4014, a performance-optimized target training optimization command sequence is determined for each training optimization command sequence based on the performance results corresponding to each training optimization command sequence.

[0086] Here, the performance optimal decision rule can be set according to actual needs, for example, if the execution time is emphasized, the training optimization command sequence with the shortest execution time can be set as the target training optimization command sequence, if multiple performance indexes need to be paid attention to comprehensively, the multiple performance indexes can be weighted and added with a certain weight ratio to obtain a comprehensive index value, and the training optimization command sequence with the optimal comprehensive index value can be set as the target training optimization command sequence, etc. Specifically, it can be set according to actual needs.

[0087] In step 4015, the training optimization parameters corresponding to the target training optimization command sequence are set as optimal optimization parameter tags corresponding to the training intermediate command sequence.

[0088] In this embodiment, the performance corresponding to each training optimization command sequence is determined according to a certain performance determination rule, and the optimal optimization parameter tag corresponding to the training intermediate command sequence is easily determined, so that accurate and effective tag data can be provided for model training.

[0089] In one alternative embodiment, step 402 of training a pre-constructed command sequence optimization network based on each training intermediate command sequence and the optimal optimization parameter tag corresponding to each training intermediate command sequence to obtain a trained command sequence optimization model includes steps 4021 to 4026.

[0090] In step 4021, each training intermediate command sequence is processed using the command sequence optimization network to obtain a predicted processing result corresponding to each respective training intermediate command sequence.

[0091] The specific operation of this step is similar to the processing of the intermediate command sequence described above, so a description thereof will be omitted here.

[0092] In step 4022, a network loss is determined based on the prediction process results corresponding to each of the training intermediate command sequences, the optimal optimization parameter tags corresponding to each of the training intermediate command sequences, and a preset loss function.

[0093] Here, the preset loss function can be set according to actual needs, and can be at least one of loss functions such as an absolute error loss function, a regularized loss function, etc.

[0094] In one alternative embodiment, in each iteration process, after determining the network loss, judge whether the training termination condition is met based on the network loss (for example, whether the network loss meets the preset condition, whether the current iteration number reaches the preset number threshold, etc.). If the training termination condition is met, the training is terminated, and the command sequence optimization network is the trained command sequence optimization model; if the training termination condition is not met, further carry out the subsequent flow.

[0095] In step 4023, the network parameters of the command sequence optimization network are updated based on the network loss to obtain updated network parameters.

[0096] Here, the updates to the network parameters can be realized by employing any feasible gradient descent algorithm (eg, stochastic gradient descent algorithm, learning rate self-adaptive gradient descent algorithm, etc.).

[0097] In step 4024, an updated command sequence optimization network is determined based on the updated network parameters.

[0098] In step 4025, the updated command sequence optimization network is designated as the command sequence optimization network, and the process returns to step 4021.

[0099] In step 4026, the training is terminated depending on whether the training condition is satisfied, and a command sequence optimization model is obtained.

[0100] Here, step 4026 can be performed at any timing in the iterative process (eg, after step 4022), and can be specifically set according to actual needs.

[0101] In one alternative embodiment, determining 4022 a network loss based on the prediction process results corresponding to each of the training intermediate command sequences, the optimal optimization parameter tags corresponding to each of the training intermediate command sequences, and a preset loss function includes: The method includes determining an absolute error loss based on a prediction processing result corresponding to each of the training intermediate command sequences, an optimal optimization parameter tag corresponding to each of the training intermediate command sequences, and an absolute error loss function; determining a regularized loss based on current network parameters and a regularized loss function; and determining a network loss based on the absolute error loss and the regularized loss.

[0102] In one alternative example, the preset loss function can be expressed as:

number

number

number

number

[0103] where L1 represents the absolute error loss function, L2 represents the regularized loss function, and f w is a command sequence optimization network with learnable parameters w, N is the total number of data batches, i is the data batch number, and x i is the training intermediate command sequence of the i-th batch of data, and y i is the optimal optimization parameter tag for the i-th batch of data, and p i is the prediction result for the training intermediate command sequence of the i-th batch of data of the command sequence optimization network, and |y i -p i| is y i -p i represents the absolute value of |w|, and |w| represents the absolute value of w.

[0104] In one alternative embodiment, when updating network parameters, determine the gradient of the learnable parameter w in the command sequence optimization network based on the loss function, and use an optimizer to update the parameter w based on the gradient of the parameter w. The optimizer can be any gradient descent optimizer or a gradientless optimizer, taking a random gradient descent optimizer as an example, and the update formula of the learnable parameter is w'=w-ηg, where w' represents the updated parameter, g represents the gradient, and η represents the learning rate (also called "learning rate"), and the learning rate can be set according to actual needs, for example, it can be set to 0.1.

[0105] FIG. 10 is a flowchart of a method for training a command sequence optimization model according to a further exemplary embodiment of the present disclosure.

[0106] In one alternative embodiment, the method of the present disclosure further includes a step 510 of obtaining training memory information corresponding to each respective training intermediate command sequence.

[0107] Here, the method of acquiring the training memory information is similar to the above-mentioned current memory information sequence, so the description is omitted here.

[0108] Step 402 of training a pre-constructed command sequence optimization network based on each training intermediate command sequence and an optimal optimization parameter tag corresponding to each training intermediate command sequence to obtain a trained command sequence optimization model includes step 402a of training a pre-constructed command sequence optimization network based on each training intermediate command sequence, training memory information corresponding to each training intermediate command sequence and an optimal optimization parameter tag to obtain a trained command sequence optimization model.

[0109] In this embodiment, the network structure of the command sequence optimization network can be seen from the network structure shown in FIG. 7, and therefore a detailed description thereof will be omitted here.

[0110] In one alternative example, Figure 11 is a schematic diagram of a network structure of a command sequence optimization model according to a further exemplary embodiment of the present disclosure. In this example, the first feature extraction network adopts a multi-layer recurrent neural network (RNN), the second feature extraction network adopts a two-layer recurrent neural network, and the prediction head network adopts a multi-layer fully connected network. The structure of the feature fusion network can be referred to the above content, so the description will be omitted here.

[0111] In some optional embodiments of the present disclosure, during the training process and application process of the command sequence optimization model, a textual method can be adopted to display the intermediate command sequence (or the training intermediate command sequence) and the current memory information sequence (or the training memory information).

[0112] Each of the above embodiments of the present disclosure may be implemented alone or in any combination manner as long as there is no conflict, and may be specifically set according to actual needs, and is not limited by the present disclosure.

[0113] The method for training a command sequence optimization model according to the embodiment of the present disclosure can be executed by any suitable device (e.g., a terminal device, a server, etc.) having data processing capabilities. Alternatively, any of the methods for training a command sequence optimization model according to the embodiment of the present disclosure can be executed by a processor, for example, the processor can execute any of the methods for training a command sequence optimization model mentioned in the embodiment of the present disclosure by calling up corresponding commands stored in a memory. Hereinafter, the description thereof will be omitted.

[0114] Exemplary Apparatus 12 is a structural schematic diagram of a neural network model compiling device according to an exemplary embodiment of the present disclosure. The device of this embodiment is for implementing the corresponding neural network model compiling method of the present disclosure. The device shown in FIG. 12 includes: a first acquisition module 601 for acquiring a neural network model to be compiled; a first processing module 602 for determining, based on the neural network model to be compiled, an intermediate command sequence corresponding to the neural network model to be compiled; a second processing module 603 for processing the intermediate command sequence using the command sequence optimization model obtained by pre-training to obtain target optimization parameters corresponding to the intermediate command sequence; a third processing module 604 for determining an optimized command sequence corresponding to the intermediate command sequence based on the target optimization parameters; and a fourth processing module 605 for converting the optimized command sequence into an executable command sequence to obtain a target command sequence executable by a neural network processor corresponding to the neural network model to be compiled.

[0115] FIG. 13 is a structural schematic diagram of a neural network model compiling apparatus according to another exemplary embodiment of the present disclosure.

[0116] In one alternative embodiment, the second processing module 603 is a first processing unit 6031a for performing feature extraction on the intermediate command sequence using a first feature extraction network in the command sequence optimization model to obtain command sequence features; a second processing unit 6032 for processing the command sequence features using a predictive head network in the command sequence optimization model to obtain a processing result including an optimized parameter prediction probability corresponding to the intermediate command sequence; and a third processing unit 6033 for determining a target optimization parameter based on the processing result and the preset mapping rule.

[0117] FIG. 14 is a structural schematic diagram of a neural network model compilation device according to a further exemplary embodiment of the present disclosure.

[0118] In one alternative embodiment, the apparatus of the present disclosure further comprises a third acquiring module 701 for acquiring a current memory information sequence.

[0119] The second processing module 603 is a fourth processing unit 6031b for performing feature extraction on the current memory information sequence using a second feature extraction network in the command sequence optimization model to obtain memory information features; It further comprises a fusing unit 6031c for fusing the command sequence feature and the memory information feature to obtain a fused feature.

[0120] The second processing unit 6032 is also for processing the fused features using the predictive head network to obtain a processing result.

[0121] In one optional embodiment, the reconciliation unit 6031c is for aggregating the command sequence features according to a first aggregation rule to obtain a first aggregate feature, aggregating the memory information features according to a second aggregation rule to obtain a second aggregate feature, making the second aggregate feature and the first aggregate feature the same dimension in at least one direction, and concatenating the first aggregate feature and the second aggregate feature to obtain a fusion feature.

[0122] In one alternative embodiment, the target optimization parameters include memory management parameters, operator execution mode parameters and tensor decomposition parameters, and the third processing module 604 includes an optimization unit 6041 for optimizing the intermediate command sequence to obtain an optimized command sequence based on the memory management parameters, the operator execution mode parameters and the tensor decomposition parameters included in the target optimization parameters.

[0123] The beneficial technical effects corresponding to the exemplary embodiments of the present apparatus may be referred to the beneficial technical effects corresponding to the exemplary method portion above, and therefore will not be described here.

[0124] 15 is a structural schematic diagram of a training device for a command sequence optimization model according to an exemplary embodiment of the present disclosure. The device of this embodiment is for implementing an embodiment of a training method for a command sequence optimization model according to the present disclosure. The device shown in FIG. 15 includes: a second obtaining module 801 for obtaining training intermediate command sequences corresponding to each of the at least one neural network model and optimal optimization parameter tags corresponding to each of the training intermediate command sequences; and a fifth processing module 802 for training a pre-constructed command sequence optimization network based on each training intermediate command sequence and the optimal optimization parameter tag corresponding to each training intermediate command sequence to obtain a trained command sequence optimization model.

[0125] FIG. 16 is a structural schematic diagram of a training device for a command sequence optimization model according to another exemplary embodiment of the present disclosure.

[0126] In one alternative embodiment, the second acquisition module 801 is A first determining unit 8011 for determining, based on the at least one neural network model, a training intermediate command sequence corresponding to each of the neural network models; a second determining unit 8012 for determining, for any training intermediate command sequence, at least one training optimization command sequence corresponding to the training intermediate command sequence and a training optimization parameter corresponding to each training optimization command sequence based on a preset heuristic search algorithm; a third determination unit 8013 for determining a performance result corresponding to each of the training optimization command sequences according to a preset performance determination rule; a fourth determining unit 8014 for determining an optimal target training optimization command sequence of performance for each training optimization command sequence based on the performance result corresponding to each training optimization command sequence; and a fifth determining unit 8015 for determining the training optimization parameter corresponding to the target training optimization command sequence as an optimal optimization parameter tag corresponding to the training intermediate command sequence.

[0127] In one alternative embodiment, the fifth processing module 802 is a fifth processing unit 8021 for processing each training intermediate command sequence using the command sequence optimization network to obtain a predicted processing result corresponding to each training intermediate command sequence; A sixth processing unit 8022 for determining a network loss based on the prediction processing result corresponding to each of the training intermediate command sequences, the optimal optimization parameter tag corresponding to each of the training intermediate command sequences, and the preset loss function; a seventh processing unit 8023 for updating network parameters of the command sequence optimization network based on the network loss to obtain updated network parameters; an eighth processing unit 8024 for determining an updated command sequence optimized network based on the updated network parameters; a ninth processing unit 8025 for obtaining the updated command sequence optimization network as a command sequence optimization network and transmitting it to the fifth processing unit 8021, so as to repeatedly perform the step of processing each training intermediate command sequence using the command sequence optimization network to obtain a prediction processing result corresponding to each training intermediate command sequence; and a tenth processing unit 8026 for terminating the training according to satisfying the training condition and obtaining a command sequence optimization model.

[0128] In one alternative embodiment, the sixth processing unit 8022 is The present invention is for determining an absolute error loss based on a prediction processing result corresponding to each training intermediate command sequence, an optimal optimization parameter tag corresponding to each training intermediate command sequence, and an absolute error loss function, determining a normalized loss based on a current network parameter and a normalized loss function, and determining a network loss based on the absolute error loss and the normalized loss.

[0129] FIG. 17 is a structural schematic diagram of a training device for a command sequence optimization model according to a further exemplary embodiment of the present disclosure.

[0130] In one alternative embodiment, the apparatus of the present disclosure further comprises a fourth acquiring module 910 for acquiring training memory information corresponding to each respective training intermediate command sequence.

[0131] The fifth processing module 802 is for training a pre-constructed command sequence optimization network based on each training intermediate command sequence, the training memory information corresponding to each training intermediate command sequence, and the optimal optimization parameter tag to obtain a trained command sequence optimization model.

[0132] The beneficial technical effects corresponding to the exemplary embodiments of the present apparatus may be referred to the beneficial technical effects corresponding to the exemplary method portion above, and therefore will not be described here.

[0133] Exemplary Electronic Devices FIG. 18 is a structural diagram of an electronic device according to an embodiment of the present disclosure, which includes at least one processor 11 and a memory 12.

[0134] The processor 11 may be a central processing unit (CPU) or other type of processing unit having data processing and / or command execution capabilities, and may control other components within the electronic device 10 to perform desired functions.

[0135] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, etc. The computer-readable storage media may store one or more computer program commands, and the processor 11 may execute the one or more computer program commands to realize the methods and / or other expected functions in each embodiment of the present disclosure described above.

[0136] In one example, electronic device 10 may further include input device(s) 13 and output device(s) 14 connected to each other via a bus system and / or other form of connection mechanism (not shown).

[0137] The input device 13 may include, for example, a keyboard and a mouse.

[0138] The output device 14 can output various information to the outside, and can include, for example, a display, a speaker, a printer, a communication network, and remote output devices to which they are connected.

[0139] Of course, for convenience of explanation, Fig. 18 shows only some of the components related to the present disclosure in the electronic device 10, and components such as a bus and an input / output interface are omitted. In addition, the electronic device 10 may further include any other appropriate components depending on a specific application situation.

[0140] Exemplary Computer Program Products and Computer-Readable Storage Media In addition to the methods and apparatus described above, embodiments of the present disclosure further provide a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform steps in the methods of the various embodiments of the present disclosure described in the "Exemplary Methods" section above.

[0141] The computer program product may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and general procedural programming languages ​​such as "C" or similar programming languages, with program code for carrying out operations of the disclosed embodiments. The program code may be executed as follows: entirely on the user computing device, partially on the user device, as a separate software package, partly on the user computing device and partly on a remote computing device, or entirely on a remote computing device or server.

[0142] Additionally, embodiments of the present disclosure further provide a computer-readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform steps in the methods of various embodiments of the present disclosure described in the "Exemplary Methods" section above.

[0143] Any combination of one or more readable media can be employed as the computer readable storage medium. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0144] Although the basic principles of the present disclosure have been described above with reference to specific examples, it should be noted that the benefits, advantages, effects, etc. mentioned in the present disclosure are merely illustrative and not limiting, and various examples of the present disclosure do not necessarily have them. In addition, the specific details of the above disclosure are merely illustrative and easily understandable actions, and are not limiting, and the above details do not necessarily limit the present disclosure to be realized by the above specific details.

[0145] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. obtaining a neural network model to be compiled; determining, based on the neural network model to be compiled, an intermediate command sequence corresponding to the neural network model to be compiled; processing the intermediate command sequence using a command sequence optimization model obtained by pre-training to obtain target optimization parameters corresponding to the intermediate command sequence; determining an optimized command sequence corresponding to the intermediate command sequence based on the target optimization parameters; converting the optimized command sequence into an executable command sequence to obtain a target command sequence executable by a neural network processor corresponding to the neural network model being compiled; A method for compiling a neural network model, comprising:

2. The step of processing the intermediate command sequence using a command sequence optimization model obtained by training in advance to obtain a target optimization parameter corresponding to the intermediate command sequence includes: performing feature extraction on the intermediate command sequence using a first feature extraction network in the command sequence optimization model to obtain command sequence features; processing the command sequence features using a predictive head network in the command sequence optimization model to obtain a processing result including optimized parameter prediction probabilities corresponding to the intermediate command sequence; determining the target optimization parameters based on the processing result and a preset mapping rule; 2. The method of claim 1, wherein the neural network model is compiled using the neural network model generation method.

3. The compiling method further includes obtaining a current memory information sequence; The step of processing the intermediate command sequence using a command sequence optimization model obtained by training in advance to obtain a target optimization parameter corresponding to the intermediate command sequence includes: performing feature extraction on the current memory information sequence using a second feature extraction network in the command sequence optimization model to obtain memory information features; fusing the command sequence feature and the memory information feature to obtain a fusion feature; The step of processing the command sequence features using a predictive head network in the command sequence optimization model to obtain a processing result includes: processing the fused features using the predictive head network to obtain the processing result; 3. The method for compiling a neural network model according to claim 2.

4. The step of fusing the command sequence feature and the memory information feature to obtain a fusion feature includes: aggregating the command sequence features according to a first aggregation rule to obtain a first aggregate feature; aggregating the memory information features according to a second aggregation rule to obtain a second aggregate feature; making the second set of features and the first set of features dimensionally identical in at least one direction; and concatenating the first set of features with the second set of features to obtain the fusion feature.

4. The method for compiling a neural network model according to claim 3.

5. The goal optimization parameters include memory management parameters, operator execution mode parameters, and tensor decomposition parameters; determining an optimized command sequence corresponding to the intermediate command sequence based on the target optimization parameters, optimizing the intermediate command sequence based on the memory management parameters, the operator execution mode parameters, and the tensor decomposition parameters to obtain the optimized command sequence.

2. The method of claim 1, wherein the neural network model is compiled using the neural network model generation method.

6. obtaining training intermediate command sequences corresponding to each of at least one neural network model and optimal optimized parameter tags corresponding to each of said training intermediate command sequences; training a pre-constructed command sequence optimization network based on each of the training intermediate command sequences and the best optimization parameter tags corresponding to each of the training intermediate command sequences to obtain a trained command sequence optimization model; 13. A method for training a command sequence optimization model, comprising:

7. training a pre-constructed command sequence optimization network based on each of the training intermediate command sequences and the best optimization parameter tags corresponding to each of the training intermediate command sequences to obtain a trained command sequence optimization model, processing each of the training intermediate command sequences using the command sequence optimization network to obtain a predicted processing result corresponding to each of the training intermediate command sequences; determining a network loss based on the prediction process results corresponding to each of the training intermediate command sequences, the optimal optimization parameter tags corresponding to each of the training intermediate command sequences, and a preset loss function; updating network parameters of the command sequence optimization network based on the network losses to obtain updated network parameters; determining an updated command sequence optimization network based on the updated network parameters; the updated command sequence optimization network is the command sequence optimization network, so as to repeatedly perform the step of processing each of the training intermediate command sequences using the command sequence optimization network to obtain a predicted processing result corresponding to each of the training intermediate command sequences; and terminating the training in response to satisfying a training condition to obtain the command sequence optimization model.

7. The method of training a command sequence optimization model according to claim 6.

8. determining a network loss based on the prediction process results corresponding to each of the training intermediate command sequences, the optimal optimization parameter tags corresponding to each of the training intermediate command sequences, and a preset loss function, determining an absolute error loss based on the prediction process results corresponding to each of the training intermediate command sequences, the best optimized parameter tags corresponding to each of the training intermediate command sequences, and an absolute error loss function; determining a regularized loss based on the current network parameters and a regularized loss function; determining the network loss based on the absolute error loss and the normalized loss.

8. The method for training a command sequence optimization model according to claim 7.

9. The step of obtaining training intermediate command sequences corresponding to each of at least one neural network model and optimal optimized parameter tags corresponding to each of said training intermediate command sequences includes: determining, based on at least one neural network model, a training intermediate command sequence corresponding to each of the neural network models; determining, for any one of the training intermediate command sequences, at least one training optimization command sequence corresponding to the training intermediate command sequence and a training optimization parameter corresponding to each of the training optimization command sequences based on a preset heuristic search algorithm; determining a performance result corresponding to each of said training optimization command sequences based on preset performance determination rules; determining a performance-optimized target training optimization command sequence for each of the training optimization command sequences based on performance results corresponding to each of the training optimization command sequences; and determining a training optimization parameter corresponding to the target training optimization command sequence as an optimal optimization parameter tag corresponding to the training intermediate command sequence.

7. The method of training a command sequence optimization model according to claim 6.

10. The training method further includes acquiring training memory information corresponding to each of the training intermediate command sequences; training a pre-constructed command sequence optimization network based on each of the training intermediate command sequences and the best optimization parameter tags corresponding to each of the training intermediate command sequences to obtain a trained command sequence optimization model, training a pre-constructed command sequence optimization network based on each of the training intermediate command sequences, training memory information corresponding to each of the training intermediate command sequences, and the optimal optimization parameter tag to obtain a trained command sequence optimization model; 7. The method of training a command sequence optimization model according to claim 6.

11. a first acquisition module for acquiring a neural network model to be compiled; a first processing module for determining, based on the neural network model to be compiled, an intermediate command sequence corresponding to the neural network model to be compiled; a second processing module for processing the intermediate command sequence using a command sequence optimization model obtained by pre-training to obtain target optimization parameters corresponding to the intermediate command sequence; a third processing module for determining an optimized command sequence corresponding to the intermediate command sequence based on the target optimization parameters; a fourth processing module for converting the optimized command sequence into an executable command sequence to obtain a target command sequence executable by a neural network processor corresponding to the neural network model to be compiled. A neural network model compiling device comprising:

12. a second acquisition module for acquiring training intermediate command sequences corresponding to each of the at least one neural network model and optimal optimization parameter tags corresponding to each of the training intermediate command sequences; and a fifth processing module for training a pre-constructed command sequence optimization network based on each of the training intermediate command sequences and the best optimization parameter tags corresponding to each of the training intermediate command sequences to obtain a trained command sequence optimization model. A command sequence optimization model training device comprising:

13. A computer-readable storage medium having a computer program stored thereon, The computer program executes the neural network model compilation method according to any one of claims 1 to 5. A computer-readable storage medium comprising:

14. A computer-readable storage medium on which a computer program is stored, comprising: The computer program executes the method for training a command sequence optimization model according to any one of claims 6 to 10. A computer-readable storage medium comprising:

15. A processor; a memory for storing commands executable by the processor; The processor reads the executable commands from the memory and executes them to implement the neural network model compilation method according to any one of claims 1 to 5.

1. An electronic device comprising:

16. A processor; a memory for storing commands executable by the processor; The processor reads the executable commands from the memory and executes them to realize the method for training a command sequence optimization model according to any one of claims 6 to 10.

1. An electronic device comprising:

Images