Training of operator optimization scheduling model, and operator optimization scheduling

Abstract

Disclosed are a method and apparatus for training an operator optimization scheduling model, and an operator optimization scheduling method, a storage medium and an electronic device. The method for training an operator optimization scheduling model comprises: acquiring a deep learning model from a training data set, and using same as a training sample; inputting the training sample into an operator optimization scheduling model to be trained; for each of a plurality of time slices involved in training, determining information of operators of the training sample at a decision-making moment of the time slice; on the basis of the information, determining an operator to be optimized that corresponds to the time slice; optimizing said operator, so as to obtain a reduced running time of said operator; and then, on the basis of the information, said operator and the reduced running time of said operator, training said operator optimization scheduling model.

Description

Training of operator optimization scheduling models and operator optimization scheduling Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a training method and device for an operator optimization scheduling model, a method for operator optimization scheduling, a storage medium, and an electronic device. Background Art

[0002] With the continuous development of science and technology, the application of deep learning models is becoming more and more widespread. Typically, deep learning models need to be trained in a model framework. After the deep learning model training is completed, it is deployed on hardware devices to run the deep learning model to execute business operations.

[0003] At present, when deploying a deep learning model on a hardware device, the operators in the deep learning model can be optimized first, and then the optimized deep learning model can be deployed on the hardware device to speed up the model deployment, so that the execution speed of the business can be accelerated when the deep learning model is subsequently run to execute the business. For example, when deploying an image classification model on a hardware device, the operators in the image classification model can be optimized first, and then the optimized image classification model can be deployed on the hardware device to speed up the deployment of the image classification model, so that when the image classification model is subsequently used to classify images, the classification speed of the image classification model can be accelerated. Therefore, how to train an operator optimization scheduling model to optimize the scheduling of operators in a deep learning model is an important issue. Based on this, the present disclosure provides a training method for an operator optimization scheduling model.

[0004] Summary of the Invention

[0005] The present disclosure provides a training method and apparatus for an operator optimization scheduling model, a method for operator optimization scheduling, a storage medium, and an electronic device to partially address the aforementioned problems existing in the prior art. The present disclosure employs the following technical solutions.

[0006] In a first aspect, the present disclosure provides a training method for an operator optimization scheduling model, comprising: obtaining a deep learning model in a training data set as a training sample; inputting the training sample into the operator optimization scheduling model to be trained; for each time slice in a plurality of time slices included in the training, determining information of each operator in the training sample at the decision moment of the time slice, wherein the information at least includes the number of floating-point operations of each operator, the number of historical optimizations of each operator before the decision moment, and the running down time of each operator before the decision moment; determining the operator to be optimized corresponding to the time slice based on the information of each operator at the decision moment of the time slice; optimizing the operator to be optimized in the time slice to obtain the running down time of the operator to be optimized in the time slice; and training the operator optimization scheduling model to be trained using a policy gradient algorithm based on the information of each operator, the operator to be optimized corresponding to each time slice, and the running down time of the operator to be optimized in each time slice.

[0007] Optionally, optimizing the operator to be optimized in the time slice to obtain the running decrease time of the operator to be optimized in the time slice includes: taking the running time of the operator to be optimized before the operator to be optimized is optimized as the historical running time; taking the running time of the operator to be optimized after the operator to be optimized is optimized as the post-optimization running time; taking the difference between the historical running time and the post-optimization running time as the running decrease time of the operator to be optimized.

[0008] Optionally, if an operator is optimized multiple times before the decision moment, the running degradation time of the operator before the decision moment is the difference between the running time of the operator before optimization and the running time after optimization in the target time slice, where the target time slice is the time slice closest to the time slice corresponding to the decision moment and in which the operator is optimized.

[0009] Optionally, if an operator has not been optimized before the decision moment, the running fall time of the operator before the decision moment is the initial value.

[0010] Optionally, the information also includes historical optimization ratios of the operators before the decision moment.

[0011] Optionally, the information of each operator in the training sample at the decision moment of the time slice is determined, including: if the time slice is not the first time slice of the target training sample, the information of each operator in the target training sample at the decision moment of the time slice is updated based on the operator to be optimized trained in the previous time slice of the time slice and the running decline time of the operator to be optimized, wherein the target training sample is one of the training samples.

[0012] Optionally, the training of the operator optimization scheduling model to be trained also includes: determining whether the training end conditions are met; wherein the training end conditions include one or more of the following: the training time reaches a preset time, the training rounds reach a preset number, and the training of the operator optimization scheduling model to be trained meets the convergence requirements, wherein a round includes one or more time slices.

[0013] Optionally, in response to the number of training samples being more than one, the training of the operator optimization scheduling model to be trained includes: for each training sample in the training samples, according to the information of each operator of the training sample, the operator to be optimized corresponding to each time slice included in the training sample, and the running down time of the operator to be optimized in each time slice, using the policy gradient algorithm to train the operator optimization scheduling model to be trained.

[0014] In a second aspect, the present disclosure provides a method for operator optimization scheduling, comprising: inputting a deep learning model to be deployed into a trained operator optimization scheduling model, wherein the trained operator optimization scheduling model is trained by the method of the first aspect above; determining information of each operator in the deep learning model to be deployed at the decision moment of the current time slice, wherein the deep learning model to be deployed is a trained model, and the information includes at least the number of floating-point operations of each operator, the number of historical optimizations of each operator before the decision moment, and the running decline time of each operator before the decision moment; according to the information at the decision moment of the current time slice, determining the operator to be optimized corresponding to the current time slice, optimizing the operator to be optimized in the current time slice, and obtaining the optimization result of the operator to be optimized; judging whether the optimization end condition is met; if the optimization end condition is met, generating an executable file of the deep learning model to be deployed running on the target hardware according to the situation of each operator after the optimization is completed.

[0015] Optionally, the method also includes: if the optimization end condition is not met, taking the next time slice as the new current time slice, repeating the operations of determining the information of each operator in the deep learning model to be deployed at the decision moment of the current time slice, determining the operator to be optimized corresponding to the current time slice, optimizing the operator to be optimized in the current time slice to obtain the optimization result of the operator to be optimized, and judging whether the optimization end condition is met.

[0016] Optionally, the target hardware includes one or more of the following: a hardware device including an image processing unit; a hardware device including a central processing unit; a hardware device including a digital signal processor.

[0017] In a third aspect, the present disclosure provides a training device for an operator optimization scheduling model, comprising: an acquisition module for acquiring a deep learning model in a training data set as a training sample; an input module for inputting the training sample into the operator optimization scheduling model to be trained; a first determination module for determining, for each of a plurality of time slices included in the training, information of each operator in the training sample at the decision moment of the time slice, wherein the information includes at least the number of floating-point operations of each operator, the number of historical optimizations of each operator before the decision moment, and the running down time of each operator before the decision moment; a scheduling module for determining the operator to be optimized corresponding to the time slice based on the information of each operator at the decision moment of the time slice; a second determination module for optimizing the operator to be optimized in the time slice to obtain the running down time of the operator to be optimized in the time slice; a training module for training the operator optimization scheduling model to be trained using a policy gradient algorithm based on the information of each operator, the operator to be optimized corresponding to each time slice, and the running down time of the operator to be optimized in each time slice.

[0018] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the training method of the operator optimization scheduling model of the first aspect or the operator optimization scheduling method of the second aspect.

[0019] In the fifth aspect, the present disclosure provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the training method of the operator optimization scheduling model of the first aspect or the operator optimization scheduling method of the second aspect is implemented.

[0020] As can be seen from the above method, when training the operator optimization scheduling model, the present disclosure first obtains training samples from the training data set, then inputs the training samples into the operator optimization scheduling model to be trained. For each time slice, the information of each operator in the training sample at the decision moment of the time slice is determined, and the operator to be optimized corresponding to the time slice is determined based on the information of each operator at the decision moment of the time slice; and the operator to be optimized is optimized in the time slice to obtain the running decrease time of the operator to be optimized in the time slice. Afterwards, based on the information of each operator, the operator to be optimized corresponding to each time slice, and the running decrease time of the operator to be optimized in each time slice, the policy gradient algorithm is used to train the operator optimization scheduling model to be trained. Through the reinforcement learning training method, the degree of reduction in the running time of the operator to be optimized after each time slice is optimized is quantified, so that the trained operator optimization scheduling model can continuously adjust the behavior of the object based on feedback, thereby achieving efficient optimization of the operator and improving the applicability to various hardware platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0022] FIG1 is a flow chart of a training method for an operator optimization scheduling model provided by the present disclosure;

[0023] FIG2 is a schematic diagram of a process of operator optimization scheduling provided by the present disclosure;

[0024] FIG3 is a schematic diagram of an operator optimization scheduling provided by the present disclosure;

[0025] FIG4 is a schematic diagram of a computational graph of a deep learning model to be deployed provided by the present disclosure;

[0026] FIG5 is a schematic diagram of a training device structure of an operator optimization scheduling model provided by the present disclosure;

[0027] FIG6 is a schematic structural diagram of an electronic device provided by the present disclosure. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and advantages of this specification more clear, the following will clearly and completely describe the technical solutions of this specification in conjunction with the specific embodiments of this specification and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this specification.

[0029] The technical solutions provided by various embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0030] FIG1 is a flow chart of a training method for an operator optimization scheduling model provided in the present disclosure, which includes the following steps.

[0031] S100: Obtain a deep learning model in a training dataset as a training sample.

[0032] In the present disclosure, a device for training an operator optimization scheduling model can obtain a model in a training data set and use it as a training sample. The device for training the operator optimization scheduling model can be a server or an electronic device such as a desktop computer, a laptop computer, or a terminal. For ease of description, the following describes the training method of the operator optimization scheduling model provided by the present disclosure using only the server as the execution entity. The training data set can be a pre-built data set containing a deep learning model. The deep learning model contained in the training data set can be a common model, such as a deep residual network (ResNet), SqueezeNet, etc., or a customized deep learning model, which is not specifically limited in the present disclosure. The deep learning model contained in the training data set can include an image classification model pre-trained based on image data. The image data can be an image pre-collected by an image acquisition device such as a camera or a still camera, or an image in any existing general image data set, which is not specifically limited in the present disclosure. The image classification model can include a variety of models trained based on image data. The multiple image classification models can include models of multiple different architectures, each model containing different numbers of parameters and parameter values.

[0033] The training data set may also include a text processing model trained based on text data. The text data may be pre-collected text data such as movie reviews, human-computer dialogues, poems, etc. The text processing model may include sentiment classification, topic classification, and text generation models trained based on the above text data. This disclosure does not make specific limitations.

[0034] The training dataset may also include other types of deep learning models, which are not specifically limited in this disclosure. During training, the deep learning model obtained from the training dataset in the same episode is the same deep learning model, and the deep learning models obtained in different episodes may be the same or different.

[0035] S102: Input the training samples into the operator optimization scheduling model to be trained.

[0036] S104: For each of the multiple time slices included in the training, determine the information of each operator in the training sample at the decision moment of the time slice, wherein the information at least includes the number of floating-point operations of each operator, the number of historical optimizations of each operator before the decision moment, and the respective running down time of each operator before the decision moment.

[0037] When training an operator optimization scheduling model to be trained, the training time can be divided into multiple time slices. For example, the number of time slices can be preset. For another example, after the operator optimization scheduling model to be trained meets convergence requirements, training can be stopped, and the total number of time slices elapsed during training can be counted.

[0038] For each time slice, the server can determine the decision moment for that time slice and the information of each operator in the training sample at that decision moment. The information of each operator includes at least the number of floating-point operations performed by each operator, the number of historical optimizations performed by each operator, and the run-down time of each operator after optimization prior to that moment. The information of each operator can also include the type of operator, which can include various types of operators, such as data acquisition, statistics, logical operations, arithmetic operations, and format conversion. The historical optimization count of each operator is the number of times each operator has been optimized prior to that moment. The server can count the number of times each operator has been optimized prior to that moment.

[0039] A corresponding decision moment can be set before each time slice. For example, assuming a time slice is 5 minutes long, the decision moment occurs at minute 0, minute 5, minute 10, and so on. The server determines the information of each operator at the decision moment. Information such as the number of floating-point operations performed by each operator can be considered static information, that is, it does not change as the time slice increases. Information such as the historical optimization count of each operator can be considered dynamic information, that is, it may change as the time slice increases. The server will update this dynamic information at each decision moment.

[0040] In the present disclosure, after the operator is optimized, the operator's running time for the model will decrease. Exemplarily, the operator is optimized by adjusting the operator's calculation method / the scheduling method of the hardware (such as the data storage location related to the operator, the allocation of threads, the splitting of dimensions, the access method of registers, etc.), thereby improving the operator's calculation speed, so that the optimized operator in the model, such as when performing image classification on image data, will speed up the calculation speed of the operator and reduce the operator's running time. Therefore, for each operator, if the operator is optimized once at a certain decision moment (such as the current moment), the running time of the operator before the current moment can be determined, and the running time of the operator after the optimization before the current moment can be determined, and then the difference between the determined running time before the optimization and the determined running time after the optimization is subtracted from the determined running time is used as the running reduction time of the operator after the optimization before the current moment. Exemplarily, for an image classification model, the above-determined unoptimized running time may be the running time of the operator when the image data is input into the image classification model to perform image classification, and the operator in the image classification model for the image data input is not optimized; the above-determined optimized running time may be the running time of the operator when the image data is input into the image classification model to perform image classification, and the operator in the image classification model for the image data input is optimized once.

[0041] The aforementioned unoptimized runtime and optimized runtime can be pre-set times, or can be determined based on historical optimization data generated when the server optimized the model used as a training sample in a time slice prior to the current time slice. Historical optimization data is data generated when the model was optimized in a previous time slice or time slices. This historical optimization data can include the runtime of the entire model before optimization, the runtime of the entire model after optimization, the runtime of each operator in the optimized model, and the runtime of each operator after optimization. Therefore, based on the historical optimization data of the model, the server can determine the runtime of the operator before optimization and the runtime of the operator after optimization.

[0042] For example, the running time of a certain operator before optimization is 0.5s, and the running time after optimization is 0.3s before the moment. Therefore, the running time of the operator after optimization is reduced to 0.2s before the moment.

[0043] Of course, there may be operators that have not been optimized before a certain decision moment. Therefore, the run-down time of such operators that have never been optimized before this moment can be set to an initial value, such as 0. In addition, there may be an operator that has been optimized multiple times before a certain decision moment. Therefore, when determining the run-down time of such an operator that has been optimized multiple times before this moment, the target time slice for optimizing the operator closest to the decision moment can be determined. The difference between the run time of the operator before optimization and the run time after optimization in the target time slice can be used as the run-down time of the operator.

[0044] In the present disclosure, there are many ways to optimize operators. The optimization method for each operator can be the same or different. The optimization method corresponding to the operator can be determined according to the type of operator, and the present disclosure does not make specific restrictions. In addition, for each operator, the optimization method used for the operator is different, and the running time of the optimized operator may be the same or different, and the running down time of the operator may be the same or different. Of course, since the type of each operator is different, for the same optimization method, the running time of each optimized operator may be different, and thus the running down time may also be different.

[0045] The specific implementations of operators (such as matrix multiplication and convolution) vary. For example, matrix multiplication can be written in a variety of ways, with common optimization techniques including loop blocking, loop unrolling, and parallelization. The optimal implementation is generally different for different hardware. Therefore, optimizing operators for different hardware involves adjusting the aforementioned optimization techniques and leveraging hardware characteristics, such as cache, to maximize operator performance.

[0046] In recent years, automatic operator optimization methods have also developed rapidly. For example, the AI ​​compiler TVM (Tensor Virtual Machine) has two tuning methods: AutoTVM and AutoScheduler. Other technical solutions exist for Halide and other platforms. However, their functions are similar: to efficiently implement automatic operator search.

[0047] The method proposed in the present disclosure can use existing automatic operator tuning solutions to optimize the operator to be optimized in each time slice, for example, AutoTVM can be used to optimize the operator.

[0048] Therefore, the server can pre-set each operator to be optimized using a different optimization method based on the optimization method and type of each operator, and calculate the runtime of each operator after optimization. Subsequently, the server can determine the runtime of each operator before optimization and the runtime of each operator after optimization based on the optimization method of each operator. Based on the two runtime times determined for each operator (i.e., the time required for the operator to perform the calculation / operation before optimization and the time required for the operator to perform the calculation / operation after optimization), the server can calculate the runtime reduction time for each operator.

[0049] S106: Determine the operator to be optimized corresponding to the time slice according to the information of each operator at the decision moment of the time slice.

[0050] S108: Optimize the operator to be optimized in the time slice to obtain the running down time of the operator to be optimized in the time slice.

[0051] The information of each operator in step S104 can be updated based on the operating decrease time of the operator to be optimized in the time slice obtained in step S108. If the current time slice and the next time slice are for the same training sample (which can be called the target training sample), the information of each operator of the target training sample can be updated at the decision moment of the next time slice based on the result obtained after optimizing the operator to be optimized in the current time slice, such as the operating decrease time of the operator to be optimized and the historical optimization count of the operator to be optimized.

[0052] S110: According to the information of each operator, the operator to be optimized corresponding to each time slice, and the running down time of the operator to be optimized in each time slice, the policy gradient algorithm is used to train the operator optimization scheduling model to be trained.

[0053] The trained operator optimization scheduling model can determine the operators that need to be optimized in the deep learning model to be deployed based on the information of each operator in the model, optimize the determined operators, determine the model to be deployed after the operator optimization, and deploy the optimized deep learning model on the target hardware to the target image.

[0054] It should be noted that in order to improve the versatility / portability of the trained operator optimization scheduling model, that is, to enable the deep learning model scheduled / compiled by the operator optimization scheduling model to be deployed on different hardware devices, such as hardware devices including GPUs (Graphics Processing Units) from different manufacturers, hardware devices including CPUs (Central Processing Units) from different manufacturers, hardware devices including DSPs (Digital Signal Processors) from different manufacturers, etc., the training method provided in the present disclosure is based on the idea of ​​reinforcement learning. For example, the information of each operator obtained before a certain time slice can be regarded as a state, and the operator to be optimized in the time slice determined can be regarded as an action, and the running down time of the operator to be optimized in the time slice can be regarded as a reward. When the operator optimization scheduling model takes a certain action in the current state, it will obtain certain feedback from the environment, and this feedback is the reward. This reinforcement learning method receives quantifiable feedback from the environment for each action (e.g., the degree of reduction in running time). By continuously adjusting the object's behavior based on the feedback, it can achieve efficient optimization of the operator and improve its applicability to various hardware platforms.

[0055] The server can first input information into the operator optimization scheduling model to be trained, and determine the operator to be optimized at each decision moment as the operator to be optimized. Thereafter, the run-down time of the operator to be optimized after optimization is determined. Then, based on the information, the operator to be optimized, and the run-down time of the operator to be optimized, the policy gradient algorithm is used to train the operator optimization scheduling model to be trained. Specifically, when determining the run-down time after optimization of the operator to be optimized, the server can determine the run time before the operator to be optimized is optimized as the historical run time, and determine the run time after the operator to be optimized is optimized as the post-optimization run time. Thereafter, the difference between the historical run time and the post-optimization run time is used as the run-down time after optimization of the operator to be optimized.

[0056] The initial runtime of the operator to be optimized can be a pre-set time. If the operator to be optimized has not been optimized, the initial runtime can be pre-set, or the runtime determined from the first run of the operator can be used as the initial runtime of the operator to be optimized. In this case, the initial runtime of the operator can be considered its historical runtime. If the operator to be optimized has been optimized before the current decision moment, the historical runtime of the operator to be optimized is the runtime of the operator after the most recent optimization before the current moment.

[0057] In some examples, when the number of times the operator optimization scheduling model to be trained has been trained reaches a preset number, the training of the operator optimization scheduling model to be trained is completed. The trained operator optimization scheduling model can be used to determine the operators that need to be optimized in the deep learning model to be deployed based on the information of each operator in the deep learning model to be deployed. The server can then optimize the determined operators, determine the model to be deployed after the operators are optimized, and deploy the optimized deep learning model to be deployed on the target hardware. Among them, the deep learning model to be deployed can be an image classification model trained based on image data, or a model sent by the user, and the present disclosure does not limit this. Exemplarily, the server can determine the image classification model to be deployed in response to the user's input operation, and the image classification model can be an image classification model that the user has pre-trained based on image data.

[0058] In some examples, a deep learning model can be selected from the training data set and input into the operator optimization scheduling model for training. The training is performed for N time slices to obtain information about each operator of the deep learning model corresponding to the decision moments of the N time slices, the operators to be optimized corresponding to the N time slices, and the running down time of each operator to be optimized in the N time slices. Based on this information, the policy gradient algorithm is used to train the operator optimization scheduling model to be trained to update the parameters of the operator optimization scheduling model to be trained. Then, another deep learning model can be selected from the training data set and input into the operator optimization scheduling model for training. The training is performed for M time slices, and so on, until the training time reaches a preset time (equivalent to the training time slice reaching a preset number of time slices), or the training rounds reach a preset number (one round can include one or more time slices), or the operator optimization scheduling model to be trained meets the convergence requirements, and the training is stopped. Wherein, N and M are integers greater than or equal to 1.

[0059] It can be seen from the above method that when the present invention trains the operator optimization scheduling model, the server can obtain the training samples in the training data set, input the training samples into the operator optimization scheduling model to be trained, and then determine the information of each operator in the training sample at each decision moment, and determine the operator to be optimized corresponding to the current time slice based on the information of each operator at the current decision moment. Afterwards, the running down time of the operator to be optimized after the operator to be optimized is determined. Then, based on the information, the operator to be optimized and the running down time of the operator to be optimized, the policy gradient algorithm is used to train the operator optimization scheduling model to be trained. When optimizing the operator of the deep learning model to be deployed, the operator scheduled for optimization at the current decision moment can be determined by the trained operator optimization scheduling model, and the scheduled operator can be optimized, thereby reducing the requirements for the operator's prior knowledge of the strategy for manually designing and selecting the operator to be optimized, and improving the operator optimization effect of the deep learning model to be deployed.

[0060] In practical applications, the time available to optimize a neural network is typically limited, and each operator has different optimization potential. This optimization potential refers to the operator's run-down time after optimization. For a particular operator, even if only that operator is optimized within the optimization time, its run-down time may still be relatively short, indicating poor optimization results and low optimization potential. However, for another operator, optimizing only that operator within the optimization time may result in a longer run-down time, indicating better optimization results and greater optimization potential. Even if only a portion of the optimization time is used to optimize that operator, its run-down time may be significantly longer. The optimization time can be pre-divided into time slices of a specified length. For each time slice, the operator to be optimized within that time slice is determined and optimized within that time slice.

[0061] In step S104, the server can determine information about each operator in the training sample before the start of the current time slice and use it as input information for the current time slice. The current time slice is a time slice of a specified length starting at the current time. The input information includes at least the number of floating-point operations performed by each operator, the number of historical optimizations performed by each operator, and the run-down time of each operator after optimization prior to the current time slice. The input information can also include the type of each operator, which can include various types such as data collection, statistics, and format conversion. The input information can also include the historical optimization ratio of each operator. Specifically, for each operator, the server can determine the number of optimizations performed by that operator in other time slices prior to the current time slice as the historical optimization count. The historical optimization ratio of that operator is then determined based on the operator's historical optimization count and the number of time slices prior to the current time slice. For example, if operator 1 was optimized three times prior to the current time slice and there were 10 time slices prior to the current time slice, the historical optimization ratio of operator 1 is 0.3. That is to say, the information of each operator in step S104 may also include the historical optimization ratio of each operator before the decision moment.

[0062] Exemplarily, when determining the run-down time of each operator after optimization before the decision moment of the current time slice, for each operator, if the operator had not been optimized before the current time slice, the run-down time of the operator before the current time slice can be set to 0. If the operator had been optimized before the current time slice, one or more time slices in which the operator was historically optimized are determined, and from the determined time slices, the time slice closest to the current time slice is determined as the historical time slice. Thereafter, the run-time of the operator at the start time of the historical time slice is determined, as well as the run-time of the operator after optimization in the historical time slice. Based on the two determined run-times, the run-down time of the operator after optimization before the decision moment of the current time slice is calculated.

[0063] Based on this, in step S106 above, the server can input the input information corresponding to the current time slice into the operator optimization scheduling model to be trained, and determine the operator to be optimized in the current time slice as the operator to be optimized. In step S108 above, the server can determine the running time of the operator to be optimized before the current time slice as the historical running time, and determine the running time of the operator to be optimized after the operator to be optimized is optimized in the current time slice as the post-optimization running time. Then, the difference between the historical running time and the post-optimization running time is used as the running down time of the operator to be optimized after the operator to be optimized is optimized in the current time slice.

[0064] Among them, since the current decision moment is the starting moment of the current time slice, the historical running time is the running time of the operator to be optimized at the starting moment. When the operator to be optimized has never been optimized before the current time slice, the historical running time can be the initial running time of the operator to be optimized. When the operator to be optimized has been optimized before the current time slice, that is, the operator to be optimized has been optimized in the time slice before the current time slice, then the time slice before the current time slice and closest to the current time slice is determined, and the operator to be optimized has been optimized in the determined time slice, so the historical running time is the running time of the operator to be optimized after the operator to be optimized has been optimized in the determined time slice.

[0065] Because time slices are pre-divided into optimization time slices, there may or may not be more time slices after the current time slice. That is, the optimization time may or may not be reached after the current time slice ends. Therefore, in step S110, the server can determine whether the training termination condition is met. If so, the policy gradient algorithm is used to train the operator optimization scheduling model to be trained based on the input information corresponding to each time slice, the operator to be optimized, and the run-down time of the operator to be optimized. If not, the next time slice after the current time slice is reset as the current time slice. The information of each operator in the training samples before the start of the current time slice is continuously determined as the input information for the current time slice. The operator to be optimized and the run-down time of the operator to be optimized are re-determined until the training termination condition is met. The training termination condition can be that there are no more time slices after the current time slice, that is, the optimization time is reached after the end of the current time slice. The next time slice after the current time slice is a time slice of a specified length starting at the end of the current time slice. Each time slice is a time slice obtained by dividing the preset optimization time into the specified length.

[0066] In the present disclosure, if there are other time slices after the current time slice, it means that the optimization time has not yet been reached, and the operator to be optimized can continue to be determined and optimized. Therefore, the server can re-use the next time slice of the current time slice as the current time slice and execute according to the above steps S104 to S110 until the training end condition is met. If there are no other time slices after the current time slice, it means that the optimization time has been reached. The server can use the policy gradient algorithm to train the operator optimization scheduling model to be trained based on the input information corresponding to each time slice, the operator to be optimized, and the running down time of the operator to be optimized.

[0067] In some examples, an optimization time can be set for a training sample. After the optimization time for the training sample has expired, the policy gradient algorithm is used to train the operator optimization scheduling model to be trained. In some examples, an optimization time can be set for multiple training samples. After the optimization time for the multiple training samples has expired, the policy gradient algorithm is used to train the operator optimization scheduling model to be trained.

[0068] The present disclosure also provides an operator optimization scheduling method that utilizes a trained operator optimization scheduling model to optimize the model's operators during model deployment. As shown in Figure 2, Figure 2 is a schematic diagram of an operator optimization scheduling process provided in the present disclosure, specifically including the following steps.

[0069] S200: Inputting the deep learning model to be deployed into the trained operator optimization scheduling model. The trained operator optimization scheduling model is trained using the aforementioned method.

[0070] S202: Determine information about each operator in the deep learning model to be deployed at the decision moment of the current time slice, where the deep learning model to be deployed is a trained model, i.e., its architecture, parameters, operators, and other information are known. This information includes at least the number of floating-point operations performed by each operator, the number of historical optimizations performed by each operator before the decision moment, and the downtime of each operator before the decision moment.

[0071] The server can determine the information of each operator in the deep learning model to be deployed at the current decision moment. The information of each operator includes the number of floating-point operations of each operator, the number of historical optimizations of each operator, and the running down time after optimization of each operator before the current moment. The information of each operator can also include the type of each operator, which can include various types of operators such as data collection, statistics, and format conversion. The number of historical optimizations of each operator is the number of times each operator has been optimized before the current moment. The server can count the number of times each operator has been optimized before the current moment. Initially, the number of historical optimizations of each operator can be set to 0. The deep learning model to be deployed can be a deep learning model that the server has pre-trained based on data, or it can be a model sent by the user. For example, the server can determine the deep learning model to be deployed in response to the user's input operation. The model can be a model that the user has pre-trained based on data, and this disclosure does not make specific limitations.

[0072] For the deep learning model to be deployed, you can set its optimization time. At the same time, you can divide the optimization time into multiple time slices. It should be noted that when optimizing the deep learning model, the optimization time and the divided time slices can be different from or the same as those set when training the operator optimization scheduling model. This disclosure does not limit the setting of the optimization time and time slice size.

[0073] In the present disclosure, since the running time of an operator will decrease after the operator is optimized, for each operator at the current decision moment, the running time of the operator before the moment can be determined, and the running time of the operator after the optimization before the moment can be determined. The difference between the determined running time of the operator before the optimization and the determined running time after the optimization is used as the running reduction time of the operator after the optimization before the moment.

[0074] Of course, there may be cases where an operator has never been optimized before this moment, so the run-down time of such an operator that has never been optimized before this moment can be set to 0. In addition, there may be cases where an operator has been optimized multiple times before this moment, so when determining the run-down time of such an operator that has been optimized multiple times before this moment, the time slice in which the operator was optimized closest to this moment is determined, and the run-down time of this operator is determined based on the run time of the operator before and after optimization within this determined time slice.

[0075] In the present disclosure, there are many ways to optimize operators. The optimization method for each operator can be the same or different. The optimization method corresponding to the operator can be determined according to the type of operator, and the present disclosure does not make specific restrictions. In addition, for each operator, the optimization method used for the operator is different, and the running time of the operator after optimization may be the same or different, resulting in the running time of the operator being the same or different. Of course, since the type of each operator is different, for the same optimization method, the running time of each optimized operator may be different, and thus the running time may also be different.

[0076] S204: Determine the operator to be optimized corresponding to the current time slice according to the decision moment information of the current time slice, optimize the operator to be optimized in the current time slice, and obtain an optimization result of the operator to be optimized.

[0077] S206: Determine whether the optimization end condition is met. If the optimization end condition is met, jump to step S210, otherwise jump to step 208.

[0078] S208: Use the next time slice as the new current time slice and repeat steps S202-S206.

[0079] The server can input the information determined at the decision moment of the current time slice into the trained operator optimization scheduling model. Based on this information, the operator optimization scheduling model determines the operator to be optimized in the current time slice as the operator to be optimized. The server then optimizes the operator to be optimized within the current time slice to obtain the corresponding optimization result. It then determines whether the operator optimization termination condition has been met. The operator optimization termination condition can be when the optimization time allocated to the model has expired. Since the optimization time for deep learning models cannot be infinite, a pre-set optimization time can be used to optimize each operator within this time.

[0080] The server can input information into the trained operator optimization scheduling model to determine the operator to be optimized for the current time slice, which serves as the current operator to be optimized. It then determines the optimization method corresponding to the current operator to be optimized and optimizes the current operator using the determined optimization method until the optimization is completed or the allocated optimization time is reached.

[0081] Based on this, the server can set multiple time slices according to the optimization time. For the i-th time slice, the operator optimization scheduling model determines the operator to be optimized corresponding to the i-th time slice based on the information of each operator obtained after the optimization of the i-1-th time slice, and optimizes the operator. When the optimization of the operator to be optimized corresponding to the i-th time slice is completed, it is determined whether there is an i+1-th time slice. If there is an i+1-th time slice, the information of each operator is updated. Based on the updated information, the operator to be optimized corresponding to the i+1-th time slice is determined and optimized. The operator to be optimized corresponding to the i-th time slice can be the same as or different from the operator to be optimized corresponding to the i+1-th time slice. And so on, until the optimization end condition is met, such as when the optimization time is reached, the deep learning model will no longer be updated.

[0082] S210: Generate an executable file of the deep learning model to be deployed that runs on the target hardware based on the status of each operator after the optimization is completed.

[0083] The server can generate an executable file for the deep learning model to be deployed based on the final optimization results of each operator after optimization. However, after operator optimization, some operators in the model to be deployed may not be optimized, while others may have been optimized multiple times. Therefore, the operators after optimization may include operators that have never been optimized.

[0084] Because the server first optimizes the operators in the model to be deployed to improve the running time of the operators, and then generates an executable file based on each operator after the optimization is completed, and deploys the model to be deployed to the hardware based on the executable file, the computing speed of the model is improved.

[0085] When deploying the image classification model to be deployed based on the operator optimization scheduling method shown in Figure 2 above, the trained operator optimization scheduling model is used to determine the corresponding operator to be optimized from the operators in the deep learning model to be deployed at the decision moment of each time slice, thereby reducing the requirement for the operator's prior knowledge in manually designing the strategy for selecting the operator to be optimized, improving the overall operator optimization effect of the image classification model to be deployed, and thus improving the running speed of the deep learning model after actual deployment.

[0086] In the above step S202, the server can determine the information of each operator in the neural network model to be deployed before the start of the current time slice, and use it as the input information corresponding to the current time slice, wherein the current time slice is a time slice of a specified length with the current moment (that is, the decision moment of the aforementioned current time slice) as the starting moment. The input information includes at least the number of floating-point operations of each operator, the number of historical optimizations of each operator, and the running down time of each operator after optimizing each operator before the current moment. The input information can also include the type of each operator, which can include various types of operators such as data collection, statistics, and format conversion. The input information can also include the historical optimization ratio of each operator. The specific determination process of the above input information is shown in the content of the above step S104, which will not be repeated here.

[0087] Based on this, in the above step S204, the server may input the input information corresponding to the current time slice into the trained operator optimization scheduling model to determine the operator to be optimized in the current time slice as the operator to be optimized.

[0088] Since the time slice is obtained by dividing the optimization time in advance, there may or may not be a time slice after the current time slice, that is, the optimization time may or may not be reached after the end of the current time slice. The server can optimize the operator to be optimized until the end of the current time slice. Determine whether the optimization end condition is met. If so, determine that the optimization of each operator is finished. If not, use the next time slice of the current time slice as the current time slice, continue to determine the information of each operator before the start of the current time slice, and re-determine the operator to be optimized, and optimize the re-determined operator to be optimized until the optimization end condition is met. Among them, the next time slice of the current time slice is a time slice of the specified length with the end time of the current time slice as the starting time. The optimization end condition is that there is no other time slice after the current time slice, that is, the optimization time is reached after the end of the current time slice.

[0089] For example, as shown in FIG3 , FIG3 is a schematic diagram of an operator optimization scheduling provided in the present disclosure. Assume that after the optimization time is divided, six time slices are obtained, namely time slices 1 to 6. The current time slice is time slice 1. There are three operators in the model to be deployed, namely operators 1 to 3. The operator to be optimized in the current time slice is operator 1. The server optimizes operator 1 in time slice 1 until the end of time slice 1. It is determined whether the optimization end condition is met. Obviously, it is not met. The next time slice after time slice 1, namely time slice 2, is re-set as the current time slice. The information of each operator before the start of time slice 2 is further determined, and the operator to be optimized is re-determined, assuming it is operator 2. The re-determined operator 2 is optimized until the end of time slice 2. The subsequent process of determining the operators to be optimized in other time periods is not further described in this example. The operators to be optimized corresponding to time slices 1 to 6 shown in FIG3 are operator 1, operator 2, operator 3, operator 2, operator 2, and operator 1, respectively.

[0090] In order to speed up the deployment of the model, the computational graph (also called high-level intermediate representation) of the model to be deployed can be determined first. For example, as shown in FIG4 , FIG4 is a schematic diagram of a computational graph of a deep learning model to be deployed provided in the present disclosure. In FIG4 , there are three operators in the model to be deployed, namely operators 1 to 3. The edges between the operators in FIG4 represent dependency relationships. The edge between operator 1 and operator 2 represents that the execution of operator 2 depends on operator 1. The meanings of the other edges in FIG4 are similar and will not be repeated in this example. Afterwards, the computational graph of the model to be deployed is optimized at the graph level, and then the optimized computational graph is optimized at the operator level. Therefore, before the above-mentioned step S200, the server can determine the first computational graph of the model to be deployed, perform graph optimization on the first computational graph, and obtain the second computational graph. Afterwards, the second computational graph is input into the operator optimization scheduling model, and the information of each operator input in step S202 may include the information of each operator in the second computational graph.

[0091] In the present disclosure, since some operators may appear multiple times in a computation graph, that is, there may be multiple operators in the computation graph, such as the computation graph shown in FIG4 , in which operator 2 appears twice. Therefore, when determining the run-down time of an operator that appears multiple times in the computation graph, the run-down time of the operator determined based on the original run-down time of the operator before optimization (i.e., historical run time) and the run-down time after optimization (i.e., post-optimization run time) is expanded by the operator's number of times, and the expanded time is used as the run-down time of the operator, where the number of times the operator appears in the computation graph. For example, the run-down time of operator 2 determined based on the run-down time before optimization and the run-down time after optimization is 0.2s, but there are two operators 2 in the computation graph, so the actual run-down time of operator 2 is 0.4s.

[0092] The above are one or more implementation methods of the present disclosure. Based on the same idea, the present disclosure also provides a training device for a corresponding operator optimization scheduling model, as shown in FIG5 .

[0093] FIG5 is a schematic diagram of a training device for an operator optimization scheduling model provided by the present disclosure, which includes the following modules.

[0094] The acquisition module 300 is used to obtain the deep learning model in the training data set as a training sample.

[0095] The input module 302 is used to input the training samples into the operator optimization scheduling model to be trained.

[0096] The first determination module 304 is used to determine, for each time slice among the multiple time slices included in the training, information of each operator in the training sample at the decision moment of the time slice, wherein the information at least includes the number of floating-point operations of each operator, the number of historical optimizations of each operator before the decision moment, and the running down time of each operator before the decision moment.

[0097] The scheduling module 306 is configured to determine the operator to be optimized corresponding to the time slice according to the information of each operator at the decision moment of the time slice.

[0098] The second determining module 308 is configured to optimize the operator to be optimized in the time slice to obtain the running down time of the operator to be optimized in the time slice.

[0099] The training module 310 is used to train the operator optimization scheduling model to be trained using a policy gradient algorithm based on the information of each operator, the operator to be optimized corresponding to each time slice, and the running down time of the operator to be optimized in each time slice.

[0100] Optionally, the second determination module 308 is specifically used to use the running time of the operator to be optimized before the operator to be optimized is optimized as the historical running time; use the running time of the operator to be optimized after the operator to be optimized is optimized as the post-optimization running time; and use the difference between the historical running time and the post-optimization running time as the running decrease time of the operator to be optimized.

[0101] Alternatively, if an operator has been optimized multiple times before the decision moment, the operator's runtime reduction time before the decision moment is the difference between the operator's runtime before and after optimization in a target time slice, where the target time slice is the time slice closest to the time slice corresponding to the decision moment and in which the operator was optimized. Alternatively, if an operator has not been optimized before the decision moment, the operator's runtime reduction time before the decision moment is the initial value.

[0102] Optionally, the information also includes historical optimization ratios of the operators before the decision moment.

[0103] The first determination module 304 is specifically used to update the information of each operator in the target training sample at the decision moment of the time slice based on the operator to be optimized trained in the time slice before the time slice and the running decline time of the operator to be optimized, if the time slice is not the first time slice of the target training sample, wherein the target training sample is one of the training samples.

[0104] Optionally, the training module 310 is also used to determine whether the optimization end conditions are met; wherein the training end conditions include one or more of the following: the training time reaches a preset time, the training rounds reach a preset number, and the operator optimization scheduling model to be trained training meets the convergence requirements, wherein a round includes one or more time slices.

[0105] Optionally, the training module 310 is also used to, in response to the number of the training samples being more than one, use the policy gradient algorithm to train the operator optimization scheduling model to be trained for each training sample in the training samples based on the information of each operator of the training sample, the operator to be optimized corresponding to each time slice included in the training sample, and the running down time of the operator to be optimized in each time slice.

[0106] The operator optimization scheduling model obtained after training by the training device of the above-mentioned operator optimization scheduling model can be used to optimize the scheduling / compilation of the deep learning model to be deployed. Specifically, the deep learning model to be deployed is input into the trained operator optimization scheduling model, and the trained operator optimization scheduling model is used to determine the information of each operator in the deep learning model to be deployed at the decision moment of the current time slice, wherein the deep learning model to be deployed is a trained model, and the information at least includes the number of floating-point operations of each operator, the number of historical optimizations of each operator before the decision moment, and the running down time of each operator before the decision moment; according to the information at the decision moment of the current time slice, the operator to be optimized corresponding to the current time slice is determined, and the operator to be optimized is optimized in the current time slice to obtain the optimization result of the operator to be optimized; it is judged whether the optimization end condition is met; if the optimization end condition is met, according to the situation of each operator after the optimization, an executable file of the deep learning model to be deployed running on the target hardware is generated.

[0107] Optionally, the trained operator optimization scheduling model is also used to, if the optimization end condition is not met, use the next time slice as the new current time slice, repeat the operations of determining the information of each operator in the deep learning model to be deployed at the decision moment of the current time slice, determining the operator to be optimized corresponding to the current time slice, optimizing the operator to be optimized in the current time slice to obtain the optimization result of the operator to be optimized, and judging whether the optimization end condition is met.

[0108] Optionally, determining the information of each operator in the deep learning model to be deployed at the decision moment of the current time slice includes: determining a first computational graph of the deep learning model to be deployed; performing graph optimization on the first computational graph to obtain a second computational graph; and determining the information of each operator in the second computational graph at the decision moment of the current time slice.

[0109] Optionally, the target hardware includes one or more of the following: a hardware device including an image processing unit; a hardware device including a central processing unit; a hardware device including a digital signal processor.

[0110] The present disclosure also provides a computer-readable storage medium storing a computer program that can be used to execute the training method for an operator optimization scheduling model provided in FIG1 or the operator optimization scheduling method provided in FIG2. The computer-readable storage medium can be a non-volatile storage medium.

[0111] This disclosure also provides a schematic structural diagram of an electronic device, shown in Figure 6. As shown in Figure 6, at the hardware level, the electronic device includes a processor, memory, and non-volatile memory. Of course, it may also include hardware required for other services, such as network interfaces and internal buses. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the training method of the operator optimization scheduling model described in Figure 1 or the operator optimization scheduling method provided in Figure 2.

[0112] Of course, in addition to software implementation, this disclosure does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0113] In the 1990s, technological improvements could be clearly distinguished as either hardware improvements (for example, improvements to circuit structures like diodes, transistors, and switches) or software improvements (improvements to process flows). However, with the advancement of technology, many process flow improvements today can now be considered direct improvements to hardware circuit structures. Designers almost always create the corresponding hardware circuit structure by programming the improved process flow into the hardware circuit. Therefore, it cannot be said that a process flow improvement cannot be implemented using hardware modules. For example, a programmable logic device (PLD), such as a field programmable gate array (FPGA), is an integrated circuit whose logical function is determined by user programming. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to hire a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly done using "logic compiler" software. This is similar to the software compiler used when developing programs. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There is not just one HDL, but many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art will also understand that by simply programming the method flow in one of these hardware description languages ​​and then programming it into an integrated circuit, a hardware circuit that implements the logic method flow can be easily obtained.

[0114] The controller can be implemented in any suitable manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in a purely computer-readable program code format, the controller can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules that implement the method and structures within the hardware component.

[0115] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0116] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0117] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0119] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0121] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0122] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0123] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0125] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present disclosure may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0127] The various embodiments of this disclosure are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so their description is relatively simple. For relevant portions, refer to the description of the method embodiments.

[0128] The foregoing is merely an embodiment of the present disclosure and is not intended to limit the present disclosure. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure are intended to be included within the scope of the claims of the present disclosure.

Claims

1. A training method for an operator optimization scheduling model, characterized in that: include: Obtain the deep learning model in the training data set as a training sample; Inputting the training samples into the operator optimization scheduling model to be trained; For each of the multiple time slices included in the training, Determine information of each operator in the training sample at the decision moment of the time slice, wherein the information at least includes the number of floating-point operations of each operator, the number of historical optimizations of each operator before the decision moment, and the running fall time of each operator before the decision moment; Determine the operator to be optimized corresponding to the time slice according to the information of each operator at the decision moment of the time slice; Optimizing the operator to be optimized in the time slice to obtain the running fall time of the operator to be optimized in the time slice; According to the information of each operator, the operator to be optimized corresponding to each time slice and the running fall time of the operator to be optimized in each time slice, the policy gradient algorithm is adopted to train the operator optimization scheduling model to be trained.

2. The method according to claim 1, characterized in that Optimizing the operator to be optimized in the time slice to obtain the running fall time of the operator to be optimized in the time slice includes: The running time of the operator to be optimized before the operator to be optimized is optimized is used as the historical running time; The running time of the operator to be optimized after the operator to be optimized is used as the running time after optimization; The difference between the historical running time and the optimized running time is taken as the running reduction time of the operator to be optimized.

3. The method according to claim 1, characterized in that If an operator is optimized multiple times before the decision moment, the running time of the operator before the decision moment is the difference between the running time of the operator before optimization and the running time after optimization in the target time slice, wherein the target time slice is the time slice closest to the time slice corresponding to the decision moment and in which the operator is optimized.

4. The method according to claim 1, characterized in that If an operator has not been optimized before the decision time, the running fall time of the operator before the decision time is the initial value.

5. The method according to claim 1, characterized in that The information also includes the historical optimization ratios of the operators before the decision moment.

6. The method according to claim 1, characterized in that The information of each operator in the training sample for determining the decision moment of the time slice includes: If the time slice is not the first time slice of the target training sample, training is performed based on the previous time slice of the time slice. The operator to be optimized and the running fall time of the operator to be optimized are trained, and the information of each operator in the target training sample at the decision moment of the time slice is updated, wherein the target training sample is one of the training samples.

7. The method according to claim 1, characterized in that The training of the operator optimization scheduling model to be trained further includes: Determine whether the training end conditions are met; Among them, the training end conditions include one or more of the following: the training time reaches a preset time, the training rounds reach a preset number, and the operator optimization scheduling model to be trained meets the convergence requirements, wherein a round includes one or more time slices.

8. The method according to claim 1, characterized in that In response to the number of the training samples being greater than one, the training of the operator optimization scheduling model to be trained includes: For each training sample in the training samples, According to the information of each operator of the training sample, the operator to be optimized corresponding to each time slice included in the training sample, and the running decline time of the operator to be optimized in each time slice, the policy gradient algorithm is adopted to train the operator optimization scheduling model to be trained.

9. A method for operator optimization scheduling, characterized in that: include: Inputting the deep learning model to be deployed into the trained operator optimization scheduling model, wherein the trained operator optimization scheduling model is trained by the method described in any one of claims 1 to 8; Determine information of each operator in the deep learning model to be deployed at the decision moment of the current time slice, wherein the deep learning model to be deployed is a trained model, and the information includes at least the number of floating-point operations of each operator, the number of historical optimizations of each operator before the decision moment, and the running decline time of each operator before the decision moment; Determine the operator to be optimized corresponding to the current time slice according to the information of the decision moment of the current time slice, optimize the operator to be optimized in the current time slice, and obtain an optimization result of the operator to be optimized; Determine whether the optimization end condition is met; If the optimization end condition is met, an executable file of the deep learning model to be deployed and run on the target hardware is generated according to the conditions of each operator after the optimization is completed.

10. The method according to claim 9, characterized in that The method further comprises: If the optimization end condition is not met, the next time slice is used as the new current time slice, and the operations of determining the information of each operator in the deep learning model to be deployed at the decision moment of the current time slice, determining the operator to be optimized corresponding to the current time slice, optimizing the operator to be optimized in the current time slice to obtain the optimization result of the operator to be optimized, and judging whether the optimization end condition is met are repeated.

11. The method according to claim 9, characterized in that The information of each operator in the deep learning model to be deployed at the decision moment of the current time slice includes: Determining a first computational graph of the deep learning model to be deployed; Performing graph optimization on the first computation graph to obtain a second computation graph; Determine information of each operator in the second computation graph at a decision moment of the current time slice.

12. The method according to claim 9, characterized in that The target hardware includes one or more of the following: A hardware device including an image processing unit; Hardware devices including central processing units; A hardware device including a digital signal processor.

13. A training device for an operator optimization scheduling model, characterized in that: include: An acquisition module is used to acquire the deep learning model in the training data set as a training sample; An input module, used for inputting the training samples into the operator optimization scheduling model to be trained; A first determination module is used to determine, for each time slice in a plurality of time slices included in the training, information of each operator in the training sample at a decision moment of the time slice, wherein the information at least includes the number of floating-point operations of each operator, the number of historical optimizations of each operator before the decision moment, and the running fall time of each operator before the decision moment; A scheduling module, used to determine the operator to be optimized corresponding to the time slice according to the information of each operator at the decision moment of the time slice; A second determination module is used to optimize the operator to be optimized in the time slice to obtain the running fall time of the operator to be optimized in the time slice; The training module is used to train the operator optimization scheduling model to be trained by using a policy gradient algorithm according to the information of each operator, the operator to be optimized corresponding to each time slice, and the running decline time of the operator to be optimized in each time slice.

14. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 8 or 9 to 12 is implemented.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method described in any one of claims 1 to 8 or 9 to 12 is implemented.

Images