A task execution method and device, a storage medium and an electronic device

Abstract

The specification discloses a task execution method, device, storage medium and electronic equipment. The task execution method comprises: obtaining a target model and each candidate training framework according to a first task instruction, keeping at least one of specified parameters involved when the target model is trained by different candidate training frameworks, operators called by different candidate training frameworks and a dependency relationship between the operators, and an updating manner when the target model is updated by different candidate training frameworks equivalent as a target, obtaining each adjusted framework, determining an operation time length when an operation operation of the target model based on the adjusted framework is executed by a terminal device deploying the target model, determining a priority corresponding to the adjusted framework according to the operation time length, determining a target training framework from each candidate training framework according to the priority of each adjusted framework, and executing a model training task through the target training framework when a second task instruction is received.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a task execution method, apparatus, storage medium, and electronic device. Background Technology

[0002] In recent years, deep learning has been widely used in various fields such as image recognition, natural language processing, and information recommendation. In order to efficiently build and train deep learning models, a training framework that is easy to use and can complete the model training task in a reasonable time is indispensable.

[0003] Among them, the emergence of deep learning frameworks provides users with a platform that shields them from the underlying computing environment. On this platform, users only need to focus on the construction of the model and do not need to care about how the model is computed in the underlying hardware, thus relieving the burden on users when training models.

[0004] However, there are currently a large number of training frameworks on the market, making it difficult for users to intuitively judge the differences and advantages and disadvantages between different training frameworks. Furthermore, since there is no mature method for evaluating different training frameworks, it is difficult for users to choose a training framework that meets their expectations from the many available options.

[0005] Therefore, how to accurately evaluate different training frameworks so that users can choose the training framework that matches their expectations based on the evaluation results is an urgent problem to be solved. Summary of the Invention

[0006] This specification provides a task execution method, apparatus, storage medium, and electronic device to partially solve the aforementioned problems existing in the prior art.

[0007] The following technical solution is adopted in this specification:

[0008] This manual provides a task execution method, including:

[0009] Receive the first task instruction;

[0010] According to the first task instruction, obtain the target model and each candidate training framework;

[0011] With the goal of maintaining equivalence among at least one of the specified parameters involved in training the target model with different candidate training frameworks, the operators called by different candidate training frameworks and the dependencies between operators, and the update methods of different candidate training frameworks when updating the target model, the candidate training frameworks are adjusted to obtain each adjusted framework.

[0012] For each adjusted framework, the duration of time when the terminal device deploying the target model performs the computational operation of the target model based on the adjusted framework is determined as the computation duration;

[0013] Based on the computation time, determine the priority of the adjusted framework;

[0014] Based on the priority corresponding to each adjusted frame, the target training frame is determined from each candidate training frame;

[0015] When the second task instruction is received, the model training task for the target model is executed through the target training framework.

[0016] Optionally, the specified parameters include at least one of the following: hyperparameters involved in preprocessing the input data of the target model, hyperparameters corresponding to the operators called by each candidate training framework, hyperparameters involved in updating the weights of the target model, and hyperparameters related to training performance.

[0017] Optionally, the update method includes at least one of the following: a transformation function applied to the gradient, a weight update function, and a regularization function.

[0018] Optionally, before adjusting each candidate training framework to obtain each adjusted framework, the method further includes:

[0019] Determine whether the operators called by each candidate training framework and the dependencies between them are the same;

[0020] If not, the candidate training frameworks are adjusted with the goal of maintaining equivalence between the operators called by each candidate training framework and the dependencies between them.

[0021] Optionally, determining whether the operators invoked by each candidate training framework and the dependencies between them are the same includes:

[0022] The target model is deployed in each candidate training framework. The same data is input and the same parameters are set for the target model in each candidate training framework. It is determined whether the target model produces the same output in each candidate training framework. If so, it is determined that the operators called by each candidate training framework and the dependencies between each operator are the same.

[0023] Optionally, the same data is input and the same parameters are set for the target model under each candidate training framework, and it is determined whether the target model produces the same output in each candidate training framework, specifically including:

[0024] One of the candidate training frameworks is selected to train the target model;

[0025] After at least one iteration, the model parameters of the target model are exported and converted into a specified parameter format and loaded into the target model of other candidate training frameworks;

[0026] Export the target model in each candidate training framework to a specified model format, input the same data into each target model in the specified model format, and determine whether the target models in each specified model format produce the same output.

[0027] Optionally, for each adjusted framework, the duration for a terminal device deploying the target model to perform computational operations on the target model based on the adjusted framework is determined as the computation duration, specifically including:

[0028] The operating status of the terminal device is sampled according to a preset sampling period;

[0029] The duration during which a terminal device deploying the target model performs computational operations on the target model based on the adjusted framework is determined within the sampling period is defined as the computation duration.

[0030] Optionally, the priority of the adjusted framework is determined based on the computation time, specifically including:

[0031] Based on the computation time, determine the device utilization rate and computational efficiency of the adjusted framework for the terminal device;

[0032] The priority is determined based on the equipment utilization rate and the computing efficiency.

[0033] Optionally, the device utilization rate is determined based on the computation time, specifically including:

[0034] The device utilization rate is determined based on the computation time and the sampling time corresponding to the sampling period, wherein the device utilization rate is positively correlated with the computation time.

[0035] Optionally, the longer the computation time, the lower the computation efficiency.

[0036] Optionally, the training framework includes a deep learning framework.

[0037] This specification provides a task execution device, including:

[0038] The receiving module receives the first task instruction;

[0039] The acquisition module acquires the target model and each candidate training framework according to the first task instruction;

[0040] The adjustment module aims to maintain equivalence among at least one of the specified parameters involved in training the target model using different candidate training frameworks, the operators called by different candidate training frameworks and the dependencies between operators, and the update methods of different candidate training frameworks when updating the target model, thereby adjusting each candidate training framework to obtain each adjusted framework.

[0041] The first determining module determines, for each adjusted framework, the duration of time when the terminal device deploying the target model performs the calculation operation of the target model based on the adjusted framework, as the calculation duration;

[0042] The second determining module determines the priority of the adjusted framework based on the computation time.

[0043] The third determination module determines the target training framework from the candidate training frameworks based on the priority of each adjusted framework.

[0044] When the execution module receives the second task instruction, it executes the model training task for the target model through the target training framework.

[0045] Optionally, the first determining module is specifically used to sample the operating status of the terminal device according to a preset sampling period; and determine the duration of the terminal device deploying the target model within the sampling period when it performs the operation of the target model based on the adjusted framework, as the operation duration.

[0046] Optionally, the second determining module is specifically used to: determine the device utilization rate and computing efficiency of the adjusted framework for the terminal device based on the computing time; and determine the priority based on the device utilization rate and the computing efficiency.

[0047] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described task execution method.

[0048] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described task execution method.

[0049] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0050] In the task execution method provided in this specification, according to the first task instruction, the target model and each candidate training framework are obtained. The goal is to maintain equivalence among at least one of the following: the specified parameters involved in training the target model using different candidate training frameworks, the operators called by different candidate training frameworks and the dependencies between operators, and the update methods used by different candidate training frameworks to update the target model. Adjusted frameworks are then obtained. The computation time for the terminal device deploying the target model to perform calculations based on the adjusted framework is determined. The priority of each adjusted framework is determined based on the computation time. The target training framework is then determined from among the candidate training frameworks based on the priority of each adjusted framework. When the second task instruction is received, the model training task is executed through the target training framework.

[0051] As can be seen from the above method, this specification can determine the priority of each training framework while ensuring that at least one of the specified parameters, invoked operators, dependencies between operators, and update methods of the target model are kept equivalent when different frameworks train the target model. Based on this priority, the target training framework is selected to train the model. Compared to existing methods, this scheme can evaluate the underlying logic of different training frameworks while controlling multiple variables, thereby selecting the target training framework that meets the expectations to execute the model training task based on the evaluation results. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0053] Figure 1 This is a flowchart illustrating one task execution method provided in this specification;

[0054] Figure 2 This is a schematic diagram illustrating the iterative process of one of the models provided in this specification;

[0055] Figure 3 This is a schematic diagram of an evaluation method for a training framework provided in this specification;

[0056] Figure 4 This is a schematic diagram of a task execution device provided in this specification;

[0057] Figure 5 This specification provides a corresponding Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0059] Existing solutions typically compare the merits of different training frameworks based on metrics such as training time, data throughput, and the quality (accuracy) of the trained models. However, these methods fail to effectively control variables during the evaluation process, resulting in inaccurate final evaluation results.

[0060] Furthermore, since many factors influence the aforementioned evaluation dimensions, using these dimensions is too macro-level and makes it difficult to analyze a specific technology or implementation of the training framework. It also provides limited insight into the algorithms and technologies used by the training framework. Because these evaluation dimensions primarily focus on application or hardware information, they cannot intuitively demonstrate the merits of a training framework.

[0061] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0062] Figure 1 This is a flowchart illustrating a task execution method provided in this specification, including the following steps:

[0063] S101: Receive the first task instruction.

[0064] S102: According to the first task instruction, obtain the target model and each candidate training framework.

[0065] Generally, model training can be viewed as an iterative process of multiple training epochs, where the entire training dataset is used once in each epoch. Each epoch then randomly divides the dataset into equal portions, each called a mini-batch. Each mini-batch undergoes a complete training process, called a training step. Therefore, multiple training steps are repeated within a single epoch until all mini-batches have undergone one training cycle.

[0066] In each training step, the training samples first undergo preprocessing, such as data augmentation and normalization. The preprocessed data is then used as input to the model for a forward propagation computation, calculating the loss function. Since model training involves continuously adjusting model parameters to minimize the loss function, the backpropagation process calculates the gradients of the model parameters with respect to the loss function according to the chain rule. Finally, the model parameters are updated along the opposite gradient, thereby reducing the loss function and completing one training step. Most of the model training process is defined by the user through code; only the backpropagation process is automatically derived by the framework using differentiation.

[0067] During model training, the training framework needs to explicitly or implicitly construct a computation graph that includes data preprocessing, forward computation, backpropagation, and gradient updates, based on the application training step defined in user code.

[0068] Based on the constructed computation graph, the training framework can handle device management, memory allocation and deallocation, and invoking computational devices to execute kernel functions. During computation execution on the terminal device, the framework maps one or more nodes in the computation graph to one or more computational functions and executes these functions on the terminal device. The training framework may map nodes in the computation graph to the same computational function (e.g., using deep learning libraries like cuDNN), or it may map them to different computational functions. Furthermore, device utilization varies. If the framework cannot allocate the next computational function before the terminal device completes its computational task, the terminal device will remain idle for a period of time. These factors collectively contribute to the performance differences between frameworks.

[0069] Based on this, this specification provides a task execution method that uses a training framework as a scheduler to schedule a specified computation graph to a terminal device for computation. Under the premise of ensuring the consistency of parameters, operators and their dependencies, and update methods under different training frameworks, the performance of the training framework is evaluated through two evaluation dimensions: efficiency (computational efficiency) and sufficiency (device utilization). The target training framework is then selected based on the evaluation results.

[0070] When the server receives the first task instruction, it may need to obtain the target model and each candidate training framework. In this specification, the first task instruction may be an instruction to evaluate the quality of each candidate training framework or to determine the target training framework for training the model to be trained.

[0071] In addition, the target model can be obtained from a known model library, and the training framework can be a deep learning framework, such as Caffe, TensorFlow, Microsoft Cognitive Toolkit (CNTK), MXNet, PyTorch, PaddlePaddle, OneFlow, etc. Of course, the training framework in this manual can also be other training frameworks such as reinforcement learning frameworks, and this manual does not make specific limitations on this.

[0072] It should be noted that the candidate training frameworks in this specification can be different training frameworks or different versions of the same training framework.

[0073] In addition, the execution subject used to implement the task execution method in this specification can be a specified device such as a server. For ease of description, this specification will only use the server as the execution subject as an example to illustrate one task execution method provided in this specification.

[0074] S103: With the goal of maintaining equivalence among at least one of the specified parameters involved in training the target model with different candidate training frameworks, the operators called by different candidate training frameworks and the dependencies between operators, and the update methods of different candidate training frameworks when updating the target model, the candidate training frameworks are adjusted to obtain each adjusted framework.

[0075] To effectively control variables among candidate training frameworks and ensure the objectivity and accuracy of evaluation results, the server needs to ensure that the specified parameters involved in training the target model by different candidate training frameworks are equivalent, the operators called by different candidate training frameworks and the dependencies between operators are equivalent (i.e., model equivalence), and the update methods of the model during training are equivalent (i.e., training equivalence). In this way, it is possible to effectively compare the operating mechanisms of different candidate training frameworks (such as the underlying logic of algorithms, technologies, etc. used by different training frameworks).

[0076] Specifically, in achieving parameter equivalence, the server can guarantee the equivalence of specified parameters involved in training the target model. These specified parameters include:

[0077] Hyperparameters involved in preprocessing the input data of the target model, such as the mean, variance, and random scaling factor used for standardized data;

[0078] The hyperparameters corresponding to the operators called by each training framework, such as whether the convolution includes a bias term, the eps value and momentum value of the BatchNormalization layer, and the weight initialization method of the operators called by each training framework.

[0079] Hyperparameters involved in updating the weights of the target model, such as learning rate and regularization factor;

[0080] Hyperparameters related to training performance, such as the number of threads performing data preprocessing in parallel.

[0081] It should be noted that the hyperparameters mentioned above are supported by all training frameworks, and setting these hyperparameters can improve performance compared to the default settings. In practical applications, these parameters may differ across different training frameworks, as they involve adjustments to the model and data processing during training. Taking the eps value as an example, this value represents the relative floating-point precision. Some training frameworks have a default eps value of 1e-6, while others use 1e-3, 1e-5, etc. Different eps values ​​can control the relative floating-point precision of the target model during training.

[0082] In the process of achieving model equivalence, the server can first determine whether the operators called by different training frameworks and the dependencies between them are the same.

[0083] Typically, the computation graph constructed by the training framework directly reflects the operators called by the training framework and the dependencies between them. In this specification, the computation graph of the entire training process can be G... g The computational graph of the model's forward propagation is denoted as G. m G m It is G g The subgraph. To make the evaluation results meaningful, user code should be implemented fairly for each framework; that is, the frameworks should construct the same G. g The training process can be viewed as an iterative process of training steps, and a single training step includes data preprocessing, forward computation, backpropagation, and parameter updates. For ease of understanding, this specification provides a schematic diagram of the model's iterative process, such as... Figure 2 As shown.

[0084] Figure 2 This is a schematic diagram of the iterative process of one of the models provided in this specification.

[0085] The model's single training step consists of four stages: data preprocessing, forward computation, backpropagation, and parameter update.

[0086] The forward computation process is based on G m The topological sorting sequentially performs the forward computation of the operators, ultimately yielding the output and loss function of each operator. Similarly, the backpropagation process is based on G... mThe topological sorting is performed in reverse order, and the operators are calculated in reverse order to obtain the gradient of the target model weights according to the chain rule. Weight update is then performed based on the weights and gradients, according to a certain update method.

[0087] The operators mentioned above can include operators of the target model itself, such as convolution operators and ReLU operators, or other operators provided by candidate training frameworks, such as vector multiplication and addition involved in updating the parameters of the target model.

[0088] In practical applications, since the G constructed by the training framework cannot be directly obtained... m The server can indirectly verify whether the model is equivalent by giving the same input and the same parameters to models implemented by different candidate training frameworks and checking whether the target model produces the same output. That is, the operators called by different training frameworks and the dependencies between the operators are equivalent.

[0089] The server can deploy the target model in each candidate training framework, input the same data and set the same parameters for the target model in each candidate training framework, and determine whether the target model produces the same output in each candidate training framework. If so, it is determined that the operators called by different candidate training frameworks and the dependencies between the operators are the same; otherwise, model equivalence has not been achieved.

[0090] If the specified parameters mentioned above are not kept equivalent, the server can adjust each candidate training framework to obtain an adjusted framework so that the specified parameters involved in each candidate training framework are kept equivalent.

[0091] For example, the server can take the average, median, or mode of the specified parameters corresponding to each candidate training frame as the specified parameters for each adjusted frame.

[0092] Furthermore, since the forward inference of the target model uses floating-point calculations, different calculation orders can cause slight differences in the results. The error accumulates during the forward calculation process, making it difficult to directly compare the outputs of different candidate training frameworks. Therefore, the server can select one of the candidate training frameworks to train the target model. After at least one iteration, the model parameters of the target model are exported and converted to a specified parameter format (such as NumPy format), and then loaded into the target model in other candidate training frameworks.

[0093] The server can then export the target model in each candidate training framework to a specified model format (such as Open Neural Network Exchange (ONNX)) and input the same data into the target model of each specified model format (ONNX), thereby running the target model using ONNX and determining whether the target models of each specified format produce the same output.

[0094] If the operators called by each candidate training framework and the dependencies between them are not kept equivalent, the server can adjust the parameters of each candidate training framework to obtain an adjusted framework, so that the operators called by each adjusted framework and the dependencies between them are not kept equivalent.

[0095] In achieving training equivalence, the server can ensure that different training frameworks maintain consistency in their update methods when updating the target model. These update methods can include: transformation functions applied to gradients, weight update functions, and regularization functions.

[0096] Taking weight update functions as an example, a widely used weight update function (also known as an optimizer) is the momentum optimizer. It adds a momentum term to the stochastic gradient descent optimizer, which has the advantage of enabling the model to converge quickly and reducing the possibility of getting stuck in local minima. However, the momentum optimizer may be implemented differently in different training frameworks.

[0097] With w t Taking the model parameters in the t-th iteration as an example, where ε is the learning rate, μ is the momentum factor, and gt is the gradient value in the t-th iteration, one type of momentum optimizer can be expressed as:

[0098] v t+1 =μv t +εg t+1

[0099] w t+1 =w t -v t+1

[0100] Another momentum optimizer can be expressed as:

[0101] v t+1 =μv t +g t+1

[0102] w t+1 =w t ―εv t+1

[0103] In this scenario, the server can adjust another momentum optimizer with the goal of maintaining equivalence in the update methods when updating the target model under different training frameworks, so that the two momentum optimizers mentioned above are equivalent. The adjusted second momentum optimizer can be expressed as:

[0104]

[0105] w t+1 =w t ―ε t+1 v t+1

[0106] By adjusting the transformation function applied to the gradient, the weight update function, and the regularization function to obtain the adjusted framework, the server can achieve the update method of the target model to ensure fairness among different candidate training frameworks.

[0107] It should be noted that the server can adjust each candidate training framework with the goal of maintaining equivalence among one or more of the specified parameters involved in training the target model with different candidate training frameworks, the operators called by different candidate training frameworks and the dependencies between the operators, and the update methods of different candidate training frameworks when updating the target model, to obtain each adjusted framework. Of course, it can also adjust each training framework with the goal of maintaining equivalence under all the above conditions.

[0108] S104 For each adjusted framework, determine the duration of the computation operation of the target model performed by the terminal device deploying the target model based on the adjusted framework, and use it as the computation duration.

[0109] S105: Determine the priority of the adjusted frame based on the computation time.

[0110] S106: Determine the target training framework from the candidate training frameworks according to the priority corresponding to each adjusted framework.

[0111] During model training, the terminal device deploying the target model framework will always be in one of two states: idle or computational (busy). The computational state refers to the terminal device executing one or more computational functions, while the idle state refers to the terminal device being idle and waiting because the computational functions have not been distributed to the terminal device in a timely manner.

[0112] If we view upper-layer applications as workloads with interdependent relationships, and terminal devices as service facilities that carry out the functions of executing those workloads, then the training framework can be seen as a scheduler that schedules workloads between the two. During model training, the training framework needs to map workloads to computational functions that can be executed on the service facilities before calling the terminal device to perform the computations.

[0113] For fixed upper-layer applications, the shorter the time the terminal device is in the computing state, the more efficiently the framework utilizes the terminal device; conversely, the shorter the time the terminal device is in the idle state, the more fully the framework utilizes the terminal device. Therefore, the server can use an evaluation system based on these two dimensions—efficiency and sufficiency—to assess the framework's performance.

[0114] A two-dimensional evaluation system of efficiency and sufficiency can better establish the relationship between evaluation results and technology. In practical applications, to achieve shorter training times, the training framework needs to be optimized in two aspects. First, by designing more efficient computation functions and employing operator fusion techniques, the execution time of computation functions can be minimized, thereby improving the efficiency of the training framework. Second, the framework should rationally allocate resources, using the time for terminal devices to execute computation functions to mask the time costs of input / output (I / O) and central processing unit (CPU) control flow, and eliminating unnecessary host-device synchronization operations to prevent terminal devices from being idle, thereby improving the sufficiency of the training framework.

[0115] Therefore, the server can determine the priority of each candidate training framework by considering the device utilization and computational efficiency corresponding to different adjusted frameworks.

[0116] Specifically, regarding efficiency, the server can measure the computation time of terminal devices deploying the target model when performing computational operations on the target model based on each adjusted framework. The server can sample the operating status of the terminal devices according to a preset sampling period, thereby measuring the total time overhead of the terminal devices executing computational functions within the sampling period, i.e., the duration of the terminal devices in the computational state within the sampling period. The preset sampling period can be set according to actual conditions, and this specification does not impose specific limitations on it. For each adjusted framework, the computation time (DCT) of the terminal devices under that adjusted framework can be expressed as:

[0117]

[0118] Among them, t s t represents the start time of the sampling period. e D represents the end time of the sampling period.active(t) D represents the state of the terminal device at time t. active(t)dt It can be represented as:

[0119]

[0120] Specifically, for any time t within the sampling period, when the terminal device is in the computation state, D active(t) The value is 1 if it is 1, otherwise it is 0.

[0121] The server can determine the computational efficiency of each adjusted framework based on its corresponding DCT (Discretionary Time Computation) calculation time. This computational efficiency is used to express the efficiency of different training frameworks. The shorter the computation time on the terminal device, the higher the corresponding computational efficiency, indicating that the training framework is more efficient in utilizing the terminal device. For example, for the same upper-layer application, when a training framework uses a shorter computation function to compute a convolutional layer, the terminal device will have less computation time and higher computational efficiency, resulting in better efficiency. Conversely, when a framework repeatedly computes certain operators, the terminal device will have more computation time and lower computational efficiency, leading to lower efficiency.

[0122] Of course, the server can also determine the time required to train the target model once using each adjusted framework, and determine the computational efficiency of each adjusted framework based on that time.

[0123] Regarding sufficiency, the server can determine the device utilization of the adjusted framework based on the ratio of the busy state duration of the terminal device to the sampling duration within the sampling period. This device utilization can be used to express the sufficiency of different training frameworks.

[0124]

[0125] The theoretical range of DOR is between 0 and 1. The longer the terminal device is idle, the closer the DOR is to 0. When the terminal device is constantly busy during the sampling period, the DOR reaches its maximum value of 1. Higher device utilization means that the training framework makes fuller use of the terminal device. For example, when the training framework cancels unnecessary synchronization operations, it can distribute the computation function to the terminal device earlier, improving device utilization and thus increasing the framework's sufficiency.

[0126] Of course, in this specification, the server may not sample the state of the terminal device according to the preset sampling period to determine the computation time, but instead determine the duration of the time that the terminal device deploying the target model is in the computation state when training the target model, as the computation time.

[0127] To facilitate understanding, this manual provides a schematic diagram of an evaluation method for the training framework, such as... Figure 3 As shown.

[0128] Figure 3 This is a schematic diagram of an evaluation method for a training framework provided in this specification.

[0129] The server needs to determine the computational efficiency and device utilization of each adjusted framework, while ensuring that the parameters, models, and training of each training framework are consistent. This efficiency and device utilization are then used as the computational efficiency and device utilization of the training framework before the adjustment, and the computational efficiency and device utilization of each training framework are used as the evaluation results of each training framework.

[0130] The server can determine the device utilization and computational efficiency corresponding to each adjusted framework using the above method, and then determine the priority of each adjusted framework based on the device utilization and computational efficiency, and finally determine the target training framework from each candidate training framework based on the priority.

[0131] For example, the server can determine whether to use an efficient training framework (a training framework with high computational efficiency) or a sufficient computing framework (a training framework with high equipment utilization) based on model information such as the type or structure of the model to be trained, and thus select the training framework that matches the model to be trained as the target training framework.

[0132] In addition, the server can also determine the type of training framework that the user (model developer) prefers based on the user's settings, and then select the target training framework according to the user's settings.

[0133] Of course, the server can also display the device utilization and training efficiency of each training framework as evaluation results to the user, so that the user can select the target training framework that meets their expectations based on the evaluation results.

[0134] In addition, the above evaluation results can also be presented to framework developers, so that they can identify the shortcomings of their own frameworks based on the device utilization and training efficiency of each training framework, and thus adjust and optimize the training frameworks to achieve breakthroughs in framework performance.

[0135] S107: When the second task instruction is received, the model training task for the target model is executed through the target training framework.

[0136] When the server receives a second task instruction (such as an instruction to execute a training task for the target model), the server can execute the training task for the target model using the determined target training framework. It should be noted that the target model for executing the training task can be the model used to determine the priority of each adjusted framework, or it can be other models that need to be trained.

[0137] As can be seen from the above method, this scheme can establish a two-dimensional (sufficiency and efficiency) evaluation system. By establishing evaluation dimensions of efficiency and sufficiency through the two optimization directions of the training framework, it can more closely link the technology and implementation methods used by the framework with the evaluation results. By measuring the device utilization and computational efficiency of each training framework on the terminal device, the evaluation dimensions of efficiency and sufficiency can be effectively quantified, which is more effective than previous indicators.

[0138] In addition, this solution provides a method to ensure fairness across different training frameworks used in the evaluation. Compared to previous methods that simply ensure identical training configurations, this method abstracts the computational load of the frameworks into a computation graph. Through three equivalent steps, it achieves full coverage of the data preprocessing, forward computation, backpropagation, and weight update processes during training. This effectively identifies and eliminates unfair factors in the implementation of different frameworks, making the evaluation results more reliable.

[0139] The above describes one or more implementation methods for task execution in this specification. Based on the same approach, this specification also provides corresponding task execution devices, such as... Figure 4 As shown.

[0140] Figure 4 A schematic diagram of a task execution device provided in this specification includes:

[0141] Receiver module 401 is used to receive the first task instruction;

[0142] The acquisition module 402 is used to acquire the target model and each candidate training framework according to the first task instruction;

[0143] The adjustment module 403 is used to adjust each candidate training framework with the goal of maintaining equivalence in at least one of the specified parameters involved in training the target model with different candidate training frameworks, the operators called by different candidate training frameworks and the dependencies between operators, and the update methods when different candidate training frameworks update the target model, so as to obtain each adjusted framework.

[0144] The first determining module 404 is used to determine, for each adjusted framework, the duration of time when the terminal device deploying the target model performs the operation of the target model based on the adjusted framework, as the operation duration;

[0145] The second determining module 405 is used to determine the priority corresponding to the adjusted frame based on the computation time.

[0146] The third determining module 406 is used to determine the target training framework from the candidate training frameworks according to the priority corresponding to each adjusted framework.

[0147] The execution module 407 is used to execute a model training task for the target model through the target training framework when a second task instruction is received.

[0148] Optionally, the specified parameters include at least one of the following: hyperparameters involved in preprocessing the input data of the target model, hyperparameters corresponding to the operators called by each candidate training framework, hyperparameters involved in updating the weights of the target model, and hyperparameters related to training performance.

[0149] Optionally, the update method includes at least one of the following: a transformation function applied to the gradient, a weight update function, and a regularization function.

[0150] Optionally, before adjusting each candidate training framework to obtain each adjusted framework, the adjustment module 403 is further configured to determine whether the operators called by each candidate training framework and the dependencies between each operator are the same; if not, the candidate training framework is adjusted with the goal of keeping the operators called by each candidate training framework and the dependencies between each operator equivalent.

[0151] Optionally, the adjustment module 403 is specifically used to deploy the target model in each candidate training framework, input the same data and set the same parameters for the target model in each candidate training framework, determine whether the target model produces the same output in each candidate training framework, and if so, determine that the operators called by each candidate training framework and the dependencies between each operator are the same.

[0152] Optionally, the adjustment module 403 is specifically used to: select one of the candidate training frameworks to train the target model; after at least one iteration, export the model parameters of the target model and convert them into a specified parameter format and load them into the target models of other candidate training frameworks; export the target models in each candidate training framework into a specified model format, input the same data into the target models of each specified model format, and determine whether the target models of each specified model format produce the same output.

[0153] Optionally, the first determining module 404 is specifically used to sample the operating status of the terminal device according to a preset sampling period; and determine the duration of the terminal device deploying the target model within the sampling period when it performs the operation of the target model based on the adjusted framework, as the operation duration.

[0154] Optionally, the second determining module 405 is specifically used to determine the device utilization rate and computing efficiency of the adjusted framework for the terminal device based on the computing time; and to determine the priority based on the device utilization rate and the computing efficiency.

[0155] Optionally, the second determining module 405 is specifically used to determine the device utilization rate based on the computation time and the sampling time corresponding to the sampling period, wherein the device utilization rate is positively correlated with the computation time.

[0156] Optionally, the longer the computation time, the lower the computation efficiency.

[0157] Optionally, the training framework includes a deep learning framework.

[0158] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 This provides a task execution method.

[0159] This instruction manual also provides Figure 5 One of the corresponding Figure 1 A schematic diagram of the structure of an electronic device. (e.g.) Figure 5 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The task execution method described herein. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0160] Improvements in a technology can be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology can now be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement in methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog are the most commonly used. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0161] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0162] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0163] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0164] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0166] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0167] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes ​ The steps of the function specified in one or more boxes.

[0168] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0169] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0170] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0171] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0172] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0173] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0174] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0175] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A task execution method, characterized in that, include: Receive the first task instruction; According to the first task instruction, obtain the target model and each candidate training framework; With the goal of maintaining equivalence among at least one of the specified parameters involved in training the target model with different candidate training frameworks, the operators called by different candidate training frameworks and the dependencies between operators, and the update methods of different candidate training frameworks when updating the target model, the candidate training frameworks are adjusted to obtain each adjusted framework. For each adjusted framework, the duration of time when the terminal device deploying the target model performs the computation operation of the target model based on the adjusted framework is determined as the computation duration; Based on the computation time, determine the priority of the adjusted framework; Based on the priority corresponding to each adjusted frame, the target training frame is determined from each candidate training frame; When the second task instruction is received, the model training task for the target model is executed through the target training framework.

2. The method as described in claim 1, characterized in that, The specified parameters include at least one of the following: hyperparameters involved in preprocessing the input data of the target model, hyperparameters corresponding to the operators called by each candidate training framework, hyperparameters involved in updating the weights of the target model, and hyperparameters related to training performance.

3. The method as described in claim 1, characterized in that, The update method includes at least one of the following: a transformation function applied to the gradient, a weight update function, and a regularization function.

4. The method as described in claim 1, characterized in that, Before adjusting each candidate training framework to obtain each adjusted framework, the method further includes: Determine whether the operators called by each candidate training framework and the dependencies between them are the same; If not, the candidate training frameworks are adjusted with the goal of maintaining equivalence between the operators called by each candidate training framework and the dependencies between them.

5. The method as described in claim 4, characterized in that, Determining whether the operators invoked by each candidate training framework and the dependencies between them are the same includes: The target model is deployed in each candidate training framework. The same data is input and the same parameters are set for the target model in each candidate training framework. It is determined whether the target model produces the same output in each candidate training framework. If so, it is determined that the operators called by each candidate training framework and the dependencies between each operator are the same.

6. The method as described in claim 5, characterized in that, Inputting the same data and setting the same parameters to the target model under each candidate training framework, and determining whether the target model produces the same output in each candidate training framework, specifically includes: One of the candidate training frameworks is selected to train the target model; After at least one iteration, the model parameters of the target model are exported and converted into a specified parameter format and loaded into the target model of other candidate training frameworks; Export the target model in each candidate training framework to a specified model format, input the same data into each target model in the specified model format, and determine whether the target models in each specified model format produce the same output.

7. The method as described in claim 1, characterized in that, For each adjusted framework, the duration of computation time for a terminal device deploying the target model to perform computational operations on the target model based on the adjusted framework is determined, specifically including: The operating status of the terminal device is sampled according to a preset sampling period; The duration during which a terminal device deploying the target model performs computational operations on the target model based on the adjusted framework is determined within the sampling period is defined as the computation duration.

8. The method as described in claim 7, characterized in that, Based on the computation time, the priority of the adjusted framework is determined, specifically including: Based on the computation time, determine the device utilization rate and computational efficiency of the adjusted framework for the terminal device; The priority is determined based on the equipment utilization rate and the computing efficiency.

9. The method as described in claim 8, characterized in that, Based on the computation time, the device utilization rate is determined, specifically including: The device utilization rate is determined based on the computation time and the sampling time corresponding to the sampling period, wherein the device utilization rate is positively correlated with the computation time.

10. The method as described in claim 8, characterized in that, The longer the computation time, the lower the computation efficiency.

11. The method as described in claim 1, characterized in that, The training framework includes a deep learning framework.

12. A task execution device, characterized in that, include: The receiving module receives the first task instruction; The acquisition module acquires the target model and each candidate training framework according to the first task instruction; The adjustment module aims to maintain equivalence among at least one of the specified parameters involved in training the target model using different candidate training frameworks, the operators called by different candidate training frameworks and the dependencies between operators, and the update methods of different candidate training frameworks when updating the target model, thereby adjusting each candidate training framework to obtain each adjusted framework. The first determining module determines, for each adjusted framework, the duration of time when the terminal device deploying the target model performs the calculation operation of the target model based on the adjusted framework, as the calculation duration; The second determining module determines the priority of the adjusted framework based on the computation time. The third determination module determines the target training framework from the candidate training frameworks based on the priority of each adjusted framework. When the execution module receives the second task instruction, it executes the model training task for the target model through the target training framework.

13. The apparatus as claimed in claim 12, characterized in that, The first determining module is specifically used to sample the operating status of the terminal device according to a preset sampling period; and determine the duration of the terminal device deploying the target model within the sampling period when it performs the operation of the target model based on the adjusted framework, as the operation duration.

14. The apparatus as claimed in claim 13, characterized in that, Based on the computation time, determine the device utilization rate and computational efficiency of the adjusted framework for the terminal device; determine the priority based on the device utilization rate and computational efficiency.

15. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 11.

16. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 11.

Images