Method for generating machine learning model, electronic device, program product

Abstract

The embodiment of the invention relates to a method for generating a machine learning model. The method includes extracting a plurality of parameters from a target machine learning model, the plurality of parameters including a learning rate, state information, a loss value, a gradient, and a weight, the target machine learning model being used to perform a task related to at least one of an image, a video, a voice, and a text. The method further includes predicting, based on the plurality of parameters, a first learning rate by the first machine learning model, predicting, based on the plurality of parameters, a second learning rate by the second machine learning model, selecting, based on the first and second learning rates, a learning rate having a minimum loss value of the first and second learning rates, and adjusting the target machine learning model based on the learning rate having the minimum loss value. Through the method disclosed by the invention, the future value of the learning rate in the training process can be predicted, so that the learning rate and other performances of the target machine learning model can be adjusted and improved in real time.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to methods, electronic devices, and computer program products for generating machine learning models. Background Art

[0002] In machine learning models, hyperparameters such as learning rate, number of iterations, initialization weights, convolution kernel size and number play a vital role in determining the performance of machine learning models.

[0003] Among these hyperparameters, the learning rate can affect the step size of updating parameters during the training process of the machine learning model, thereby affecting the training speed, quality and generalization ability of the machine learning model. However, the methods commonly used to set the learning rate rely on heuristics or simple search methods, resulting in unsatisfactory results. Summary of the invention

[0004] The embodiments of the present disclosure relate to methods, electronic devices, and computer program products for generating machine learning models.

[0005] According to a first aspect of the present disclosure, a method for machine model training is provided. The method includes extracting multiple parameters from a target machine learning model, the multiple parameters including learning rate, state information, loss value, gradient, and weight, and the target machine learning model is used to perform tasks related to at least one of images, videos, voices, and texts. The method also includes predicting a first learning rate associated with the target machine learning model by a first machine learning model based on multiple parameters, predicting a second learning rate associated with the target machine learning model by a second machine learning model based on multiple parameters, selecting a learning rate with a minimum loss value in the first learning rate and the second learning rate based on the first learning rate and the second learning rate, and adjusting the target machine learning model based on the learning rate with the minimum loss value.

[0006] According to a second aspect of the present disclosure, an electronic device for generating a machine learning model is provided. The electronic device includes at least one processor; and a memory, coupled to the at least one processor and having instructions stored thereon, the instructions causing the electronic device to perform an action when executed by the at least one processor, the action including extracting multiple parameters from the target machine learning model, the multiple parameters including learning rate, state information, loss value, gradient, and weight, the target machine learning model is used to perform tasks related to at least one of images, videos, voices, and texts. The action also includes predicting a first learning rate associated with the target machine learning model by a first machine learning model based on multiple parameters, predicting a second learning rate associated with the target machine learning model by a second machine learning model based on multiple parameters, selecting a learning rate with a minimum loss value between the first learning rate and the second learning rate based on the first learning rate and the second learning rate, and adjusting the target machine learning model based on the learning rate with the minimum loss value.

[0007] According to a third aspect of the present disclosure, there is provided a computer program product, which is tangibly stored on a non-volatile computer-readable medium and includes machine-executable instructions, which when executed cause a machine to perform the steps of the method implemented in the first aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like elements throughout the exemplary embodiments of the present disclosure.

[0009] Figure 1 Schematic diagrams showing examples in which the devices and / or methods of embodiments of the present disclosure may be implemented;

[0010] Figure 2 A flowchart of a method for generating a machine learning model according to an embodiment of the present disclosure is shown;

[0011] Figure 3 A schematic diagram of a reinforcement learning model training process according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A schematic block diagram of an example device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0013] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some specific embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0014] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects, unless explicitly indicated to be different.

[0015] Methods commonly used to train machine models for optimal learning rates include using grid search or random search. However, these methods are often computationally expensive, and the ability to find the best hyperparameters for the best learning rate may be limited. Another approach is to use a dynamic learning rate schedule, such as step decay or exponential decay, which adjusts the learning rate according to a predefined schedule. Although these methods can provide improved performance compared to fixed learning rates, they are still limited by the choice of schedule. In existing methods, the optimized values ​​for hyperparameters such as learning rates are not easy to determine from previously provided data, are costly and time-consuming, and traditional methods for adjusting hyperparameters are often implicit and unintelligent, resulting in poor performance, and the choice of hyperparameters can greatly affect the convergence speed and overall performance of the machine model. In addition, these methods can also use other features or statistics, such as channel quality or collision probability, and are designed for specific fields (e.g., wireless communications or robotics).

[0016] At least to solve the above and other potential problems, an embodiment of the present disclosure provides a method for generating a machine learning model. The method includes extracting multiple parameters from a target machine learning model, the multiple parameters including learning rate, state information, loss value, gradient, and weight, and the target machine learning model is used to perform tasks related to at least one of images, videos, voices, and texts. The method also includes predicting a first learning rate associated with the target machine learning model by a first machine learning model based on multiple parameters, predicting a second learning rate associated with the target machine learning model by a second machine learning model based on multiple parameters, selecting a learning rate with a minimum loss value in the first learning rate and the second learning rate based on the first learning rate and the second learning rate, and adjusting the target machine learning model based on the learning rate with the minimum loss value. By using this method, it is possible to adjust the learning rate by combining a variety of different methods (including but not limited to reinforcement learning models, neural network models, and / or any other known or unknown machine learning models), thereby forming a smarter and more effective solution, predicting the future value of the learning rate during training, so as to make real-time adjustments and improve convergence, and provide faster convergence speed and better machine learning model performance.

[0017] The basic principles and several exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Figure 1 A schematic diagram of an example 100 is shown in which the apparatus and / or method of embodiments of the present disclosure may be implemented. Figure 1 It may include a target machine learning model 101, a first machine learning model 105, and a second machine learning model 110. It should be understood that Figure 1 The types and numbers of models, data transmission processes, arrangements, etc. shown are only exemplary, and example 100 may include models of different numbers and arranged in different ways, data transmission processes, and various additional elements, etc. It should be understood that the above examples are only used to illustrate the application of the target machine learning model 101. With the development of technology, the target machine learning model 101 may include various known or unknown applications in various fields and aspects.

[0018] In the example environment 100, the target machine learning model 101, the first machine learning model 105, and the second machine learning model 110 can be installed in any computing device that has processing computing resources or storage resources. For example, the computing device can have common capabilities such as receiving and sending data requests, real-time data analysis, local data storage, and real-time network connection. Computing devices can generally include various types of devices. Examples of computing devices may include, but are not limited to: desktop computers, laptops, smart phones, wearable devices, security equipment, smart manufacturing equipment, smart home devices, Internet of Things devices, smart cars, drones, etc., and the present disclosure does not impose any restrictions on this.

[0019] According to an embodiment of the present disclosure, the loss function for the target machine learning model 101 can be expressed as L(w), where w represents the model parameters. The learning rate for the target machine learning model 101 can be expressed as α, which represents the amplitude of the parameter update of the target machine learning model 101 in each iteration. The goal can be to find the optimal learning rate α * , thereby improving the target machine learning model 101 to the optimal parameter w * The process can be expressed as:

[0020] α * =argmin α L(w(α)) (1)

[0021] Where w(α) is a model parameter with a learning rate α at iteration t. According to embodiments of the present disclosure, a variety of different models including but not limited to reinforcement learning models and machine learning models can be used to predict and adjust the learning rate during training to achieve the optimal α * .

[0022] According to an embodiment of the present disclosure, the first machine learning model 105 may be a reinforcement learning model. The training process of the reinforcement learning model may provide a framework for optimizing decision making in a dynamic environment. The reinforcement learning model focuses on how an agent takes actions in a dynamic environment to maximize cumulative rewards.

[0023] The training process of the reinforcement learning model can include multiple elements such as agents, environments, states, actions, and rewards. The agent is responsible for taking actions in the environment. The agent can be a robot or other entity that needs to learn how to make decisions. The environment can be real or virtual, and it can give feedback to the agent's actions, including rewards and state transitions. The state represents a specific state of the environment at a certain moment, which can be used to describe the information of the environment. At each time step, the agent observes the current state and makes decisions based on the state. The reward function for the reward can be the feedback provided by the environment to the agent at each time step, indicating the evaluation of the agent's actions. The reward can be positive (encouraging the agent to continue taking similar actions) or negative (punishing the agent to avoid similar actions). The goal of the reinforcement learning model is to learn the optimal strategy through interaction with the environment, so that the agent can choose the best action in continuous trial and error to maximize the cumulative reward. The process of the reinforcement learning model is usually an iterative process. During the training process, the agent constantly tries and improves its own strategy to obtain better results.

[0024] In the context of machine learning, the learning rate can be viewed as a decision made at each iteration based on the current state of the model and information from previous iterations. A reinforcement learning model can be trained to make this decision, with the goal of minimizing a loss function. Additionally or alternatively, the first machine learning model 105 can also be based on other types of machine learning models, and the present disclosure does not impose any limitations on this.

[0025] According to an embodiment of the present disclosure, the first machine learning model 105 can be trained by using the current state and loss value of the target machine learning model 101 to predict the optimal learning rate for the target machine learning model 101 at a given iteration of the training process. According to some embodiments of the present disclosure, the state data of the model at the tth iteration of the target machine learning model 101 during its generation process can be represented by the vector S t Represented by, and the loss value of the tth iteration can be expressed by L t The training goal of the first machine learning model 105 is to predict the optimal learning rate α for the target machine learning model 101 at the next iteration t+1 t .

[0026] During training, the action space of the first machine learning model 105 can be the set of all possible learning rates at a given iteration during training. The state space can be a representation of the current state of the model, including information such as current weights, gradients, loss values, etc. The reward space / reward function is the set of all possible loss values ​​that can be generated by the first machine learning model 105, and the goal is to minimize this value. The reward signal can be defined as the difference between the current loss and the target loss, and serves as an indicator of the quality of the selected learning rate.

[0027] The reinforcement learning model can be represented as a Markov decision process (MDP), where the state s t , Action α t and reward L t Can form tuples (s t , α t , L t ). Action α t is the first learning rate selected by the first machine learning model 105, and the reward L t is the loss value. The goal of the first machine learning model 105 is to find the policy network / optimal policy π * , given the current state s t Maximize the expected reward over time in the case of:

[0028]

[0029] where π * (st ) represents the optimal strategy, a t is the action taken at time step t, Represents the current state s given t The next state s t+1 The expected distribution and action a t , r t+1 is the immediate reward at time step t+1, γ is the discount factor, and the sum represents the discounted cumulative reward over an infinite horizon.

[0030] It should be understood that various methods can be used to train the first machine learning model 105, including but not limited to Q learning or policy gradient methods, etc. During training, the first machine learning model 105 can be updated based on observed rewards and estimated values ​​of future rewards.

[0031] Once the first machine learning model 105 is trained, it can be used to predict the optimal learning rate α for each iteration t during the training process. t The first machine learning model 105 can be based on the current state s of the target machine learning model 101 t and the current loss L t as input, and output predictions with the first learning rate α t The target machine learning model 101 can then use this predicted learning rate to update the weights of the model in the next iteration.

[0032] Additionally or alternatively, in some embodiments, multiple learning rates predicted by the first machine learning model 105 can be compared with previously marked target baseline truth learning rates to select a predicted learning rate that is closest to the target baseline truth learning rate for the target machine learning model 101. In other words, the predicted learning rate and the target baseline truth learning rate can have a minimum loss value. The target baseline truth learning rate for the target machine learning model 101 can be confirmed by a previous manual or automatic grid search. For example, multiple parameters 102 of the target machine learning model 101 can be extracted first, and the learning rate for the target machine learning model 101 can be between 0.5 and 0.05. By gradually decreasing the learning rate, for example, decreasing by 0.01 each time, and debugging the parameters during the decreasing process, one or more target baseline truth learning rates suitable for the target machine learning model 101 are obtained during the debugging process.

[0033] According to an embodiment of the present disclosure, the second machine learning model 110 can also be trained so that the second machine learning model 110 predicts a second learning rate associated with the target machine learning model 101. As an example, in the present disclosure, the second machine learning model 110 can be a neural network machine learning model, which can simulate the way information is processed in the human brain, so as to learn and perform tasks such as classification, regression, image processing, etc. The neural network machine learning model can be composed of multiple layers, each layer containing multiple neurons or nodes. These layers are connected by weights, and the patterns and features of the data are learned by adjusting the weights during the training process. As an example, the neural network machine learning model may include, but is not limited to, a feedforward neural network layer for passing information from the input layer to the output layer through the hidden layer; a convolutional neural network layer for extracting or pooling features such as in image data; a generator and a discriminator for adversarial training of data; a transformer for implementing an attention mechanism and other components, and the present disclosure does not impose any restrictions on this.

[0034] It should be understood that the above description of the second machine learning model 110 is merely exemplary, and the second machine learning model 110 can be any known or unknown machine learning model, including but not limited to decision trees, random forests, support vector machines (SVMs), logistic regression models, K-means clustering, hierarchical clustering, principal component analysis (PCA), autoencoders, reinforcement learning models, transfer learning models, generative adversarial network models, sequence models, recurrent neural network (RNN) models, long short-term memory networks (LSTM) models, transformers, gradient boosting trees, integrated learning models, etc. The present disclosure does not impose any limitations on this.

[0035] Similarly, multiple parameters 102 for the target machine learning model 101 may first be obtained as input features, including but not limited to its current weight, loss value, and target value for the next iteration. These acquired parameters 102 may be fed into the second machine learning model 110 as training data for training the second machine learning model 110. Given the current weight and loss of the main model, the second machine learning model 110 may predict the optimal learning rate for the next iteration. The above training process may be represented by the following loss function:

[0036]

[0037] Where n is the number of samples in the training data, α true，i is the target baseline truth learning rate for the i-th sample, α pred，i is the learning rate predicted by the second machine learning model 110.

[0038] By making the learning rate α predicted by the second machine learning model 110 pred，iand the target ground truth learning rate α true，i The difference between them is minimized (i.e., there is a minimum loss function value between the two) to continuously adjust and train the second machine learning model 110. For example, the second machine learning model 110 can be continuously adjusted and optimized by adjusting the content of the sample data used for training the second machine learning model 110 and the number of sample data.

[0039] After the training process for the second machine learning model 110 is completed, the second machine learning model 110 can be used to predict the optimal learning rate during the training process for the target machine learning model 101, and the process can be represented by the following formula:

[0040] α pred =f(weight, loss) (4)

[0041] Where f is the trained second machine learning model 110, weight and loss are the current weight and loss values ​​of the target machine learning model 101.

[0042] According to an embodiment of the present disclosure, an inference phase may also be included. In this phase, the first machine learning model 105 and the second machine learning model 110 may be used to respectively predict the optimal learning rate for the target machine learning model 101. Specifically, one or more current values ​​of the learning rate, state information, loss value, gradient, and weight in the current iteration for the target machine learning model 101 may be first obtained. Subsequently, the first machine learning model 105 may be used to predict and generate the first learning rate 108 based on the learning rate, state information, loss value, gradient, and weight for the target machine learning model 101.

[0043] Similarly, the second machine learning model 110 can be used to predict and generate a second learning rate 112 based on the learning rate, state information, loss value, gradient, and weight for the target machine learning model 101. The first learning rate 108 and the second learning rate 112 can then be compared with the target benchmark truth learning rate 116, respectively, so as to select the learning rate closest to the target benchmark truth learning rate (with the smallest loss value or the highest prediction reward) 120. The target machine learning model 101 can then be adjusted based on the learning rate 120 with the smallest loss value.

[0044] Additionally or alternatively, after selecting the learning rate predicted and generated by one model, another model may continue to be adjusted and trained. For example, after selecting the first learning rate 108 predicted and generated by the first machine learning model 105, the second machine learning model 110 may be adjusted and trained, and the new second learning rate 112 predicted and generated by the adjusted second machine learning model 110 may be used for comparison in the next round of iterative training. The above process may be repeated iteratively until the training process is completed, for example, the learning rate or optimization effect expected by the user is achieved.

[0045] Combined with the above Figure 1 A schematic diagram is described in which the method and / or process according to the embodiment of the present disclosure can be implemented. Figure 2 A flowchart of a method 200 for generating a machine learning model according to an embodiment of the present disclosure is described. The method 200 for generating a machine learning model according to an embodiment of the present disclosure can be executed at an edge device with computing power or at a cloud server, and the present disclosure does not limit this. In order to more efficiently generate a machine learning model and its related learning rate, a method 200 for generating a machine learning model according to an embodiment of the present disclosure is proposed.

[0046] At box 201, multiple parameters are extracted from a target machine learning model, the multiple parameters including learning rate, state information, loss value, gradient, and weight, and the target machine learning model is used to perform tasks related to at least one of images, videos, voices, and texts. The target machine learning model can be a machine learning model for performing one or more of tasks such as image processing, natural language (NLP) processing, data modeling, personalized recommendation, demand forecasting, anomaly detection, video processing, voice processing, and text processing. The multiple parameters can be any parameters associated with the target machine learning model.

[0047] At box 204, a first learning rate associated with the target machine learning model is predicted based on multiple parameters. According to an embodiment of the present disclosure, the reinforcement learning model as the first machine learning model can dynamically predict the first learning rate associated with the target machine learning model based on state information, loss value, gradient, and weight parameters among the multiple parameters previously extracted and according to the progress of training and changes in the environment.

[0048] At box 208, based on multiple parameters, a second learning rate associated with the target machine learning model is predicted. As an example, according to an embodiment of the present disclosure, a machine learning model such as a neural network model as the second machine learning model can similarly predict the second learning rate associated with the target machine learning model based on parameters such as state information, loss value, gradient, and weight among the multiple parameters previously extracted and additionally based on historical sample data accumulated in previous experiments.

[0049] At block 212, a learning rate having a minimum loss value among the first learning rate and the second learning rate may be selected based on the first learning rate and the second learning rate. The first learning rate and the second learning rate may be compared with the target baseline learning rate, respectively, to select a learning rate having the minimum difference from the target baseline learning rate, and wherein the target baseline learning rate has been determined using a grid search method. In practice, the target baseline learning rate may usually be finally determined by performing experiments and adjusting parameters, and trying different learning rate strategies.

[0050] According to some embodiments of the present disclosure, if the first learning rate predicted by the first machine learning model is determined to be the learning rate with the minimum loss value, the second machine learning model can be adjusted to minimize the difference between the second learning rate predicted by the second machine learning model and the target baseline learning rate. Specifically, the difference between the second learning rate generated by the second machine learning model and the target baseline learning rate can be reduced by adjusting the sample data and the number of sample data used by the second machine learning model to predict the second learning rate.

[0051] According to some embodiments of the present disclosure, if the second learning rate predicted by the second machine learning model is determined to be the learning rate with the minimum loss value, the first machine learning model can be adjusted to minimize the difference between the first learning rate predicted by the first machine learning model and the target baseline learning rate. Specifically, the first machine learning model can be adjusted by one or more methods of the Q learning method and / or the policy gradient method to reduce the difference between the first learning rate generated by the first machine learning model and the target baseline learning rate.

[0052] Additionally or alternatively, when the difference between the first learning rate generated by the first machine learning model and the target baseline learning rate is reduced, a reward value associated with the first learning rate (the difference between the first learning rate and the target baseline learning rate) is obtained, and based on the reward value, the first machine learning model can be trained, for example, in a direction in which the difference between the first learning rate and the target baseline learning rate is reduced and another learning rate in the next iteration is predicted. In each iteration of the multiple iterative training adjustment process of the target machine learning model, one of the first machine learning model or the second machine learning model can be adjusted.

[0053] At box 216, the target machine learning model can be adjusted based on the determined learning rate with the minimum loss value. For example, the target machine learning model can adjust the step size of the corresponding training process for updating the parameters based on the determined learning rate, that is, the amplitude of updating the parameters in the direction of gradient descent in the parameter space. Ultimately, faster convergence speed and stability of the training of the target machine learning model can be achieved, overfitting can be avoided, and the efficiency of calculation can be improved.

[0054] Figure 3 FIG. 3 is a schematic diagram of a reinforcement learning model training process 300 according to some embodiments of the present disclosure. Figure 3 As shown, it may include an environment 301 and a reinforcement learning model 303. The environment 301 is a scenario in which the reinforcement learning model 303 performs learning and decision-making. As an example, in an embodiment of the present disclosure, the environment 301 may be a process for generating a target machine learning model. At each time step t, the environment 301 receives the action (A_t) of the reinforcement learning model 303, and returns a reward value based on the current state and action, and transfers to a new state. The state (S_t) of the environment 301 represents the current observation value of the environment, which is information describing the current situation of the environment. The reinforcement learning model 303 selects an action based on the state (S_t). As an example, in the present disclosure, the state (S_t) may be one or more parameters associated with the target machine learning model.

[0055] The reinforcement learning model / agent 303 is the subject of learning and decision-making. It learns and optimizes strategies by interacting with the environment 301, selecting actions (A_t) according to the current state, and learning and optimizing strategies according to the reward value returned by the environment. The action (A_t) can be the behavior performed by the reinforcement learning model 303 at each time step. The reinforcement learning model 303 selects an action according to the current state and passes it to the environment 301. As an example, in an embodiment of the present disclosure, the action (A_t) can be the learning rate predicted by the reinforcement learning model 303 for the target machine learning model.

[0056] The reward value (R_t) is the signal returned by the environment after the reinforcement learning model 303 performs an action, which is used to evaluate the quality of the action. The reward value (R_t) is a key feedback signal in reinforcement learning. The goal of the reinforcement learning model 303 is to maximize the cumulative reward by learning to adjust the strategy. As an example, in an embodiment of the present disclosure, the reward value (R_t) can be the difference between the learning rate predicted by the reinforcement learning model 303 and the baseline truth learning rate. If the action performed by the reinforcement learning model 303 reduces the difference between the two, the reward value (R_t) is increased to encourage such behavior; if the action performed by the reinforcement learning model 303 increases the difference between the two, the reward value (R_t) is reduced and such behavior is not encouraged.

[0057] In reinforcement learning, the reinforcement learning model 303 continuously interacts with the environment 301 to learn the optimal strategy by observing states, selecting actions, receiving reward values, etc. The reinforcement learning model 303 gradually improves its strategy through trial and error and feedback mechanisms to maximize the cumulative reward and thus complete the desired task, such as approaching the benchmark truth learning rate.

[0058] One of the main challenges in tuning the learning rate during training for a target machine learning model is balancing exploration and exploitation. Exploration refers to trying different learning rates, which may lead to better convergence and performance, while exploitation refers to sticking to the learning rate that has proven to work so far. The trade-off between exploration and exploitation is necessary to avoid getting stuck in a local optimum or going beyond the global optimum.

[0059] In an embodiment according to the present disclosure, the proposed method combines two different methods of predicting learning rates: a reinforcement learning (RL) model and a machine learning (ML) model. The reinforcement learning model can explore different learning rates based on trial-and-error feedback of a loss function, which acts as a reward signal, and the machine learning model can utilize historical data of previous training tasks and generalize well in different settings. By combining these two models, a balance between exploration and exploitation and robustness and accuracy can be achieved.

[0060] Reinforcement learning models can adapt to dynamic changes in loss conditions and discover new learning rates to improve convergence and performance of training tasks. However, due to their stochastic nature, reinforcement learning models can also suffer from high variance and instability. Machine learning models can leverage prior knowledge and experience from previous training tasks and predict learning rates that may be applicable to similar tasks. However, due to their deterministic nature, machine learning models can also be biased and overfit. Therefore, by using these two models in parallel and selecting the best model based on the quality metric - the reduction in predicted loss value. The reduction in predicted loss value can be calculated by comparing the current loss value with the expected loss value after applying a certain learning rate. The selected learning rate indicates the maximum reduction in the predicted loss value for the next round of training. In this way, the advantages of both models can be utilized and their limitations can be compensated.

[0061] It should be understood that the method implemented according to the present disclosure is not intended to completely eliminate hyperparameter adjustment, but to automate and optimize one of its most important aspects-learning rate. The method implemented according to the present disclosure does not need to fine-tune all hyperparameters of the two models. In fact, the method implemented according to the present disclosure can work well in most cases using reasonable default values. The method implemented according to the present disclosure can also benefit from existing hyperparameter optimization methods, such as grid search, random search, evolutionary algorithm or Bayesian optimization. Grid search is a simple and widely used technique that involves testing different combinations of hyperparameters on a validation set and selecting the best combination based on performance indicators. Bayesian optimization is a more complex method that involves building a probability model of the objective function (e.g., validation error) and using it to guide the search for the best hyperparameters. By treating the hyperparameters of the two models as inputs and the reduction of the prediction loss as output, both techniques can be applied to the method implemented according to the present disclosure. According to an embodiment of the present disclosure, the method can be summarized as an ensemble method, in which two models (reinforcement learning model and machine learning model) predict the value of the same parameter (learning rate), and the best one prediction value is selected based on the quality metric (reduction of the prediction loss).

[0062] Additionally or alternatively, the method implemented according to the present disclosure can be more easily extended to predict other hyperparameters or parameters that affect the training process, such as the momentum, weight decay, loss rate, batch size, etc. for the target machine learning model with the minimum loss value, respectively predicted by the reinforcement learning model and the machine learning model based on the multiple parameters extracted from the target machine learning model. For example, these variable parameters can be added to the state and action space of the reinforcement learning model, and the reward function can be modified accordingly, so that the reinforcement learning model and / or the machine learning model can predict the corresponding parameters with the minimum loss value.

[0063] The following is experimental result data of experiments using grid search and Bayesian optimization to adjust some key hyperparameters of the method implemented according to the present disclosure, such as the discount factor and exploration rate of the reinforcement learning model, and the number of hidden layers and neurons for the machine learning model. The method implemented according to the present disclosure was compared with different baselines by using different hyperparameter optimization techniques. The results show that no matter what hyperparameter optimization technique is used, the method implemented according to the present disclosure can achieve faster convergence and better performance than existing methods.

[0064] First, experiments can be conducted on benchmark datasets for machine learning model tasks, including but not limited to MNIST (handwritten digit recognition), CIFAR-10 (image classification), and IMDB (sentiment analysis). The method implemented in the present disclosure can be compared with three baselines, such as a constant learning rate, an adaptive learning rate based on gradient descent (AdaGrad), and an adaptive learning rate based on momentum (Adam). In addition to the learning rate, the same network architecture and hyperparameters can be used for each dataset and each method. Each experiment can be run 10 times with different random seeds, and the mean and standard deviation of the accuracy and loss metrics are reported.

[0065] The experimental results are shown in Table 1 below. From the experimental results, it can be seen that the method implemented according to the embodiment of the present disclosure outperforms the baselines of all data sets in terms of accuracy and loss. The method implemented according to the embodiment of the present disclosure achieved a maximum accuracy of 99.2% on MNIST, 86.7% on CIFAR-10, and 91.3% on IMDB. The method implemented according to the embodiment of the present disclosure also achieved a minimum loss of 0.03 on MNIST, a minimum loss of 0.41 on CIFAR-10, and a minimum loss of 0.22 on IMDB. The results show that the method implemented according to the embodiment of the present disclosure can effectively adapt to the learning rate during training, thereby improving the convergence and performance of machine learning tasks.

[0066]

[0067]

[0068] Table 1

[0069] In some embodiments, the performance of the method implemented according to the embodiments of the present disclosure under different hyperparameter settings can also be evaluated. The method implemented according to the embodiments of the present disclosure can be compared with existing adaptive learning rate selection methods (such as AdaGrad, Adam, and cyclic learning rates). One of the three benchmark data sets mentioned above, MNIST, is used. The following hyperparameters are adjusted: the discount factor (γ) and exploration rate (∈) of the reinforcement learning model, and the number of hidden layers (L) and neurons (N) of the machine learning model. Different combinations of these hyperparameters can be tested on the validation set using grid search, and the best combination is selected based on the accuracy index. Other hyperparameters can be fixed to their default values, as shown in Table 2 below for the default hyperparameters for the method implemented according to the embodiments of the present disclosure.

[0070] Hyperparameters Numeric Learning rate (α) 0.01 Batch size 32 Activation Function ReLU Loss Function Cross Entropy

[0071] Table 2

[0072] Table 3 below shows the accuracy and loss of the method implemented according to the embodiment of the present disclosure and the baseline of each data set test set, as well as the training time and the number of epochs required to reach convergence, and the hyperparameter adjustment results.

[0073]

[0074] In order to evaluate the effectiveness of the method implemented according to the embodiment of the present disclosure, experiments can be conducted on multiple benchmark datasets of image classification tasks using the method implemented according to the embodiment of the present disclosure. The results are compared with the results obtained using traditional methods (such as fixed learning rate and step-decay learning rate). The experimental results show the following improvements: faster convergence, the method implemented according to the embodiment of the present disclosure shows that the number of iterations required for convergence is reduced by an average of 10% compared with the traditional method; improved accuracy, compared with the traditional method on the benchmark dataset, the average accuracy of the method implemented according to the embodiment of the present disclosure is improved by 3%; stable performance, compared with the traditional method, the method implemented according to the embodiment of the present disclosure shows more stable performance under various hyperparameter settings.

[0075] The above experimental results show that compared with the traditional method of learning rate adaptation in machine learning, the method implemented according to the embodiment of the present disclosure provides improved and more stable performance. The method of the embodiment of the present disclosure combines the advantages of reinforcement learning and machine learning models to provide a smarter and more effective method to adjust the learning rate. The method of the embodiment of the present disclosure also introduces a reward function to measure the improvement of the loss function and guide the learning rate update. The method of the embodiment of the present disclosure also designs a policy network to predict the future value of the learning rate based on the current loss and weight. In addition, the method of the embodiment of the present disclosure can also use machine learning to directly update parameters, thereby improving speed.

[0076] In summary, the method implemented according to the embodiment of the present disclosure significantly improves the traditional method by combining reinforcement learning and machine learning models to adjust the learning rate during machine learning training. The method implemented according to the embodiment of the present disclosure predicts the future value of the learning rate and adjusts it during the training process, thereby achieving faster convergence and better model performance. The experimental results demonstrate the effectiveness of the proposed solution, which provides a smarter and more efficient way to adapt the learning rate in machine learning. The combination of these two methods provides a desired direction for future research and development in this field.

[0077] Figure 4 A schematic block diagram of an example device 400 that may be used to implement embodiments of the present disclosure is shown. Figure 1 The computing device in the embodiment can be implemented by using device 400. As shown in the figure, device 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 402 or computer program instructions loaded from a storage unit 408 to a random access memory (RAM) 403. In RAM 403, various programs and data required for the operation of storage device 400 can also be included. CPU 401, ROM 402 and RAM 403 are connected to each other via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0078] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage page 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0079] The various processes and processing described above, such as method 200, may be performed by processing unit 401. For example, in some embodiments, method 200 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by CPU 401, one or more actions of method 200 described above may be performed.

[0080] The present disclosure may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0081] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination of the above. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0082] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0083] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0084] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0085] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0086] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0087] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0088] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating a machine learning model, the method comprising: Extracting a plurality of parameters from a target machine learning model, the plurality of parameters comprising a learning rate, state information, a loss value, a gradient, and a weight, the target machine learning model being used to perform a task related to at least one of an image, a video, a voice, and a text; Based on the plurality of parameters, predicting, by the first machine learning model, a first learning rate associated with the target machine learning model; Based on the plurality of parameters, predicting, by a second machine learning model, a second learning rate associated with the target machine learning model; Selecting a learning rate having a minimum loss value between the first learning rate and the second learning rate based on the first learning rate and the second learning rate; as well as The target machine learning model is adjusted based on the learning rate with the minimum loss value.

2. The method according to claim 1, wherein selecting a learning rate having a minimum loss value between the first learning rate and the second learning rate based on the first learning rate and the second learning rate comprises: Compare the first learning rate and the second learning rate with a target benchmark learning rate respectively, so as to select a learning rate having the smallest difference with the target benchmark learning rate; and The target baseline learning rate has been determined using a grid search method.

3. The method according to claim 1, further comprising: The first learning rate associated with the target machine learning model is predicted by the first machine learning model based on the state information and the loss value of the target machine learning model in the multiple parameters.

4. The method according to claim 1, further comprising: The second learning rate associated with the first machine model is predicted by the second machine learning model based on the weights and the loss value of the target machine model in the plurality of parameters.

5. The method according to claim 1, further comprising: Based on the first learning rate being determined as the learning rate with the minimum loss value, adjusting the second machine learning model, the adjustment comprising: By adjusting the sample data and the number of sample data used by the second machine learning model to predict the second learning rate, the difference between the second learning rate generated by the second machine learning model and the target baseline learning rate is reduced.

6. The method according to claim 1, further comprising: Based on the second learning rate being determined as the learning rate with the minimum loss value, adjusting the first machine learning model, the adjustment comprising: The first machine learning model is adjusted by one or more methods including Q learning method and / or policy gradient method to reduce the difference between the first learning rate generated by the first machine learning model and the target baseline learning rate.

7. The method according to claim 6, further comprising: In response to a decrease in the difference between the first learning rate generated by the first machine learning model and the target baseline learning rate, obtaining a reward value associated with the first learning rate; as well as The first machine learning model is adjusted based on the reward value.

8. The method according to claim 1, further comprising: Adjust one of the first machine learning model or the second machine learning model in each iteration of multiple iterative adjustments of the target machine learning model.

9. The method according to claim 1, further comprising: Based on the multiple parameters extracted from the target machine learning model, the first machine learning model and the second machine learning model respectively predict the momentum, weight decay, dropout rate, and batch size for the target machine learning model with the minimum loss value.

10. An electronic device, comprising: processor; as well as A memory is coupled to the processor and stores instructions that, when executed by the processor, cause the device to perform the following actions: Extracting multiple parameters from a target machine learning model, the multiple parameters including a learning rate, state information, a loss value, a gradient, and a weight, the target machine learning model being used to perform a task related to at least one of an image, a video, a voice, and a text; Based on the plurality of parameters, predicting, by the first machine learning model, a first learning rate associated with the target machine learning model; Based on the plurality of parameters, predicting, by a second machine learning model, a second learning rate associated with the target machine learning model; Selecting a learning rate having a minimum loss value between the first learning rate and the second learning rate based on the first learning rate and the second learning rate; as well as The target machine learning model is adjusted based on the learning rate with the minimum loss value.

11. The electronic device of claim 10, wherein selecting a learning rate having a minimum loss value between the first learning rate and the second learning rate based on the first learning rate and the second learning rate comprises: Compare the first learning rate and the second learning rate with a target benchmark learning rate respectively, so as to select a learning rate having the smallest difference with the target benchmark learning rate; and The target baseline learning rate has been determined using a grid search method.

12. The electronic device according to claim 10, further comprising: The first learning rate associated with the target machine learning model is predicted by the first machine learning model based on the state information and the loss value of the target machine learning model in the multiple parameters.

13. The electronic device according to claim 10, further comprising: The second learning rate associated with the first machine model is predicted by the second machine learning model based on the weights and the loss value of the target machine model in the plurality of parameters.

14. The electronic device according to claim 10, further comprising: Based on the first learning rate being determined as the learning rate with the minimum loss value, adjusting the second machine learning model, the adjustment comprising: By adjusting the sample data and the number of sample data used by the second machine learning model to predict the second learning rate, the difference between the second learning rate generated by the second machine learning model and the target baseline learning rate is reduced.

15. The electronic device according to claim 10, further comprising: Based on the second learning rate being determined as the learning rate with the minimum loss value, adjusting the first machine learning model, the adjustment comprising: The first machine learning model is adjusted by one or more methods including Q learning method and / or policy gradient method to reduce the difference between the first learning rate generated by the first machine learning model and the target baseline learning rate.

16. The electronic device according to claim 15, further comprising: In response to a decrease in the difference between the first learning rate generated by the first machine learning model and the target baseline learning rate, obtaining a reward value associated with the first learning rate; as well as The first machine learning model is adjusted based on the reward value.

17. The electronic device according to claim 10, further comprising: Adjust one of the first machine learning model or the second machine learning model in each iteration of multiple iterative adjustments of the target machine learning model.

18. The electronic device according to claim 10, further comprising: Based on the multiple parameters extracted from the target machine learning model, the first machine learning model and the second machine learning model respectively predict the momentum, weight decay, dropout rate, and batch size for the target machine learning model with the minimum loss value.

19. A computer program product tangibly stored on a non-transitory computer readable storage medium and comprising computer executable instructions that, when executed, cause a computer to: Extracting multiple parameters from a target machine learning model, the multiple parameters including a learning rate, state information, a loss value, a gradient, and a weight, the target machine learning model being used to perform a task related to at least one of an image, a video, a voice, and a text; Based on the plurality of parameters, predicting, by the first machine learning model, a first learning rate associated with the target machine learning model; Based on the plurality of parameters, predicting, by a second machine learning model, a second learning rate associated with the target machine learning model; Selecting a learning rate having a minimum loss value between the first learning rate and the second learning rate based on the first learning rate and the second learning rate; as well as The target machine learning model is adjusted based on the learning rate with the minimum loss value.

20. The computer program product of claim 19, wherein selecting a learning rate having a minimum loss value among the first learning rate and the second learning rate based on the first learning rate and the second learning rate comprises: Compare the first learning rate and the second learning rate with a target benchmark learning rate respectively, so as to select a learning rate having the smallest difference with the target benchmark learning rate; and The target baseline learning rate has been determined using a grid search method.

Images