Machine learning tuning method, device, equipment and medium
By building a deep learning model, using preset training data to train and verify the model with different hyperparameter groups, iterative optimization process, the problem of inappropriate selection of hyperparameters in the deep learning model is solved, and the model performance is improved and generalized Improvement of abilities.
Patent Information
- Application Number
- CN202410319364.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-20
- Publication Date
- 2025-01-07
AI Technical Summary
There are inappropriate problems in the selection of hyperparameters in deep learning models, which makes it difficult to optimize the model performance.
By building a deep learning model, using preset training data to train and verify the model with different hyperparameter groups, adjust the hyperparameter group according to the optimization score, retrain the model, iterate the optimization process until the preset requirements are met.
Quickly finding the appropriate hyperparameter combination improves the performance and generalization ability of the model, and solves the problem of inappropriate hyperparameter selection.
Smart Images
Figure CN119272848A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field related to deep learning, and specifically to a machine learning tuning method, device, equipment and medium. Background Art
[0002] Machine learning is based on models. Due to the complexity of deep learning models, building models according to book knowledge often results in "ideals are full, but reality is very skinny", and the model may have problems with inappropriate hyperparameter selection. Summary of the invention
[0003] In view of this, the embodiments of the present application are committed to providing a machine learning tuning method, apparatus, device and medium.
[0004] The first aspect of the present application provides a machine learning tuning method, comprising:
[0005] Build deep learning models;
[0006] Based on the preset training data, the deep learning model is trained according to different preset numbers of hyperparameter groups to obtain multiple target models;
[0007] Verify each target model separately to obtain the optimization score of each target model;
[0008] Adjusting the preset number of hyperparameter groups based on the optimization score;
[0009] Based on the preset training data, the deep learning model is retrained according to different adjusted hyperparameter groups to obtain multiple target models;
[0010] Verify each target model separately to obtain the optimization score of each target model;
[0011] If there is no target model that meets the first preset requirement, repeat the step of adjusting the preset number of hyperparameter groups based on the optimization score.
[0012] In some embodiments, based on the correspondence between the optimization score and the hyperparameter group, a plurality of parameters of the hyperparameter group closely related to the optimization score are determined;
[0013] Multiple parameters that are closely related to the optimization score form a target hyperparameter group.
[0014] In some embodiments, the method further includes: training the deep learning model based on the preset training data according to different preset numbers of target hyperparameter groups to obtain multiple target models;
[0015] Verify each target model separately to obtain the optimization score of each target model;
[0016] Adjusting the preset number of target hyperparameter groups based on the optimization score;
[0017] Based on the preset training data, the deep learning model is retrained according to different adjusted target hyperparameter groups to obtain multiple target models;
[0018] Verify each target model separately to obtain the optimization score of each target model;
[0019] If there is no target model that meets the second preset requirement, repeat the step of adjusting the preset number of target hyperparameter groups based on the optimization score.
[0020] In some embodiments, it also includes:
[0021] The deep learning model is trained based on the preset training data according to different preset numbers of hyperparameter groups to obtain multiple target models; each target model is verified to obtain an optimization score of each target model, including:
[0022] For each set of hyperparameters, the deep learning model is trained K times according to the hyperparameters to obtain k target models in a K-fold cross-validation manner;
[0023] Based on K-fold cross validation, k target models are validated respectively to obtain the optimization score;
[0024] Among them, the mean of the optimization scores of the k target models corresponding to the hyperparameter group is, and the hyperparameter group corresponds to the optimization score.
[0025] In some embodiments, adjusting the preset number of hyperparameter groups based on the optimization score includes:
[0026] The preset number of hyperparameter groups are adjusted based on the optimization score according to the gradient descent method.
[0027] In some embodiments, it also includes:
[0028] Get the original data;
[0029] The original data is preprocessed to obtain training data.
[0030] In some embodiments, it also includes:
[0031] If the target model that meets the first preset requirement is still not obtained after the preset time, the deep learning model is rebuilt based on a data architecture that has not been used before.
[0032] A second aspect of the present application provides a machine learning tuning device, comprising:
[0033] Building blocks for building deep learning models;
[0034] A training module, used to train the deep learning model based on preset training data according to different preset numbers of hyperparameter groups to obtain multiple target models;
[0035] The verification module is used to verify each target model separately and obtain the optimization score of each target model;
[0036] An adjustment module, configured to adjust the preset number of hyperparameter groups based on the optimization score;
[0037] A training module, for retraining the deep learning model based on preset training data according to different adjusted hyperparameter groups to obtain multiple target models;
[0038] The verification module is used to verify each target model separately to obtain the optimization score of each target model; if there is no target model that meets the first preset requirement, the step of repeatedly executing is to adjust the preset number of hyperparameter groups based on the optimization score.
[0039] A third aspect of the present application provides an electronic device, including:
[0040] A processor, and a memory for storing a program executable by the processor;
[0041] The processor is used to implement the machine learning tuning method provided in any of the above embodiments by running the program in the memory.
[0042] A fourth aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes a machine learning tuning method as provided in any of the above embodiments.
[0043] The present application provides a machine learning tuning method, which constructs a deep learning model; based on the preset training data, the deep learning model is trained according to different preset numbers of hyperparameter groups to obtain multiple target models; each target model is verified to obtain the optimization score of each target model; the preset number of hyperparameter groups are adjusted based on the optimization score; based on the preset training data, the deep learning model is retrained according to different adjusted hyperparameter groups to obtain multiple target models; each target model is verified to obtain the optimization score of each target model; if there is no target model that meets the first preset requirement, the step of adjusting the preset number of hyperparameter groups based on the optimization score is repeated. The scheme provided by the present application can quickly find suitable hyperparameters to tune machine learning. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0045] Figure 1 It is a flowchart of a machine learning tuning method provided by an embodiment of the present application.
[0046] Figure 2 It is a schematic diagram of a gradient algorithm provided by an embodiment of the present application.
[0047] Figure 3 It is a structural diagram of a machine learning tuning device provided in one embodiment of the present application.
[0048] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] Application Overview
[0051] At present, machine learning is based on models. Due to the complexity of deep learning models, building models according to book knowledge often results in "ideals are full, but reality is very skinny", and the models may have problems with inappropriate hyperparameter selection.
[0052] In order to solve the above problems, the present application provides a solution, firstly constructing a deep learning model; based on the preset training data, the deep learning model is trained according to different preset numbers of hyperparameter groups to obtain multiple target models; each target model is verified to obtain the optimization score of each target model; based on the optimization score, the preset number of hyperparameter groups are adjusted; based on the preset training data, the deep learning model is retrained according to different adjusted hyperparameter groups to obtain multiple target models; each target model is verified to obtain the optimization score of each target model; if there is no target model that meets the first preset requirement, repeating the step of adjusting the preset number of hyperparameter groups based on the optimization score. The solution provided by the present application can quickly find suitable hyperparameters to tune machine learning.
[0053] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.
[0054] Exemplary Methods
[0055] Figure 1 FIG. 1 is a flow chart of a machine learning tuning method provided by an embodiment of the present application. Figure 1 As shown, the method includes the following contents.
[0056] S101, building a deep learning model;
[0057] Specifically, you first need to design and implement a deep learning model. This includes choosing a suitable network architecture (such as convolutional neural network CNN, recurrent neural network RNN, Transformer, etc.), defining the number of layers of the model and the type of each layer (fully connected layer, convolutional layer, pooling layer, etc.), setting the activation function, and selecting the loss function and optimizer.
[0058] S102, training the deep learning model based on the preset training data according to different preset numbers of hyperparameter groups to obtain multiple target models;
[0059] In practical applications, you can use a preset training dataset to train your deep learning model. Try different hyperparameter combinations, which may include learning rate, batch size, regularization parameter, number of network layers, number of nodes per layer, etc. Each hyperparameter combination will produce a different target model.
[0060] S103, verifying each target model respectively to obtain an optimization score of each target model;
[0061] The solution provided in this application evaluates the performance of each trained model on an independent validation set. Specifically, it involves calculating the accuracy, recall, F1 score or other related indicators of the model. These indicators will be used as optimization scores for subsequent hyperparameter adjustment.
[0062] S104, adjusting the preset number of hyperparameter groups based on the optimization score;
[0063] Based on the optimization scores obtained through verification, analyze which hyperparameter combinations produce better model performance. Then, adjust these hyperparameters to try to further improve the performance of the model. In practical applications, this can be fine-tuning existing hyperparameters or trying completely new hyperparameter combinations.
[0064] S105, retraining the deep learning model based on the preset training data according to different adjusted hyperparameter groups to obtain multiple target models;
[0065] Specifically, the model is trained again using the adjusted hyperparameter group to verify whether the new hyperparameter settings can indeed improve the performance of the model.
[0066] S106, verifying each target model respectively to obtain an optimization score for each target model;
[0067] If there is no target model that meets the first preset requirement, repeat the step of adjusting the preset number of hyperparameter groups based on the optimization score.
[0068] If after one or more rounds of hyperparameter tuning and model retraining, no model reaches the preset performance requirements, then the process needs to be iterated to further adjust the hyperparameters or reconsider the choice of model architecture.
[0069] The solution provided in this application is an iterative process, the purpose of which is to find the best hyperparameter combination through continuous experimentation and evaluation, so as to train a deep learning model with optimal performance.
[0070] In some embodiments, it also includes:
[0071] Based on the corresponding relationship between the optimization score and the hyperparameter group, multiple parameters of the hyperparameter group that are closely related to the optimization score are determined; and the multiple parameters that are closely related to the optimization score constitute a target hyperparameter group.
[0072] Specifically, during the hyperparameter optimization process of a deep learning model, a series of hyperparameter combinations and their corresponding optimization scores (such as accuracy and loss values on the validation set) are collected. These scores reflect the performance of the model under different hyperparameter settings. In order to find the best hyperparameter combination, it is necessary to analyze the correspondence between the optimization score and the hyperparameters, and determine which hyperparameters have the greatest impact on the model performance. The following are the detailed steps:
[0073] Collect hyperparameter and optimization score data: During training and validation, record each hyperparameter combination and its corresponding optimization score. This data is usually stored in a table or database for analysis.
[0074] Analyze the correspondence: Use statistical methods (such as correlation analysis, regression analysis) to evaluate the relationship between different hyperparameters and the optimization score. This helps identify which hyperparameters have a significant impact on model performance.
[0075] Identify key parameters: Based on the analysis results, identify the hyperparameters that are closely related to the optimization score. These parameters may include learning rate, batch size, regularization coefficient, number of network layers, etc. Key parameters are those that have a significant positive or negative impact on the model performance.
[0076] Construct a target hyperparameter group: Combine these key parameters into a new hyperparameter group that aims to optimize model performance by adjusting these key parameters. The target hyperparameter group may contain multiple candidate combinations, each of which is tuned for the key parameters.
[0077] Further optimization: Train and validate the model using the target set of hyperparameters. If the new hyperparameter combination improves model performance, the process can continue to further fine-tune these key parameters. If performance does not improve significantly, other potential key parameters may need to be re-evaluated.
[0078] Iterative process: Hyperparameter optimization is an iterative process. In each iteration, new parameters that are closely related to the optimization score may be discovered. Therefore, it may be necessary to repeat the above steps many times until a hyperparameter combination that meets the preset performance requirements is found.
[0079] In this way, it is possible to more specifically adjust those hyperparameters that have the greatest impact on model performance, rather than blindly adjusting all possible parameters. This not only improves optimization efficiency, but also helps to better understand how different hyperparameters affect model performance. In practice, this process can be achieved through automated tools (such as Hyperopt, Optuna, etc.), which can automatically perform hyperparameter search and optimization.
[0080] Furthermore, to select the target hyperparameter group, repeated iteration is required. The specific iteration method is as follows:
[0081] Based on the preset training data, the deep learning model is trained according to different preset numbers of target hyperparameter groups to obtain multiple target models;
[0082] Verify each target model separately to obtain the optimization score of each target model;
[0083] Adjusting the preset number of target hyperparameter groups based on the optimization score;
[0084] Based on the preset training data, the deep learning model is retrained according to different adjusted target hyperparameter groups to obtain multiple target models;
[0085] Verify each target model separately to obtain the optimization score of each target model;
[0086] If there is no target model that meets the second preset requirement, repeat the step of adjusting the preset number of target hyperparameter groups based on the optimization score.
[0087] The above process is an iterative hyperparameter optimization process, which aims to find the best hyperparameter combination through repeated training, verification, and adjustment.
[0088] In some embodiments, the deep learning model is trained based on the preset training data according to different preset numbers of hyperparameter groups to obtain multiple target models; each target model is verified to obtain an optimization score of each target model, including:
[0089] For each set of hyperparameters, the deep learning model is trained K times according to the hyperparameters to obtain k target models in a K-fold cross-validation manner;
[0090] Based on K-fold cross validation, k target models are validated respectively to obtain the optimization score;
[0091] Among them, the mean of the optimization scores of the k target models corresponding to the hyperparameter group is, and the hyperparameter group corresponds to the optimization score.
[0092] With this setting, we combine K-fold cross validation to perform hyperparameter optimization. Through multiple training and validation, we evaluate the performance of different hyperparameter combinations and reduce the model's dependence on specific data subsets, thereby improving the model's generalization ability. The specific process is as follows:
[0093] Select a set of hyperparameters: Determine the initial values for a set or hyperparameters, which may include learning rate, batch size, regularization coefficients, etc.
[0094] K-fold cross-validation training: Split the preset training data set into K subsets of equal size (or as equal as possible). For each set of hyperparameters, perform K training times, each time using a different K-1 subset as the training set and the remaining subset as the validation set.
[0095] K-fold validation: In each K-fold cross validation, the performance of the trained model is evaluated on the validation set. The optimization score of each validation is recorded. These scores can be accuracy, loss value or other performance indicators.
[0096] Calculate the mean optimization score: For each set of hyperparameters, calculate the mean of the optimization scores obtained by its K validations. This mean represents the average performance of the hyperparameter group on different data subsets.
[0097] Evaluate hyperparameter groups: Compare the mean optimization scores of different hyperparameter groups to evaluate their relative performance. Select the hyperparameter group with the best mean optimization score as the current optimal hyperparameter configuration.
[0098] Iterative optimization: If the current optimal hyperparameter group does not meet the preset performance requirements, you can fine-tune it and repeat the above steps. This process can be iterated until a hyperparameter configuration that meets the requirements is found.
[0099] In some embodiments, adjusting the preset number of hyperparameter groups based on the optimization score includes:
[0100] The preset number of hyperparameter groups are adjusted based on the optimization score according to the gradient descent method. The core content of machine learning is to feed data to an artificially designed model, and then let the model automatically "learn" to optimize the various parameters of the model itself, so that the model can best match the learning task under a certain set of parameters. Then this "learning" process is the key to the machine learning algorithm. The gradient descent method is one of the most common ways to implement the "learning" process, especially in deep learning (neural network) models. The core of the BP back propagation method is to continuously use gradient descent to optimize the weight parameters of each layer.
[0101] Gradient descent is a commonly used first-order optimization method and one of the simplest and most classical methods for solving unconstrained optimization problems. Let's consider an unconstrained optimization problem min x , f(x), where f(x) is a continuously differentiable function. If we can construct a sequence x 0 , x 1 , x 2 ,... and satisfy:
[0102] f(x t+1 ) < f(x t ), t = 0, 1, 2,...
[0103] Then we can continuously execute this process to converge to a local minimum point. For reference, see Figure 2 .
[0104] The problem is how to find the next point x t+1 , and ensure that f(x t+1 ) < f(x t )? Suppose the current form of the function f(x) is Figure 2 in shape. Now we randomly find an initial point x1. For a univariate function, the function value only changes with the change of t+1 . Then we design the next x t to be obtained by taking a small step Δx along a certain direction from the previous x t . The key problem here is: which direction is this small step towards?
[0105] For a univariate function, there are two directions for x: either the positive direction (Δx > 0) or the negative direction (Δx < 0). To choose the direction of each step, we need to use the famous Taylor formula. First, look at the following Taylor expansion:
[0106]
[0107] The left side is the next position after the current x moves a small step Δx, and it is approximately equal to the right side. As we said before, the key problem is to find a direction such that f(x + Δx) < f(x). Then according to the above Taylor expansion, obviously we need to ensure:
[0108] We can choose to make:
[0109]
[0110] where the step size α is a small positive number. Thus:
[0111]
[0112] Since the square of any non-zero number is greater than 0, it is guaranteed that Δx▽f(x)<0.
[0113] Thus, setting:
[0114]
[0115] It can guarantee:
[0116] f(x+Δx) <f(x)
[0117] Then the calculation method for updating x is very simple. You can update x according to the following formula
[0118]
[0119] This is called taking a small step in the direction of the negative gradient.
[0120] In some embodiments, the method further includes: obtaining raw data; and preprocessing the raw data to obtain training data. The specific process is as follows:
[0121] Get the raw data:
[0122] Raw data may come from a variety of sources, including databases, files (such as CSV, Excel, JSON, etc.), online APIs, log files, or data scraped from websites using crawlers.
[0123] Data may exist in structured or unstructured form. Structured data is usually easier to process, while unstructured data (e.g., text, images) may require additional preprocessing steps.
[0124] Data cleaning:
[0125] Handling missing values: Identify and handle missing values in your data. This may include removing records with missing values, filling in missing values (e.g., using the mean, median, mode, or more complex interpolation methods), or using a model to predict missing values.
[0126] Removing outliers: Identify and handle outliers, which may be due to errors or other atypical situations. You can use statistical methods (such as IQR, Z-score) to identify outliers and decide whether to remove them or make corrections.
[0127] Data conversion:
[0128] Normalization / Standardization: Scaling data to a uniform range or distribution to eliminate the effects of different magnitudes and dimensions. Normalization usually scales the data to the range [0,1], while standardization converts the data to a distribution with a mean of 0 and a standard deviation of 1.
[0129] Encoding: For categorical data, it needs to be converted into numerical form. One-Hot Encoding is suitable for unordered categorical variables, while Label Encoding is suitable for ordered categorical variables.
[0130] Feature Engineering:
[0131] Feature selection: Select the most informative features from the original features to reduce the complexity of model training and improve performance. You can use correlation analysis, feature importance assessment and other methods to select features.
[0132] Feature extraction: Increase the information content of data by combining existing features or creating new features. For example, using TF-IDF in text data, or using convolutional neural networks to extract features in image data.
[0133] Dataset division:
[0134] Divide the dataset into training, validation, and test sets. This helps in evaluating the performance of the model at different stages and prevents overfitting.
[0135] Data augmentation (optional):
[0136] For some types of data (such as images or text), data augmentation can be used to increase the diversity of the data, such as rotating, scaling, and cropping images, or performing synonym replacement on text.
[0137] Data Storage:
[0138] Save the preprocessed data in an appropriate format for subsequent model training and evaluation. Common formats include CSV, HDF5, Pandas DataFrame, etc.
[0139] Through these steps, the raw data is converted into a format suitable for machine learning model training, namely training data. Preprocessing not only improves data quality, but also helps the model converge faster and improves its generalization ability. In practical applications, data preprocessing is an iterative process, and the preprocessing strategy may need to be continuously adjusted according to the results of model training.
[0140] Furthermore, if the target model that meets the first preset requirement is still not obtained after the preset period of time, the deep learning model is rebuilt based on a data architecture that has not been used.
[0141] Specifically, when the target model that meets the first preset requirements cannot be obtained within the preset time, this usually means that the current model architecture may not be suitable for solving the problem at hand, or there may be other problems, such as data quality issues, overfitting during training, resource limitations, etc. In this case, it is a reasonable step to rebuild the deep learning model based on the unused data architecture. The following is a detailed description of this process:
[0142] Analyze the current model:
[0143] Before trying a new architecture, first analyze the performance of the current model. Determine why the model is not performing as expected. This may involve examining metrics such as loss function, precision, recall, and the model’s performance on a validation set.
[0144] Explore new data architectures:
[0145] Consider using different network architectures. This might include changing the network’s depth (number of layers), width (number of units per layer), connectivity pattern (fully connected, convolutional, recurrent, etc.), or trying different activation functions and regularization techniques.
[0146] Redesign the model: Design a deep learning model based on a new data architecture. This can be building a model from scratch or modifying an existing model.
[0147] Retrain the model: Use the preprocessed training dataset to train the newly designed model. During the training process, you may need to adjust hyperparameters such as learning rate, batch size, and the choice of optimizer.
[0148] Validate the new model: Evaluate the performance of the new model on the validation set. Compare the performance of the new model with the old model to determine whether the new architecture has improved performance.
[0149] Iterative optimization: If the new model still does not meet the expected requirements, it may be necessary to further adjust the model architecture or hyperparameters. This process may require multiple iterations until a model that meets the preset requirements is found.
[0150] It should be noted that if you still cannot obtain a satisfactory model after trying multiple architectures, you may need to consider other factors, such as data enhancement, data quality improvement, and increased computing resources.
[0151] Exemplary Devices
[0152] The device embodiments of the present application can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.
[0153] Figure 3FIG. 1 is a block diagram of a machine learning optimization device provided by an embodiment of the present application. Figure 3 As shown, the machine learning tuning device includes:
[0154] A construction module 31 is used to construct a deep learning model;
[0155] A training module 32 is used to train the deep learning model based on preset training data according to different preset numbers of hyperparameter groups to obtain multiple target models;
[0156] A verification module 33 is used to verify each target model and obtain an optimization score of each target model;
[0157] An adjustment module 34, configured to adjust the preset number of hyperparameter groups based on the optimization score;
[0158] A training module 35 is used to retrain the deep learning model based on preset training data according to different adjusted hyperparameter groups to obtain multiple target models;
[0159] The verification module 36 is used to verify each target model separately to obtain the optimization score of each target model; if there is no target model that meets the first preset requirement, repeat the step of adjusting the preset number of hyperparameter groups based on the optimization score.
[0160] Exemplary Electronic Devices
[0161] Below, reference Figure 4 To describe an electronic device according to an embodiment of the present application. Figure 4 A block diagram of an electronic device according to an embodiment of the present application is illustrated.
[0162] like Figure 4 As shown, electronic device 400 includes one or more processors 410 and memory 420 .
[0163] The processor 410 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.
[0164] The memory 420 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 410 may run the program instructions to implement the machine learning tuning method of each embodiment of the present application described above and / or other desired functions. Various contents such as category correspondences may also be stored in the computer-readable storage medium.
[0165] In one example, the electronic device 400 may further include: an input device 430 and an output device 440 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0166] In addition, the input device 430 may also include, for example, a keyboard, a mouse, an interface, etc. The output device 440 may output various information to the outside, including analysis results, etc. The output device 440 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.
[0167] Of course, to simplify, Figure 4 Only some of the components in the electronic device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application conditions.
[0168] Exemplary computer program products and computer-readable storage media
[0169] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the machine learning tuning method according to various embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.
[0170] The computer program product may be written in any combination of one or more programming languages to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.
[0171] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the machine learning tuning method according to various embodiments of the present application described in the above "Exemplary Method" section of this specification.
[0172] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0173] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. A machine learning tuning method, characterized in that: include: Build deep learning models; Based on the preset training data, the deep learning model is trained according to different preset numbers of hyperparameter groups to obtain multiple target models; Verify each target model separately to obtain the optimization score of each target model; Adjusting the preset number of hyperparameter groups based on the optimization score; Based on the preset training data, the deep learning model is retrained according to different adjusted hyperparameter groups to obtain multiple target models; Verify each target model separately to obtain the optimization score of each target model; If there is no target model that meets the first preset requirement, repeat the step of adjusting the preset number of hyperparameter groups based on the optimization score.
2. The machine learning tuning method according to claim 1, characterized in that: Also includes: Based on the correspondence between the optimization score and the hyperparameter group, determining a plurality of parameters of the hyperparameter group that are closely related to the optimization score; Multiple parameters that are closely related to the optimization score form a target hyperparameter group.
3. The machine learning tuning method according to claim 2, characterized in that: Also includes: Based on the preset training data, the deep learning model is trained according to different preset numbers of target hyperparameter groups to obtain multiple target models; Verify each target model separately to obtain the optimization score of each target model; Adjusting the preset number of target hyperparameter groups based on the optimization score; Based on the preset training data, the deep learning model is retrained according to different adjusted target hyperparameter groups to obtain multiple target models; Verify each target model separately to obtain the optimization score of each target model; If there is no target model that meets the second preset requirement, repeat the step of adjusting the preset number of target hyperparameter groups based on the optimization score.
4. The machine learning tuning method according to claim 2, characterized in that: Also includes: The deep learning model is trained based on the preset training data according to different preset numbers of hyperparameter groups to obtain multiple target models; Verify each target model separately and obtain the optimization score of each target model, including: For each set of hyperparameters, the deep learning model is trained K times according to the hyperparameters to obtain k target models in a K-fold cross-validation manner; Based on K-fold cross validation, k target models are validated respectively to obtain the optimization score; Among them, the mean of the optimization scores of the k target models corresponding to the hyperparameter group is, and the hyperparameter group corresponds to the optimization score.
5. The machine learning tuning method according to claim 2, characterized in that: The adjusting the preset number of hyperparameter groups based on the optimization score includes: The preset number of hyperparameter groups are adjusted based on the optimization score according to the gradient descent method.
6. The machine learning tuning method according to claim 1, characterized in that: Also includes: Get the original data; The original data is preprocessed to obtain training data.
7. The machine learning tuning method according to claim 1, characterized in that: Also includes: If the target model that meets the first preset requirement is still not obtained after the preset time, the deep learning model is rebuilt based on a data architecture that has not been used before.
8. A machine learning tuning device, characterized in that: include: Building blocks for building deep learning models; A training module, used to train the deep learning model based on preset training data according to different preset numbers of hyperparameter groups to obtain multiple target models; The verification module is used to verify each target model separately and obtain the optimization score of each target model; An adjustment module, configured to adjust the preset number of hyperparameter groups based on the optimization score; A training module, for retraining the deep learning model based on preset training data according to different adjusted hyperparameter groups to obtain multiple target models; The verification module is used to verify each target model separately and obtain the optimization score of each target model; If there is no target model that meets the first preset requirement, repeat the step of adjusting the preset number of hyperparameter groups based on the optimization score.
9. An electronic device, characterized in that: include: A processor, and a memory for storing a program executable by the processor; The processor is used to implement the machine learning tuning method as described in any one of claims 1 to 7 by running the program in the memory.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to perform the machine learning tuning method as described in any one of claims 1 to 7.