Image classification model training method and device based on automatic machine learning

Abstract

The invention discloses an image classification model training method and device based on automatic machine learning. The method comprises the following steps: acquiring a task image sample data set; dividing the image sample data set into a plurality of training sets and verification sets; selecting hyper-parameters from the hyper-parameter search space according to a plurality of preset classification algorithms, and establishing a candidate algorithm model; training each candidate algorithm model by using each training set, and verifying the trained model by using a verification set to obtain an algorithm evaluation index; according to the algorithm evaluation indexes, screening from the candidate algorithm models to obtain a to-be-trained algorithm model; training the to-be-trained algorithm model by using the plurality of training sets and verification sets, and obtaining a model evaluation index; performing hyper-parameter pruning according to the model evaluation indexes to obtain an optimal hyper-parameter range; performing hyper-parameter assignment on the algorithm model in training according to the optimized hyper-parameter range; and repeatedly executing the model training process until an iteration stopping condition is met, and obtaining an image classification model.

Description

Technical Field

[0001] The present application relates to an image classification model training method and device based on automatic machine learning. Background Art

[0002] In recent years, deep learning theory and technology have developed rapidly, and have been widely used in the field of oil and gas exploration and development, but there is still a long way to go before it can be truly put into production. Deep learning is a data-driven machine learning method. Machine learning algorithms can be divided into three categories: supervised, unsupervised, and semi-supervised.

[0003] In supervised learning, the label of the instance in the data is known. For classification problems, this label is the class of the instance. Commonly used techniques in machine learning include classification, clustering, association rules, etc. Among them, classification technology is one of the most important technologies in the field of data mining. It is the field with the longest history of research and is also the field with more thorough research. The so-called classification is to find a set of models that can describe the common characteristics of all data from known data so as to identify the category to which unknown data belongs. Therefore, the classification algorithm belongs to the supervised algorithm. Generally speaking, the classification process is divided into two parts, namely training and testing, that is, first training a classification model from the training data, and then applying the model to the test data. This classification model represents a collection of classification "knowledge", that is, the relationship between attribute values ​​and categories. For a given instance with unknown category, the classification model can predict the category it belongs to through its attribute values.

[0004] One of the main goals of a classification algorithm is to maximize the classification accuracy of the training model on the test data. To achieve this goal, different researchers have proposed hundreds of classification algorithms, including binary classification algorithms and multivariate classification algorithms.

[0005] At present, the academic community has accumulated a large number of classification algorithms, multiple strategies for decomposing multi-class classification problems into binary classification problems, and different integrated learning methods. However, these methods are huge in number, and people often do not know how to efficiently select methods that are more suitable for their data.

[0006] At the same time, the classification algorithm mainly contains two types of data variables during the construction process: parameters and hyperparameters. Parameters refer to internal variables whose values ​​can be estimated based on data, and they do not need to be set manually, such as support vectors in support vector machines. Hyperparameters are external variables whose values ​​cannot be estimated from data and need to be set manually by practitioners, such as the kernel function (kermel) in support vector machines and the learning rate (learning rate) in neural networks. That is, "setting parameters" often refers to setting the hyperparameters of the algorithm. Algorithms usually expose a large number of hyperparameters, and people can improve the performance of the model by adjusting the values ​​of hyperparameters.

[0007] In the past, people usually used manual methods to explore the search space of hyperparameters, but this method is only applicable to cases with fewer hyperparameters, such as support vector machines (SVMs) which have only 7 adjustable hyperparameters. In practical applications, this method is inadequate when the hyperparameter search space grows rapidly due to too many algorithm parameters. For example, the adjustable hyperparameters of convolutional neural networks (CNNs) increase with the number of connection layers, and can reach dozens or even hundreds of hyperparameters. In addition to complex algorithms, machine learning toolkits also provide a large number of learning algorithms and preprocessing methods for users to choose from. When the algorithm selection problem and hyperparameter optimization problem are considered together, the size of the search space is even more difficult to predict.

[0008] From the perspective of the optimization scope of the machine learning process, the automatic machine learning can be divided into five categories: feature selection-oriented, model selection-oriented, hyperparameter optimization-oriented, and partial range or full range-oriented. In addition, according to the different tasks of machine learning applications, automatic machine learning can be divided into supervised, semi-supervised, and unsupervised from the perspective of whether the data contains labels.

[0009] In-depth research on classification methods has great practical significance. In fact, studying how to efficiently select classification methods has become a long-term goal pursued by researchers, especially field experts. Summary of the invention

[0010] In order to better achieve image classification, the embodiments of the present application provide an image classification model training method, device, equipment and medium based on automatic machine learning.

[0011] In a first aspect, an embodiment of the present application provides an image classification model training method based on automatic machine learning, the method comprising:

[0012] Get a sample dataset of task images;

[0013] Dividing the image sample data set into multiple training sets and validation sets;

[0014] According to the preset multiple classification algorithms, hyperparameters are selected from the hyperparameter search space to establish a candidate algorithm model;

[0015] Use each training set to train each candidate algorithm model, and use the verification set to verify the trained model to obtain an algorithm evaluation index;

[0016] According to the algorithm evaluation index, the algorithm model to be trained is obtained by screening from each candidate algorithm model;

[0017] Using the multiple training sets and validation sets to train the algorithm model to be trained, and obtaining a model evaluation index;

[0018] Perform hyperparameter pruning according to the model evaluation index to obtain an optimal hyperparameter range;

[0019] Assigning hyperparameters to the algorithm model in training according to the preferred hyperparameter range;

[0020] The above model training process is repeated until the iteration stop condition is met to obtain the image classification model.

[0021] As one or some optional implementations of the embodiments of the present application, the hyperparameter pruning is performed according to the model evaluation index to obtain the optimal hyperparameter range, including:

[0022] During each algorithm model training process, the model evaluation indicators corresponding to each training set are used to determine whether the algorithm model in training is overfitting;

[0023] If yes, then according to the model evaluation index, the importance of each hyperparameter of the algorithm model is cyclically calculated to obtain the corresponding hyperparameter importance result;

[0024] Sort the hyperparameter importance results of each hyperparameter, and take the hyperparameters whose importance is greater than the first importance threshold as key hyperparameters, and take the hyperparameters whose importance is less than the second importance threshold as non-key hyperparameters; wherein the first importance threshold is greater than the second importance threshold;

[0025] Discard non-critical hyperparameters and prune each critical hyperparameter to obtain the optimal hyperparameter range.

[0026] As one or some optional implementations of the embodiments of the present application, each key hyperparameter is pruned in the following manner to obtain an optimal hyperparameter range:

[0027] Determine the type of each key hyperparameter:

[0028] For categorical hyperparameters, K-Means is used to cluster the configuration performance, retaining the elements in the cluster with high performance to obtain the corresponding optimal hyperparameter range;

[0029] For numerical hyperparameters, Spearman is used to perform correlation analysis between configuration performance and parameter value range, and selection is made based on the correlation to obtain the corresponding optimal hyperparameter range.

[0030] As one or some optional implementations of the embodiments of the present application, the step of selecting hyperparameters from a hyperparameter search space according to a plurality of preset classification algorithms and establishing a candidate algorithm model includes:

[0031] For each classification algorithm, the corresponding algorithm hyperparameter search space is determined in the hyperparameter search space;

[0032] Hyperparameters are selected from a corresponding algorithm hyperparameter search space to construct a candidate algorithm model corresponding to the classification algorithm.

[0033] As one or some optional implementations of the embodiment of the present application, the step of acquiring a task image sample dataset includes:

[0034] The acquired original image data set is subjected to data preprocessing to obtain a task image sample data set; wherein the data preprocessing includes missing value processing, data type judgment, outlier detection, and data segmentation.

[0035] As one or some optional implementations of the embodiment of the present application, the method further includes:

[0036] The task image sample data set is subjected to feature preprocessing to obtain a task image sample data set after feature construction; the feature preprocessing includes feature transformation and feature selection.

[0037] In a second aspect, an embodiment of the present application provides an image classification model training device based on automatic machine learning, the device comprising:

[0038] An acquisition module is used to obtain a task image sample dataset;

[0039] A division module, used for dividing the image sample data set into multiple training sets and verification sets;

[0040] Establishing a module for selecting hyperparameters from a hyperparameter search space and establishing a candidate algorithm model according to a plurality of preset classification algorithms;

[0041] A first training module is used to train each candidate algorithm model using each training set, and to verify the trained model using the verification set to obtain an algorithm evaluation index;

[0042] A screening module is used to screen the algorithm model to be trained from the candidate algorithm models according to the algorithm evaluation index;

[0043] A second training module is used to train the algorithm model to be trained using the multiple training sets and validation sets, and obtain a model evaluation index;

[0044] A pruning module, used to prune hyperparameters according to the model evaluation indicators to obtain an optimal hyperparameter range;

[0045] An assignment module, used for assigning hyperparameters to the algorithm model in training according to the preferred hyperparameter range;

[0046] The model determination module is used to determine whether the model meets the iteration stop condition and obtain the image classification model.

[0047] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned image classification model training method based on automatic machine learning.

[0048] In a fourth aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image classification model training method based on automatic machine learning as described above is implemented.

[0049] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer device, enables the computer device to execute the image classification model training method based on automatic machine learning as described above.

[0050] In a sixth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run a computer program or instructions to implement the image classification model training method based on automatic machine learning as described above.

[0051] The beneficial effects of the above technical solution provided by the embodiment of the present application include at least:

[0052] The image classification model training method based on automatic machine learning provided in the embodiment of the present application is to select hyperparameters from the hyperparameter search space according to multiple preset classification algorithms, establish a candidate algorithm model, and during the model development, multiple parallel tasks with different models and hyperparameters will be created. During the training process, the model will be evaluated at each iteration, and the model and hyperparameters will be optimized through the evaluation indicators to screen the algorithm model to be trained. During the training process of the algorithm model to be trained, the hyperparameter range is optimized through hyperparameter pruning, which can effectively reduce the cost of automatic machine learning training and make it cheaper for users to build image classification applications. At the same time, the automatic machine learning of classification algorithms reduces the threshold for the use of AI technology, helps model developers save modeling and training time, and allows them to have more energy to focus on the business itself, thereby promoting the implementation of AI in professional fields, greatly improving the industry penetration rate of artificial intelligence technology in the field of image processing, and providing strong support for digital transformation and intelligent development. Provide certain practical guidance for domain experts and actual developers on the selection of multivariate classification algorithms.

[0053] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings.

[0054] The technical solution of the present application is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are used to provide a further understanding of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings:

[0056] Figure 1 A flowchart of an image classification model training method based on automatic machine learning provided in an embodiment of the present application;

[0057] Figure 2 A schematic diagram of a machine learning pipeline and corresponding components provided for an embodiment of the present application;

[0058] Figure 3 A schematic diagram of the structure of an image classification model training device based on automatic machine learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0060] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0061] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0062] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0063] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0064] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0065] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0066] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0067] The inventors found that in the prior art, based on the premise that different hyperparameters in classification problems have different effects on model performance, the model can usually be gradually established by correcting the hyperparameters during the training process. The topology, such as the number of layers or dimensions, is adjusted during the training process. When searching for hyperparameters, the problem of too high a search space dimension is faced, which does not meet the inventors' expectations. Based on this, the inventors have made this application after further research and development, providing an image classification model training method, device, equipment and medium based on automatic machine learning.

[0068] Embodiment 1

[0069] The present application embodiment provides an image classification model training method based on automatic machine learning, referring to Figure 1 As shown, the method includes:

[0070] S101: Obtain a task image sample dataset;

[0071] S102: Divide the image sample data set into multiple training sets and validation sets;

[0072] S103: According to a plurality of preset classification algorithms, a hyperparameter is selected from a hyperparameter search space to establish a candidate algorithm model;

[0073] S104: Use each training set to train each candidate algorithm model, and use the verification set to verify the trained model to obtain an algorithm evaluation index;

[0074] S105: According to the algorithm evaluation index, the algorithm model to be trained is obtained from each candidate algorithm model;

[0075] S106: Using the multiple training sets and validation sets to train the algorithm model to be trained, and obtaining a model evaluation index;

[0076] S107: Perform hyperparameter pruning according to the model evaluation index to obtain an optimal hyperparameter range;

[0077] S108: assigning hyperparameters to the algorithm model in training according to the preferred hyperparameter range, and performing model training;

[0078] S109: Determine whether the model training meets the iteration stop condition, if yes, execute step S110; if no, re-execute the above steps S106 to S108;

[0079] S110: Obtain an image classification model.

[0080] The inventors found that the operation process of automatic machine learning shows that it searches for the best configuration by constantly exploring the search space. This is an iterative process constrained by computing power, time and cost. We learn knowledge from historical experiments and apply it to new experiments. Hyperparameter optimization in machine learning aims to find hyperparameters that make the machine learning algorithm perform best on the validation data set. Usually, there is no precise solution, so we focus on the importance of evaluating hyperparameters based on historical experiments. Common hyperparameter evaluations include manual parameter adjustment, grid search (GridSearch), random search, etc. Based on this, the inventors considered improving the performance of automatic machine learning by streamlining the search space based on the Bayesian optimization method theory.

[0081] In an embodiment of the present application, the specific process of obtaining the task image sample data set in the above step S101 includes: determining the data set size and category according to the image classification task, and selecting the original image data. The acquired original image data set is subjected to data preprocessing to obtain the task image sample data set; wherein the data preprocessing includes missing value processing, determining the data type, outlier detection, and data segmentation. Data preprocessing refers to the processing of the original data before the machine learning task in order to better adapt to the requirements of the model and improve the model performance. Among them, common data preprocessing steps include transformation, unique hot encoding, filling and balancing. In practical applications, it is necessary to select appropriate data preprocessing steps according to the characteristics of the data and the requirements of the machine learning task, and to tune and evaluate according to the feedback of the model to improve the performance and generalization ability of the model.

[0082] In a specific embodiment, the method of implementing the division of the image sample data set into multiple training sets and validation sets in the above step S102 can be selected according to the actual implementation method of automatic machine learning. By dividing the task image sample data set into two parts, the training set is used to train the model, and then the test set is used to evaluate the performance of the model. The data set can be divided by holding out (Holdout), that is, the data set is randomly divided into two mutually exclusive sets, k-fold cross validation (Cross Validation), etc., so as to evaluate the optimal solution of hyperparameters in the process of automatic machine learning, thereby achieving efficient model construction and parameter optimization. For example, in order to ensure the objectivity of the evaluation result, the order of samples in the task image sample data set can be disrupted by random shuffle, and then the data set is divided according to the division ratio, and the image sample data set is used for training at a ratio of 70% to 80%, and the remaining 20% ​​to 30% is used for testing. The ratio of the training set and the validation set, wherein the training set can be divided into multiple according to the number of classification algorithms used later. Another example is that the task image sample dataset is randomly shuffled to eliminate the order correlation of the data, and then the dataset is divided into multiple folds of similar size, each fold is kept as balanced as possible, one of the folds is selected as the validation set, and the remaining k-1 folds are selected as the training set. The value of k can be set according to the actual training requirements and computing overhead, for example, k is 5 or 10.

[0083] In a specific embodiment, the specific implementation process of selecting hyperparameters from the hyperparameter search space and establishing the candidate algorithm model in the above step S103 according to the preset multiple classification algorithms includes: for each classification algorithm, determining the corresponding algorithm hyperparameter search space in the hyperparameter search space; selecting hyperparameters from the corresponding algorithm hyperparameter search space, and constructing the candidate algorithm model corresponding to the classification algorithm. Generate the hyperparameter search space based on the value range of each hyperparameter and the data type of each hyperparameter in the algorithm configuration information; Generate the hyperparameter value corresponding to each hyperparameter based on the hyperparameter search space; wherein different hyperparameters can correspond to different hyperparameter values, and each hyperparameter value comes from the value range of the corresponding hyperparameter and conforms to its data type.

[0084] In the embodiment of the present application, multiple classification algorithms are pre-selected according to the data characteristics, task requirements and advantages and disadvantages of the algorithm of the image classification task, including classifier algorithms such as RF (random forest), AdaBoost (adaptive boost) and KNN (K nearest neighbor). Then, the hyperparameter search space is determined according to the configuration information, wherein the configuration information includes algorithm configuration information, task configuration information, and experimental configuration information, and the algorithm configuration information may include: algorithm address, value range of each hyperparameter, data type of each hyperparameter and other information. The task configuration information may include: data path of the running task, algorithm version and other information. The experimental configuration information may include: experiment name, resource group of the experiment, number of processors (such as GPU) used to execute the task, number of tasks in parallel, maximum running time, maximum number of running tasks, parameter adjustment algorithm type, parameter adjustment algorithm parameters, aborter type, aborter parameters and other information. Then, select hyperparameters from the hyperparameter search space to establish a candidate algorithm model.

[0085] In the embodiment of the present application, an automatic machine learning configurator in the prior art may be selected for implementation. Figure 2 As shown, it is a schematic diagram of a machine learning pipeline and corresponding components, which includes multiple classification algorithms, feature preprocessing methods and data preprocessing methods. Each of these methods is parameterized, and finally a hyperparameter search space with multiple hyperparameters is obtained. Most of the hyperparameters are conditional hyperparameters, that is, the corresponding hyperparameters are activated only when the corresponding components are selected. In an embodiment of the present application, based on an automatic machine learning method, an iterative search process including configuration generation and evaluation is formed by an optimizer and an evaluator, which is used to optimize the performance of the model and evaluate the model. The optimizer automatically selects a suitable optimization algorithm according to the hyperparameter search space and performance indicators. The performance of the model on a given data set is evaluated by an evaluator, and hyperparameter tuning and model selection are performed according to the performance of the performance indicators.

[0086] In the iterative search process, configuration plays an important role. Its generation and evaluation are the most time-consuming parts and determine the quality of automatic machine learning tools. Selecting potential candidate configurations from the search space is the key to configuration generation. This application learns the generation strategy of potential candidate configurations, such as training a meta-learner in a classification problem to predict useful feature transformations. Meta-learning must first characterize the learning problem and the learning tool (such as the statistical characteristics of the data, the hyperparameters of the learning tool), and then extract meta-features from past experience. Finally, the meta-learner can be trained with meta-knowledge. On the one hand, by characterizing the learning problem and the learning tool, important information can be discovered, such as data drift in the data (the model is no longer accurate over time). Secondly, with characterization, it is easy to find similar problems, so that knowledge can be reused and transferred between different problems. On the other hand, the meta-learner encodes past knowledge as a guide to solve future problems. Among them, meta-learning often takes small sample learning and rapid adaptation to new tasks as its entry point. The optimizer-based meta-learning model targets the characteristics of traditional neural network models, which have slow iteration speed and easy overfitting. It proposes to replace the gradient descent process with a separate neural network model, and use a separate neural network to update the gradient more accurately and quickly to achieve the purpose of rapid adaptation.

[0087] The data distribution of a single dataset may change, especially in a data stream. This change in data distribution is called "concept drift." Meta-learning can detect whether there is concept drift in the learning problem through changes in data or features, such as detecting changes in data distribution based on distance metrics and detecting concept drift by monitoring distribution changes in data streams. Once concept drift is detected, it predicts potential configurations that are suitable for the current data and performs dynamic configuration adaptation on them.

[0088] When selecting the conditional hyperparameters corresponding to each classification algorithm, you can build a classification algorithm model based on the machine learning algorithm, map the parameter configuration to the performance indicators of the problem, select a new parameter configuration in the hyperparameter search space based on the current model, run the problem instance under the selected parameter configuration, and measure its performance indicators. Add the new parameter configuration and performance indicators to the existing data set, repeat model training, and gradually optimize the parameter configuration until the set termination condition is reached (such as reaching the maximum number of iterations or the performance reaches a certain threshold), stop the search, and find the optimal hyperparameters.

[0089] In a specific embodiment, in the above step S104, each candidate algorithm model is trained using each training set, and the trained model is verified using the verification set, and the specific process of obtaining the algorithm evaluation index can be to input the training set into the machine learning pipeline of automatic machine learning, train each candidate algorithm model, and use the verification set to verify the trained model to obtain the algorithm evaluation index. The algorithm evaluation index may include accuracy, precision, recall, F1 score, mean squared error (MSE), logarithmic loss, etc., and may also include algorithm memory usage, running time, etc.

[0090] In a specific embodiment, in the above step S105, the specific process of selecting the algorithm model to be trained from each candidate algorithm model according to the algorithm evaluation index can be to sort the candidate algorithm models according to the algorithm evaluation index, and determine the algorithm model with the highest algorithm evaluation index as the algorithm model to be trained.

[0091] In a specific embodiment, in the above step S106, the process of using the multiple training sets and validation sets to train the algorithm model to be trained and obtaining the model evaluation index can refer to the above step S104 and the model training process of automatic machine learning in the prior art, wherein the model evaluation index may include accuracy, precision, recall, F1 value (F1 score), mean squared error (MSE), logarithmic loss (Log Loss), etc., and may also include computing resource usage, running time, etc.

[0092] In a specific embodiment, in the above step S107, the specific process of performing hyperparameter pruning according to the model evaluation index to obtain the optimal hyperparameter range may include:

[0093] During each algorithm model training process, the model evaluation indicators corresponding to each training set are used to determine whether the algorithm model in training is overfitting;

[0094] If yes, then according to the model evaluation index, the importance of each hyperparameter of the algorithm model is cyclically calculated to obtain the corresponding hyperparameter importance result;

[0095] Sort the hyperparameter importance results of each hyperparameter, and take the hyperparameters whose importance is greater than the first importance threshold as key hyperparameters, and take the hyperparameters whose importance is less than the second importance threshold as non-key hyperparameters; wherein the first importance threshold is greater than the second importance threshold;

[0096] Discard non-critical hyperparameters and prune each critical hyperparameter to obtain the optimal hyperparameter range.

[0097] In an embodiment of the present application, the importance of each hyperparameter in the algorithm model can be calculated in the following manner: randomly perturb the hyperparameter, change its value, keep other hyperparameters unchanged, then retrain the model on the hyperparameter combination after the perturbation, and calculate the evaluation performance of the algorithm model on the validation set. Finally, by comparing the changes in model performance before and after the perturbation, the importance of the hyperparameter is calculated, for example, the difference between the initial performance and the performance after the perturbation is used as a measure of the importance of the hyperparameter. After completing the cyclic calculation, the importance value of each hyperparameter is obtained, and it can be sorted according to the importance value to determine which hyperparameters contribute more to the model performance. The greater the performance change of the model before and after the perturbation, the greater the impact of the hyperparameter on the model performance and the higher the importance.

[0098] In a specific embodiment, each key hyperparameter may be pruned in the following manner to obtain an optimal hyperparameter range:

[0099] Determine the type of each key hyperparameter:

[0100] For categorical hyperparameters, K-Means is used to cluster the configuration performance, retaining the elements in the cluster with high performance to obtain the corresponding optimal hyperparameter range;

[0101] For numerical hyperparameters, Spearman is used to perform correlation analysis between configuration performance and parameter value range, and selection is made based on the correlation to obtain the corresponding optimal hyperparameter range.

[0102] The inventors have discovered that in the process of model training for automatic machine learning, as the depth of the model increases, overfitting gradually occurs. Through pruning, parameters with too deep dimensions or layers are removed to prevent model overfitting. Usually, the model can be gradually established by correcting the hyperparameters during the training process, and the topology, such as the number of layers or dimensions, is adjusted during the training process. However, the implementation method of the prior art faces the problem of too high a dimension of the search space when searching for hyperparameters. Based on this, the inventors turned their attention to the weights related to the regularization terms and gates, and proposed a new pruning method based on the selection of key hyperparameters. In this application, the value range of hyperparameters with different degrees of importance is adjusted during the operation, so as to continuously streamline the search space and avoid additional evaluation overhead to improve the performance of supervised automatic machine learning. This application uses pruning technology based on the selection of key hyperparameters to prune the search space, avoiding additional evaluations in known low-performance areas due to high uncertainty in the configuration.

[0103] During the model training process, the model evaluation indicators corresponding to the two adjacent training sets are used to determine whether the algorithm model is overfitting. For example, if the model evaluation indicators obtained by the algorithm model training under the first set of training sets are good, while the model evaluation indicators obtained by the algorithm model training under the second set of training sets are poor, it indicates that the algorithm model training may be overfitting. At this time, the hyperparameters of the algorithm model can be pruned by pruning to establish the depth of the model. First, according to the historical model evaluation indicators, the importance of each hyperparameter of the algorithm model is calculated cyclically to obtain the corresponding hyperparameter importance results. Then, the key hyperparameters are screened, and the hyperparameter importance calculated in the previous step is sorted according to the hyperparameter importance results of each hyperparameter. The hyperparameters with an importance greater than the first important threshold p are regarded as key hyperparameters, and the hyperparameters with an importance less than the second important threshold q are regarded as non-key hyperparameters. This division method is relatively simple and direct. The whole process belongs to recursive elimination of features and is greatly affected by the calculation results of the hyperparameter importance in the previous stage. Then, non-key hyperparameters should be directly discarded, and key hyperparameters should be detected and pruned. For key hyperparameters, changing their values ​​can greatly improve the performance of the model. The high-performance interval in the value range should be retained. Different methods should be used to determine the pruning interval for hyperparameters of different types (categorical and numerical). For categorical hyperparameters, K-Means is used to cluster the configuration performance and only the elements in the cluster with high performance are retained; for numerical hyperparameters, Spearman is used to perform a correlation analysis between the configuration performance and the parameter value range, and the value range is pruned to the high interval (positive correlation), low interval (negative correlation), middle interval or both end intervals (irrelevant) according to the direction of the correlation.

[0104] In the embodiment of the present application, the method of assigning hyperparameters to the algorithm model in training according to the preferred hyperparameter range in the above step S108 can refer to the detailed description of the prior art, and is not specifically limited here. After configuring the hyperparameters of the algorithm model, continue the process of model training.

[0105] In a specific embodiment, the iteration stop condition in the above step S109 can be determined according to the type and quantity of hardware resources running the automatic learning task, the model accuracy threshold of the trained algorithm model, or according to the number of iterations or the training time threshold. Those skilled in the art can determine it according to actual conditions. Before the model reaches the iteration termination condition, AutoML will continue to monitor the model training process and return the current performance value of the model.

[0106] In an embodiment of the present application, during the model hyperparameter search process, the judgment of the quality of each set of hyperparameters depends on the currently trained model and the inference results of the test set using the current model, so as to efficiently complete the hyperparameter search and model optimization.

[0107] In a specific embodiment, the method further comprises:

[0108] The task image sample data set is subjected to feature preprocessing to obtain a task image sample data set after constructing features; the feature preprocessing includes feature transformation and feature selection. The original features can be converted into a set of features with obvious physical meaning or statistical meaning or kernel through feature transformation, such as reducing the number of values ​​of a certain feature in the original data by transforming feature values; feature selection is to select a set of most statistically significant feature subsets from the feature set, and delete irrelevant features, so as to achieve the effect of dimensionality reduction. Feature preprocessing methods may include PCA (Principal Component Analysis) and fastICA (Fast Independent Component Analysis) to reduce data dimension, perform data denoising, and extract main features.

[0109] In a specific embodiment, in order to achieve feature selection, the optimal feature can be selected based on the decision tree algorithm to perform node division and evaluate the feature importance. The specific process may include: first, for each feature, node division is performed on each value of the feature, and the impurity of the child node after division is calculated, then, the impurity difference before and after the node division using the feature is calculated, that is, the impurity before division minus the impurity after division, then, for each feature, the average value of the impurity difference of all possible division points is calculated, and finally, the feature with the largest average impurity difference is selected as the optimal division feature. In this way, features that are more discriminatory for classification tasks can be found during feature selection, thereby improving the performance and accuracy of the model.

[0110] In order to maximize the use of computing power and avoid being trapped in a long-term operation of a specific combination of preprocessing and machine learning algorithms, in an embodiment of the present application, the time of each evaluation of the machine learning framework instance can be limited, and the memory usage space of the evaluation can be limited to prevent the swapping and freezing of the operating system. When the evaluation does not meet one of these restrictions, the evaluation will automatically terminate and return the worst possible score on the given evaluation indicator, which combines the duration, computing resource usage, etc.

[0111] In order to further reduce the total amount of long runs, in the embodiment of the present application, some combinations of preprocessors and classification algorithms can be prohibited by the Auto-sklearn tool. In particular, Auto-sklearn prohibits the combination of kernel approximation methods with nonlinear methods, tree-based methods and KNN algorithms. In the SMAC method, Auto-sklearn processes such prohibited combinations. For the same reason, Auto-sklearn omits feature learning algorithms such as dictionary learning.

[0112] In the embodiment of the present application, after the image classification model is obtained based on the above step S110, the model can be exported and deployed to the application. By using automatic machine learning tools, users can save a lot of time and energy and obtain high-quality image classification models.

[0113] The image classification model training method based on automatic machine learning provided in the embodiment of the present application is to select hyperparameters from the hyperparameter search space according to multiple preset classification algorithms, establish a candidate algorithm model, and during the model development, multiple parallel tasks with different models and hyperparameters will be created. During the training process, the model will be evaluated at each iteration, and the model and hyperparameters will be optimized through the evaluation indicators to screen the algorithm model to be trained. During the training process of the algorithm model to be trained, the hyperparameter range is optimized through hyperparameter pruning, which can effectively reduce the cost of automatic machine learning training and make it cheaper for users to build image classification applications. At the same time, the automatic machine learning of classification algorithms reduces the threshold for the use of AI technology, helps model developers save modeling and training time, and allows them to have more energy to focus on the business itself, thereby promoting the implementation of AI in professional fields, greatly improving the industry penetration rate of artificial intelligence technology in the field of image processing, and providing strong support for digital transformation and intelligent development. Provide certain practical guidance for domain experts and actual developers on the selection of multivariate classification algorithms.

[0114] Embodiment 2

[0115] Based on the same inventive concept, the present application embodiment also provides an image classification model training device based on automatic machine learning, referring to Figure 3 As shown, the device comprises:

[0116] An acquisition module 101 is used to acquire a task image sample data set;

[0117] A division module 102, used to divide the image sample data set into multiple training sets and verification sets;

[0118] Establishing module 103, used for selecting hyperparameters from the hyperparameter search space according to a plurality of preset classification algorithms, and establishing a candidate algorithm model;

[0119] A first training module 104 is used to train each candidate algorithm model using each training set, and to verify the trained model using the verification set to obtain an algorithm evaluation index;

[0120] A screening module 105 is used to screen the algorithm model to be trained from each candidate algorithm model according to the algorithm evaluation index;

[0121] A second training module 106 is used to train the algorithm model to be trained using the multiple training sets and validation sets, and obtain a model evaluation index;

[0122] A pruning module 107, configured to prune hyperparameters according to the model evaluation index to obtain an optimal hyperparameter range;

[0123] An assignment module 108, configured to assign hyperparameters to the algorithm model in training according to the preferred hyperparameter range;

[0124] The model determination module 109 is used to determine whether the model meets the iteration stop condition and obtain the image classification model.

[0125] Embodiment 3

[0126] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the image classification model training method based on automatic machine learning as described in the above embodiment one is implemented.

[0127] Embodiment 4

[0128] Based on the same inventive concept, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the image classification model training method based on automatic machine learning as described in the above embodiment 1 is implemented.

[0129] Embodiment 5

[0130] Based on the same inventive concept, an embodiment of the present application also provides a computer program product comprising instructions. When the computer program product is run on a computer device, the computer device executes the image classification model training method based on automatic machine learning as described in the above embodiment 1.

[0131] Embodiment 6

[0132] Based on the same inventive concept, an embodiment of the present application also provides a chip, the chip includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run a computer program or instructions to implement the image classification model training method based on automatic machine learning as described in the above embodiment one.

[0133] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0134] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0137] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for training an image classification model based on automatic machine learning, characterized in that: include: Get a sample dataset of task images; Dividing the image sample data set into multiple training sets and validation sets; According to the preset multiple classification algorithms, hyperparameters are selected from the hyperparameter search space to establish a candidate algorithm model; Use each training set to train each candidate algorithm model, and use the verification set to verify the trained model to obtain an algorithm evaluation index; According to the algorithm evaluation index, the algorithm model to be trained is obtained by screening from each candidate algorithm model; Using the multiple training sets and validation sets to train the algorithm model to be trained, and obtaining a model evaluation index; Perform hyperparameter pruning according to the model evaluation index to obtain an optimal hyperparameter range; Assigning hyperparameters to the algorithm model in training according to the preferred hyperparameter range; The above model training process is repeated until the iteration stop condition is met to obtain the image classification model.

2. The method according to claim 1, characterized in that The hyperparameter pruning is performed according to the model evaluation index to obtain an optimal hyperparameter range, including: During each algorithm model training process, the model evaluation indicators corresponding to each training set are used to determine whether the algorithm model in training is overfitting; If yes, then according to the model evaluation index, the importance of each hyperparameter of the algorithm model is cyclically calculated to obtain the corresponding hyperparameter importance result; Sort the hyperparameter importance results of each hyperparameter, and take the hyperparameters whose importance is greater than the first importance threshold as key hyperparameters, and take the hyperparameters whose importance is less than the second importance threshold as non-key hyperparameters; wherein the first importance threshold is greater than the second importance threshold; Discard non-critical hyperparameters and prune each critical hyperparameter to obtain the optimal hyperparameter range.

3. The method according to claim 2, characterized in that Each key hyperparameter is pruned in the following way to obtain the optimal hyperparameter range: Determine the type of each key hyperparameter: For categorical hyperparameters, K-Means is used to cluster the configuration performance, retaining the elements in the cluster with high performance to obtain the corresponding optimal hyperparameter range; For numerical hyperparameters, Spearman is used to perform correlation analysis between configuration performance and parameter value range, and selection is made based on the correlation to obtain the corresponding optimal hyperparameter range.

4. The method according to claim 1, characterized in that The method of selecting hyperparameters from a hyperparameter search space according to a plurality of preset classification algorithms and establishing a candidate algorithm model includes: For each classification algorithm, the corresponding algorithm hyperparameter search space is determined in the hyperparameter search space; Hyperparameters are selected from a corresponding algorithm hyperparameter search space to construct a candidate algorithm model corresponding to the classification algorithm.

5. The method according to claim 1, characterized in that The step of obtaining a task image sample dataset includes: The acquired original image data set is subjected to data preprocessing to obtain a task image sample data set; wherein the data preprocessing includes missing value processing, data type judgment, outlier detection, and data segmentation.

6. The method according to claim 1, characterized in that Also includes: Performing feature preprocessing on the task image sample data set to obtain a task image sample data set after constructing features; The feature preprocessing includes feature transformation and feature selection.

7. An image classification model training device based on automatic machine learning, characterized in that: include: An acquisition module is used to obtain a task image sample dataset; A division module, used for dividing the image sample data set into multiple training sets and verification sets; Establishing a module for selecting hyperparameters from a hyperparameter search space and establishing a candidate algorithm model according to a plurality of preset classification algorithms; A first training module is used to train each candidate algorithm model using each training set, and to verify the trained model using the verification set to obtain an algorithm evaluation index; A screening module is used to screen the algorithm model to be trained from the candidate algorithm models according to the algorithm evaluation index; A second training module is used to train the algorithm model to be trained using the multiple training sets and validation sets, and obtain a model evaluation index; A pruning module, used to prune hyperparameters according to the model evaluation indicators to obtain an optimal hyperparameter range; An assignment module, used for assigning hyperparameters to the algorithm model in training according to the preferred hyperparameter range; The model determination module is used to determine whether the model meets the iteration stop condition and obtain the image classification model.

8. A computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal, the terminal executes the image classification model training method based on automatic machine learning as described in any one of claims 1 to 6.

9. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image classification model training method based on automatic machine learning as described in any one of claims 1 to 6.

10. A computer program product comprising instructions, which, when executed on a computer device, enables the computer device to execute the image classification model training method based on automatic machine learning as described in any one of claims 1 to 6.

11. A chip, comprising a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run a computer program or instruction to implement the image classification model training method based on automatic machine learning as described in any one of claims 1-6.

Images