An image classification training method based on sharpness perception minimization of AUC optimization

Abstract

The application provides an image classification training method based on AUC optimization sharpness perception minimization, and the technical scheme of the application adjusts the perturbation used in the sharpness perception minimization technology, maps the first gradient about the model parameter by using a preset scrambling hyperparameter, and determines the perturbation corresponding to the model parameter, so as to avoid the complex minimax-minimax optimization problem caused by the direct adaptation of the instance level form I-AUC and the sharpness perception minimization technology, effectively reduce the model training time, and reduce the energy consumption; at the same time, with the aid of the sharpness perception minimization technology, the generalization ability of the model is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to the technical field of image classification based on neural networks, and more particularly to an image classification training method based on AUC optimization and sharpness-aware minimization. BACKGROUND

[0002] In real-world datasets, there is often a long-tailed distribution of classes, i.e., the number of samples in the head classes is much larger than that in the tail classes. In this case, the traditional evaluation metrics (such as accuracy) usually ignore the performance of the tail classes, thus failing to effectively measure the performance of the model. Therefore, the area under the ROC curve (Area Under Curve, AUC for short) has gradually become a popular evaluation metric due to its insensitivity to class distribution, and is widely used in many long-tailed scenarios, such as financial fraud detection, disease diagnosis, and anomaly detection.

[0003] Currently, there are two main methods for optimizing AUC. The original AUC optimization method based on the empirical risk minimization principle is called the pairwise form (P-AUC). The pairwise form refers to forming sample pairs composed of positive and negative samples to guide model training, aiming to maximize the average pairwise ranking accuracy between positive and negative classes. Although the pairwise form can effectively optimize the unbiased estimate of the proxy metric, the time complexity of each iteration is where and represent the number of positive and negative examples, respectively. In view of this, the literature [1] reformulates the original problem as an instance-level minimax problem, called the instance-level form (I-AUC). The instance-level form refers to a single sample as an instance, and uses the label in the instance to guide model training. Compared with the pairwise form, the time cost of each iteration of the instance-level form is linear complexity, but due to the complexity of the minimax problem, its convergence speed is slow.

[0004] From a geometric perspective, both frameworks of AUC optimization have poor generalization ability. As pointed out in the literature [2], the geometric shape of the loss function, especially its flatness, is closely related to the generalization ability. However, due to the much more complex AUC loss function than the traditional loss function, it naturally forms a sharp loss surface, which in turn leads to poor generalization ability. For traditional losses, the sharpness-aware minimization method [3] (Sharpness-aware Minimization, SAM for short) has been proven to be able to help get a flatter loss surface, thus improving the generalization ability of the model.

[0005] In the field of image binary classification, there are certain challenges in directly using existing sharpness-aware minimization methods in the scenario of AUC optimization in instance-level form. Because for instance-level form, direct adaptation will lead to a complex minimax-minimax optimization problem, which is almost impossible to solve.

[0006] The information of the above cited documents is as follows:

[0007] [1] Ying, Y.; Wen, L.; and Lyu, S. 2016. Stochastic Online AUCMaximization. In Annual Conference on Neural Information Processing Systems,451–459.

[0008] [2] Keskar, N. S.; Mudigere, D.; Nocedal, J.; Smelyanskiy, M.; andTang, P. T. P. 2017. On Large-Batch Training for Deep Learning:Generalization Gap and Sharp Minima. In International Conference on LearningRepresentations, 1–16.

[0009] [3] Foret, P.; Kleiner, A.; Mobahi, H.; and Neyshabur, B. 2021.Sharpness-aware Minimization for Efficiently Improving Generalization. InInternational Conference on Learning Representations, 1–20.

[0010] It should be noted that: the background art is only used to introduce the relevant information of the present application, in order to help understand the technical solutions of the present application, but it does not mean that the relevant information must be prior art. The relevant information is submitted and disclosed together with the present application scheme, and in the absence of evidence that the relevant information has been disclosed before the filing date of the present application, the relevant information should not be regarded as prior art. SUMMARY

[0011] Therefore, the purpose of the present application is to overcome the defects of the prior art, and provide an image classification training method and an image classification method.

[0012] The purpose of the present application is achieved by the following technical solutions:

[0013] According to a first aspect of the present application, an image classification training method is provided, comprising: obtaining a binary classification training set of images, which includes a plurality of samples, a sample including a sample image and a label, the label indicating the category to which the corresponding sample image belongs, the training set having only two categories in the label space and the number of samples of the two categories being unbalanced; obtaining a current batch of samples from the training set, inputting the sample images of the current batch into an image classification model, determining an original classification prediction value of the input images using the model parameters of the model, and determining a first gradient with respect to the model parameters according to the original classification prediction value, the label, and a preset first total loss function; mapping the first gradient with respect to the model parameters using a preset perturbation hyperparameter to determine a perturbation corresponding to the model parameters; adding the model parameters and the perturbation to obtain perturbed parameters of the model; determining a perturbed classification prediction value of the sample images of the current batch using the perturbed parameters of the model, and determining a second gradient with respect to the model parameters and a gradient of a learnable parameter in a second total loss function according to the perturbed classification prediction value, the label, and a preset second total loss function; and updating the model parameters and the learnable parameter in the second total loss function with the optimization goal of minimizing the loss value of the second total loss function according to the second gradient with respect to the model parameters and the gradient of the learnable parameter in the second total loss function. The technical solution of this embodiment can at least achieve the following beneficial technical effects: the present application maps the first gradient using a perturbation hyperparameter to obtain a perturbation corresponding to the model parameters, which can adjust the optimization process of the original sharpness-aware minimization technique from a min-max-min optimization problem to a min-max optimization problem, and the computational complexity of the min-max optimization problem is much lower than that of the min-max-min optimization problem, effectively reducing the time of model training and reducing energy consumption; at the same time, with the aid of the sharpness-aware minimization technique, the generalization ability of the model is effectively improved.

[0014] Optionally, the perturbation is obtained by dividing the first gradient with respect to the model by a preset perturbation hyperparameter.

[0015] Optionally, the perturbation is calculated in the following manner:

[0016]

[0017] wherein, represents the perturbation, represents the model parameters, represents the first total loss function, represents gradient calculation, represents the first gradient with respect to the model, represents a preset perturbation hyperparameter, The technical scheme of this embodiment can achieve at least the following beneficial technical effects: through reasoning, the inventors prove that the perturbation obtained in this way can be used as a simplified replacement of the original sharpness perception minimization technique, significantly reducing the difficulty of solving.

[0018] Optionally, the first loss function is: .

[0019] Optionally, the second loss function is:

[0020]

[0021] wherein, represents the number of samples, represents the number of samples in the current batch, , represents the model parameters, , , represents the function set in the function, , represents the sample image, represents the label, represents the expectation about the sample; represents an indicator function, the output of the indicator function is 1 when the condition in the bracket is true, otherwise 0; represents the sample image corresponding to the label; represents the confidence that the sample image determined by the model parameters belongs to a certain class. The technical scheme of this embodiment can achieve at least the following beneficial technical effects: the total loss function incorporates the contents of classification loss minimization and sharpness perception minimization techniques, which can train the model parameters from multiple aspects to better improve the classification performance and generalization of the model.

[0022] Optionally, during the training process, the gradient descent method is used to update the model parameters, the learnable parameters and the learnable parameters , and the gradient ascent method is used to update the learnable parameters . The technical scheme of this embodiment can achieve at least the following beneficial technical effects: after using the perturbation and the total loss function of the application, the gradient descent method is used to update the model parameters, the learnable parameters and the learnable parameters , and the gradient ascent method is used to update the learnable parameters The gradient ascent method is used for updating, which can simplify the calculation process of classification loss minimization and sharpness perception minimization, accelerate the convergence of the model, and improve the performance of the trained model.

[0023] According to a second aspect of the present application, an image classification method is provided, which comprises: obtaining a trained image classification model trained by the method according to the first aspect; obtaining an image to be classified, and classifying the image to be classified by using the trained image classification model.

[0024] According to a third aspect of the present application, an anomaly detection method is provided, which comprises: obtaining a trained image classification model trained by the method according to the first aspect, wherein the trained image classification model is trained by a preset anomaly detection classification training set, and the labels of the samples in the anomaly detection classification training set are divided into two categories, one of which is a normal category and the other of which is an abnormal category; obtaining an image to be classified, and classifying the image to be classified by using the trained image classification model to determine whether there is an anomaly.

[0025] According to a fourth aspect of the present application, an electronic device is provided, which comprises: one or more processors; and a memory, wherein the memory is used to store executable instructions; and the one or more processors are configured to implement the steps of the method according to the first, second or third aspect by executing the executable instructions. BRIEF DESCRIPTION OF DRAWINGS

[0026] The embodiments of the present application will be further described below with reference to the accompanying drawings, in which:

[0027] Figure 1 A flowchart of an image classification training method according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0029] As mentioned in the background section, in the field of image binary classification, there are certain challenges in directly applying the existing sharpness-aware minimization method in the scenario of AUC optimization in instance-level form. Because for instance-level form, direct adaptation will lead to a complex minimax-minimax optimization problem, which is almost impossible to solve. In this regard, the method of the present application adjusts the perturbation adopted in the sharpness-aware minimization technology, maps the first gradient with respect to the model parameters using a preset perturbation hyperparameter, determines the perturbation corresponding to the model parameters, thereby simplifying the minimax-minimax optimization problem into a minimax optimization problem, effectively reducing the time of model training and reducing the consumption of energy; at the same time, with the help of the sharpness-aware minimization technology, the generalization ability of the model is effectively improved.

[0030] In order to illustrate the problems faced by the present application, the inventors first explain the reasons why directly applying instance-level form I-AUC to the original sharpness-aware minimization technology leads to difficulties.

[0031] The expected loss of I-AUC is defined as:

[0032]

[0033] wherein, represents an expected function, represents the loss function of the loss of each sample corresponding to the loss, represents the model parameter, , , represents the learnable parameter set in the loss function.

[0034] The empirical loss of I-AUC is defined as:

[0035]

[0036] wherein, represents the number representing the sample, represents the number of samples in a batch, or the batch size (BatchSize).

[0037] Directly applying instance-level form I-AUC to the original sharpness-aware minimization technology can obtain the following complex optimization problem:

[0038]

[0039] wherein, represents the model parameter, represents the perturbation parameter vector set in the original sharpness-aware minimization technology, each element of the dimension in the above formula is subject to a Gaussian distribution with a mean of 0 and a variance of , represents the regularization coefficient.

[0040] As can be seen, this leads to a minimax-minimax optimization problem that is almost impossible to solve. If applied directly, it will make it difficult for the model training process to converge.

[0041] To this end, the inventor considered performing reasoning optimization to find alternative feasible solutions to achieve similar effects. The reasoning process is as follows:

[0042] make , and then make the following assumptions:

[0043] Assume 1 that For a about of Weakly concave function, that is, for any It's about A concave function, where .

[0044] Therefore, under the above assumptions, we can directly use the minimum and maximum theorem to simplify the above objectives and obtain:

[0045]

[0046] For the inner maximization problem, the first-order Taylor expansion can be used to approximate the solution :

[0047]

[0048]

[0049]

[0050]

[0051] in, represents the gradient, represents the L2 distance function, Indicates transpose.

[0052] In this way, the optimization process of the original sharpness-aware minimization technology was adjusted from a minimax-minimax optimization problem to a minimax optimization problem, effectively reducing the model training time and energy consumption; at the same time, with the help of sharpness-aware minimization technology, the generalization ability of the model was effectively improved.

[0053] In order to further understand the principle of the present invention, it is described below in conjunction with various embodiments.

[0054] Implementation method 1:

[0055] This embodiment is explained from the model structure, training data, loss function, training process and model application.

[0056] I. Model structure

[0057] According to an embodiment of the present application, the image classification model comprises a feature extractor for extracting image features from an input image and a classification layer for determining a classification prediction value (i.e. the probability that the input image belongs to the corresponding category) corresponding to the input image according to the image features.

[0058] Optionally, the image classification model can use an existing classification model, such as a binary classification image classification model constructed by ResNet model, EfficientNet model, DenseNet model or VGGNet model, etc. Of course, a model defined by the implementer can also be used, such as constructing a feature extractor by convolutional layers, attention layers, residual layers, linear layers and classification layers or their combinations, and constructing a binary classification layer by a fully connected layer and a Sigmoid function. The present application does not make any limitation on this. After obtaining the model, the parameters of the image classification model can be randomly initialized as the parameters of the model used for the first training.

[0059] II. Training data

[0060] According to an embodiment of the present application, samples are randomly sampled from an unbalanced binary classification image dataset to form a training set, and then the samples of the training set are randomly divided into multiple batches. The data set of this embodiment is a data set for training of contraband detection, specifically collected for detecting contraband in security check scenes. The data set includes multiple samples, each sample including a sample image and a label indicating the category to which the corresponding sample image belongs. The label space in the data set only has two categories, and the number of samples of the two categories is unbalanced. One category is normal, indicating that there is no contraband in the image; the other category is abnormal, indicating that there is contraband in the image. Usually, the number of normal samples is relatively large, and the number of abnormal samples is relatively small, to form an imbalance in quantity. The data set can use an existing data set, or a self-made data set collected and labeled by the implementer. The present application does not make any limitation on this.

[0061] In addition, in order to facilitate subsequent understanding, some letters are defined in combination with the training set:

[0062] In the image binary classification problem, the training set comes from the distribution , where represents the input, represents a real number space of dimension , and represents the height, width and channel number of the image, respectively; Indicates a label, represents the 0th category, Indicates that in class 1, the number of samples in the two classes is unbalanced; Denotes the number of samples. and Represents the number of samples in class 0 and class 1 respectively. The score function is defined as ,in, Represents model parameters, scores express The probability of belonging to class 0.

[0063] 3. Loss Function

[0064] In the method of the present invention, two total loss functions are provided, wherein the first total loss function is only used to calculate the first gradient with respect to the model parameters and is not actually used to guide the update of the model parameters. The update of the model parameters is guided by the second total loss function.

[0065] According to one embodiment of the present invention, the first total loss function is:

[0066] ,

[0067] in, represents the number of the sample, Indicates the number of samples in the current batch, , represents the model parameters, 、 、 Representation function The learnable parameters set in , represents a sample image, Indicates a label, Express expectations about the sample; Indicates an indicator function. When the condition in the brackets is met, the output of the indicator function is 1, otherwise it is 0. ,For example, Indicates that when the label value of the sample is 1 ,otherwise ; Represents a sample image The corresponding label, Indicates the use of model parameters Determined sample images The confidence level of a class. In a binary classification scenario, assuming there are two classes, class 0 and class 1, we can The result of is set as the confidence that the image belongs to class 0, then the confidence of the other class can be obtained by 1- get.

[0068] According to an embodiment of the present application, the second total loss function is:

[0069]

[0070] wherein, represents the perturbation of the method of the present application, represents a preset perturbation hyperparameter, represents an L2 distance function, represents a regularization coefficient. The explanations of the remaining parameters in the second total loss function can refer to the introduction of the first total loss function, which will not be repeated here.

[0071] Preferably, the perturbation used in the second total loss function is calculated in the following way:

[0072]

[0073] wherein, represents the perturbation, represents the model parameter, represents the first total loss function, represents the gradient, represents a preset perturbation hyperparameter, It can be known from the previous reasoning process that the inventors have proved that the perturbation obtained in this way can be used as a simplified substitute for the perturbation of the original sharpness-aware minimization technique. The setting of this perturbation can adjust the optimization process that combines the instance-level form I-AUC and the sharpness-aware minimization technique from a minimax-minimax optimization problem to a minimax optimization problem, significantly reducing the difficulty of solving.

[0074] IV. Training process

[0075] According to an embodiment of the present application, referring to Figure 1 , an image classification training method is provided, comprising steps S1-S7. In order to better understand the present application, each step will be described in detail below in combination with specific embodiments.

[0076] Step S1: Obtain a binary classification training set of images, which includes a plurality of samples, and each sample includes a sample image and a label. The label indicates the category to which the corresponding sample image belongs. The label space in the training set only has two categories, and the number of samples of the two categories is uneven.

[0077] According to one embodiment of the present application, the binary classification training set can be obtained from the data set in the second part. Thus, a training set for detecting contraband in a security check scene can be obtained. Among them, one class is a normal class, indicating that there is no contraband in the image; the other class is an abnormal class, indicating that there is contraband in the image; and the number of samples of the normal class is more than that of the abnormal class. It should be understood that it is also feasible to set more samples of the abnormal class, which only needs to select more samples of the abnormal class to join the training set when sampling from the data set.

[0078] Step S2: obtaining a current batch of samples from the training set, inputting the sample images in the current batch into the image classification model, determining the original classification prediction value of the input image by using the model parameters of the model, and determining the first gradient of the model parameters according to the original classification prediction value, the label and the preset first total loss function.

[0079] According to one embodiment of the present application, a batch of samples can be obtained from the training set each time, and the model training process (i.e. the model parameter updating process) is completed by using the samples in the batch. For example, assuming that there are 100 samples in the training set and an epoch is divided into 10 batches, then each batch gets 10 samples, and the model is trained by using the samples in each batch.

[0080] Among them, during training, each sample image in each batch is input into the image classification model, the image features are extracted by using the feature extractor, and the original classification prediction value corresponding to the sample image is obtained by using the classifier according to the image features. Subsequently, the loss value is calculated according to the original classification prediction value and the label of the sample image in the batch by using the first total loss function (i.e. ); and the first backward propagation is performed based on the loss value to obtain the gradient (referred to as the first gradient of the model parameters for distinction) of the model parameters, which can be represented as:

[0081]

[0082] Among them, represents the first total loss function.

[0083] Step S3: mapping the first gradient of the model parameters by using the preset perturbation hyperparameter to determine the perturbation corresponding to the model parameters.

[0084] According to one embodiment of the present application, the perturbation of this step is obtained by dividing the first gradient of the model by the preset perturbation hyperparameter, which is represented as:

[0085]

[0086] Among them, represents the perturbation, represents the model parameters, denotes the first total loss function, denotes the gradient, denotes a preset perturbation hyperparameter, . The specific value of the perturbation hyperparameter can be set by the implementer according to experience or a relatively optimal value selected by actual multiple training when training the model.

[0087] Step S4: adding the model parameters and the perturbation to obtain perturbed parameters of the model.

[0088] According to an embodiment of the present application, the perturbed parameters of the model are: It should be understood that in the field of neural networks, the model parameters are multi-dimensional and have multiple components, and are not scalar. Correspondingly, the perturbation is also multi-dimensional and has the same dimension as the model parameters. This step is equivalent to adding the corresponding perturbation component in the perturbation to each parameter in the model parameters, thereby obtaining the perturbed parameters of the model.

[0089] Step S5: determining a perturbed classification prediction value of the sample image in the current batch by using the perturbed parameters of the model, and determining a second gradient of the model parameters and a gradient of the learnable parameters in the second total loss function according to the perturbed classification prediction value, the label, and a preset second total loss function.

[0090] According to an embodiment of the present application, after the model parameters are perturbed, the perturbed parameters of the model are used to obtain a prediction value (i.e., a perturbed classification prediction value) through a forward propagation again, and the loss value is calculated by the second total loss function together with the label. The second gradient of the model refers to the gradient of the second total loss function with respect to each model parameter based on the sample image in the current batch and the label. The second gradient can be calculated in the following manner:

[0091]

[0092] wherein, denotes the second total loss function.

[0093] Step S6: updating the model parameters and the learnable parameters in the second total loss function according to the second gradient of the model parameters and the gradient of the learnable parameters in the second total loss function, with the optimization objective being to minimize the loss value of the second total loss function.

[0094] According to an embodiment of the present application, during the training process, the model parameters, the learnable parameters and the learnable parameters are updated using the gradient descent method, and the learnable parameters are updated using the gradient ascent method.

[0095] wherein, illustratively, the model parameters are iteratively updated in the following manner:

[0096]

[0097] wherein, denotes the model parameters obtained after the i-th batch update, denotes the model parameters obtained after the i-th batch update, denotes the model parameters used before the i-th batch update, denotes the learning rate, denotes the second gradient. For example, set to 0.01, 0.001 or 0.0001, etc., the specific value can be set according to the needs of the implementer.

[0098] Illustratively, the learnable parameters are updated in the following manner:

[0099]

[0100] wherein, denotes the learnable parameters obtained after the i-th batch update, denotes the learnable parameters obtained after the i-th batch update, denotes the learnable parameters used before the i-th batch update, denotes the learning rate, denotes the gradient of the learnable parameters . Illustratively, the learnable parameters are updated in the following manner:

[0101]

[0102]

[0103] wherein, denotes the learnable parameters obtained after the i-th batch update, denotes the learnable parameters obtained after the i-th batch update, denotes the learnable parameters used before the i-th batch update, denotes the learning rate, denotes the gradient of the learnable parameters . Illustratively, the learnable parameters are updated in the following manner:

[0104]

[0105]

[0106] wherein, denotes the learnable parameters obtained after the i-th batch update,​​​​​​​ batch-updated learnable parameters , denote the learnable parameters before the batch update batch-updated learnable parameters , denote a learning rate, denote the learnable parameters gradient.

[0107] Step S7: Repeat steps S2-S6 until a preset training requirement is reached, to obtain a trained image classification model.

[0108] According to an embodiment of the present application, in step S7, it is checked whether the preset training requirement is reached. If not, the next batch is entered and the process returns to step S2. If yes, the training is stopped and a trained model is obtained. The preset training requirement can be, for example, a requirement for the total training batch or round, such as 10000 batches or 20 rounds of training, etc. Alternatively, the preset training requirement can be set as a loss value being reduced to a preset threshold range and being maintained in the threshold range for a preset number of batches. Of course, other training requirements set by the implementer can also be used, and the present application does not limit the same.

[0109] V. Model application

[0110] According to an embodiment of the present application, an anomaly detection method, in particular a contraband detection method, is provided. The method comprises:

[0111] A1. Obtain a trained image classification model trained according to the method of the preceding embodiments, wherein the trained image classification model is trained by a training set for detecting contraband in a security inspection scene;

[0112] A2. Obtain an image to be classified, and classify the image to be classified using the trained image classification model to determine whether there is contraband.

[0113] Embodiment 2

[0114] The difference between this embodiment and embodiment 1 is that the data set of this embodiment is a data set for identifying a certain disease. For example, it is a data set for identifying whether there is a lesion in the fundus, wherein the sample image is a fundus image taken by a fundus camera, and the normal class in the label represents no fundus lesion, and the abnormal class represents the presence of fundus lesion (such as leopard fundus, retinal detachment, retinal hole, macular lesion, etc.).

[0115] Correspondingly, training a model using the training set can obtain a trained image classification model for judging whether there is a fundus disease.

[0116] Embodiment 3

[0117] The embodiment is different from embodiments 1 and 2 in that the data set of the embodiment is a data set for identifying whether an abnormality exists in a certain device (or a part of the device). In the data set, the sample image is an image of the device (or the part of the device) captured by a camera, and the normal class in the label represents that the device (or the part of the device) is normal, and the abnormal class represents that an abnormal working condition (such as a broken clamp, a broken prop, an open protective cover, a loose bolt, etc.) exists.

[0118] Correspondingly, the trained image classification model for judging whether an abnormality exists in a certain device (or a part of the device) can be obtained by training the model using the training set.

[0119] In order to verify the effect of the present application, the inventors also carried out the following comparative experiments:

[0120] The existing method used for comparison is as follows:

[0121] CE represents Cross-Entropy, that is, the cross-entropy loss function;

[0122] Focal represents Focal loss, that is, the focal loss function;

[0123] CB represents Class-Balanced loss, that is, the class-balanced loss function;

[0124] LDAM+DRW represents the combination of LDAM technology and DRW technology, wherein LDAM represents Label-distribution-aware margin loss, that is, the label-distribution-aware margin loss function; and DRW represents Deferred re-weighting, that is, the loss function of the deferred re-weighting method;

[0125] LA represents Logit adjustment, that is, the loss function (or score adjustment loss function) of the model output adjustment method, which is a loss function based on the cross-entropy of the adjusted model output logic value based on the class distribution;

[0126] VS represents Vector-scaling loss, that is, the vector-scaling loss function;

[0127] DDC represents Data dependent contraction loss, that is, the data-dependent contraction loss function.

[0128] P-AUC represents the existing pair form of AUC;

[0129] I-AUC, denotes the AUC in the existing instance-level form;

[0130] AUC+CE, denotes the combination of AUC and CE techniques;

[0131] AUC-Warm, denotes AUC Warmup, i.e., AUC loss with warm-up strategy;

[0132] AUC-M, denotes AUC Margin, i.e., AUC margin loss;

[0133] CTAUC, denotes Compositional-training AUC, i.e., AUC loss using compositional training.

[0134] Training data of the experiment:

[0135] Since the collection of the data set of the above embodiment requires a large amount of cost, in order to verify the effectiveness of the method of the present application at low cost, Cat&Dog, CIFAR-10, CIFAR-100, STL-10, PatchCamelyon are selected to perform training experiments of binary classification. For the training set, the multi-class classification data set is divided into two classes (the first half of the classes are marked as 0, and the second half of the classes are marked as 1). Assuming that the positive class is the minority class, the imbalance ratio is set to 0.01. Then, according to the specified imbalance ratio, the samples in the positive class are randomly deleted to obtain an imbalanced binary classification training set.

[0136] Training process of the experiment:

[0137] First, for each batch of data, the predicted value is obtained by forward propagation of the network, and the loss value is calculated by the first total loss function together with the label;

[0138] Then, the loss value is used for the first time back propagation to obtain the gradient of the model parameters, and then the approximate solution of the perturbation parameter is calculated, i.e., ;

[0139] Second, the model parameters are perturbed, and the predicted value is obtained by forward propagation again, and the loss value is calculated by the second loss function together with the label;

[0140] Next, the gradient of the parameter is calculated by back propagation again, and the parameter is updated using the gradient ascent algorithm (i.e., Stochastic Gradient Descent-Ascent, abbreviated as SGDA algorithm), and the other parameters are updated using the gradient descent method.

[0141] The results of the experiment are shown in Table 1:

[0142] Table 1

[0143]

[0144] The best results in Table 1 are marked in grey, and the suboptimal results are underlined. It can be seen that the method of the present application (ISA-AUC) achieves the best performance on all data sets. Therefore, the method of the present application can not only reduce the difficulty of solving the model training process, so that the model converges faster, but also effectively enhance the generalization ability of the model.

[0145] In general, the inventors of the present application found that the instance-level form of AUC optimization performs poorly in terms of generalization due to the sharp loss surface. In the accuracy-oriented learning, sharpness-aware minimization is a widely used technique to improve the generalization ability by achieving a flat loss surface. Therefore, it is a natural choice to smooth the loss surface in AUC optimization. However, applying I-AUC directly to the original sharpness-aware minimization method leads to a complex optimization problem. To solve this problem, the present application extends the sharpness-aware minimization method and proposes a min-max sharpness-aware minimization method based on the regularization problem and the min-max theorem (i.e., the image classification training method of the present application) to adapt to the complexity of I-AUC optimization. This method can obtain a flatter loss surface, which not only reduces the difficulty of training optimization, but also significantly improves the generalization ability of the model.

[0146] It should be noted that although the above describes the steps in a specific order, it does not mean that the steps must be performed in the above specific order. In fact, some of these steps can be performed concurrently, or even in a different order, as long as the desired function can be achieved.

[0147] The present application can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions embodied therewith, which instructions are used to program processors to implement the various aspects of the present application.

[0148] A computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves.

[0149] Embodiments of the application have been described above, with the understanding that these embodiments are exemplary only, and are not restrictive, in terms of the scope of the embodiments disclosed. Many modifications and variations of the described embodiments are possible, in light of the above teachings, without departing from the scope and spirit of the described embodiments. The choice of words in this document is intended to best explain the principles of the embodiments, practical application, or technical improvement in the art, or to enable others skilled in the art to utilize the embodiments disclosed herein.

Claims

1. An image classification training method, characterized in that: include: Obtain a binary classification training set of images, which includes multiple samples, each of which includes a sample image and a label, where the label indicates the category to which the corresponding sample image belongs. In the training set, the label space has only two categories and the number of samples in the two categories is unbalanced. Obtain samples of the current batch from the training set, input the sample images of the current batch into the image classification model, determine the original classification prediction value of the input image using the model parameters of the model, and determine a first gradient with respect to the model parameters based on the original classification prediction value, the label, and a preset first total loss function; Mapping the first gradient with respect to the model parameter using a preset scrambling hyperparameter to determine the perturbation corresponding to the model parameter; Add the model parameters to the disturbance to obtain the disturbance parameters of the model; Determine the perturbed classification prediction value of the sample images of the current batch using the perturbation parameters of the model, and determine the second gradient of the model parameters and the gradient of the learnable parameters in the second total loss function based on the perturbation prediction value, the label, and a preset second total loss function; According to the second gradient of the model parameters and the gradient of the learnable parameters in the second total loss function, the model parameters and the learnable parameters in the second total loss function are updated with minimizing the loss value of the second total loss function as the optimization goal.

2. The method according to claim 1, characterized in that The perturbation is obtained by dividing the first gradient with respect to the model by a preset perturbation hyperparameter.

3. The method according to claim 1 or 2, characterized in that The disturbance is calculated as follows: in, represents disturbance, represents the model parameters, represents the first total loss function, represents the gradient, represents the first gradient about the model, represents the preset scrambling hyperparameters, .

4. The method according to claim 3, characterized in that The first loss function is: The second loss function is: in, represents the number of the sample, Indicates the number of samples in the current batch, , represents the model parameters, 、 、 Representation function The learnable parameters set in , represents a sample image, Indicates a label, Express expectations about the sample; Represents an indicator function. When the condition in the brackets is met, the output of the indicator function is 1, otherwise it is 0; Represents a sample image Corresponding labels; Indicates the use of model parameters Determined sample images The confidence level that a class belongs to.

5. The method according to claim 4, characterized in that During the training process, model parameters and learnable parameters are and learnable parameters Use gradient descent to update the learnable parameters Update using gradient ascent.

6. An image classification method, characterized in that: The method comprises: Obtaining a trained image classification model trained according to the method of any one of claims 1 to 5; An image to be classified is acquired, and the image to be classified is classified using the trained image classification model.

7. A method for detecting anomalies, characterized in that: The method comprises: Obtaining a trained image classification model trained according to the method of any one of claims 1 to 5, wherein the trained image classification model is trained using a preset anomaly detection classification training set, and labels of samples in the anomaly detection classification training set are divided into two categories, one being a normal category and the other being an abnormal category; Obtain an image to be classified, classify the image to be classified using the trained image classification model, and determine whether there is an abnormality.

8. A computer program product comprising a computer program / instruction, which implements the steps of the method according to any one of claims 1 to 7 when executed by a processor.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: one or more processors; as well as a memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1 to 7 by executing the executable instructions.

Images