AUC optimization-based sharpness perception minimization image classification training method

Abstract

The invention provides an AUC optimization-based sharpness perception minimization image classification training method, and the method comprises the steps: carrying out the adjustment of disturbance adopted in a sharpness perception minimization technology, carrying out the mapping of a first gradient related to a model parameter through a preset scrambling hyper-parameter, determining the disturbance corresponding to the model parameter, and carrying out the optimization of the sharpness perception minimization. Therefore, the complex minimum-maximum-minimum-maximum optimization problem caused by direct adaptation of the instance-level form I-AUC and the sharpness perception minimization technology is avoided, the model training time is effectively shortened, and the energy consumption is reduced; meanwhile, by means of the sharpness perception minimization technology, the generalization ability of the model is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, in particular to the field of image classification technology based on neural networks, and more particularly to an image classification training method based on AUC optimization and sharpness perception minimization. Background Art

[0002] In real-world datasets, long-tail distributions often occur, that is, there are more samples in the head category and fewer samples in the tail category. In this case, traditional evaluation metrics (such as accuracy) usually ignore the performance of the tail and cannot effectively measure the performance of the model. Therefore, the area under the ROC curve (AUC) has gradually become a popular evaluation metric because of its insensitivity to category distribution and is widely used in many long-tail scenarios, such as financial fraud detection, disease diagnosis, and anomaly detection.

[0003] At present, there are two main methods for optimizing AUC. The original AUC optimization method based on the principle of empirical risk minimization is called paired form (P-AUC). Paired form refers to the form of sample pairs composed of positive and negative classes to guide model training, aiming to maximize the average pairwise ranking accuracy between positive and negative classes. Although the paired form can effectively optimize the unbiased estimation of the proxy indicator, its time complexity for each iteration is ,in, and denote the number of positive and negative examples, respectively. In view of this, the literature [1] reformulated the original problem as an instance-level minimax problem, called the instance-level form (I-AUC). The instance-level form refers to a single sample as an instance, and the label in the instance is used to guide model training. Compared with the pairwise form, the time cost of each iteration of the instance-level form is linear, but due to the complexity of the minimax problem, its convergence speed is slower.

[0004] From a geometric perspective, both frameworks for AUC optimization suffer from poor generalization. As pointed out in [2], the geometry of the loss function, especially its flatness, is closely related to generalization. However, since the AUC loss function is much more complex than the traditional loss function, it naturally forms a sharp loss surface, which leads to poor generalization. For the traditional loss, the sharpness-aware minimization method [3] (Sharpness-aware Minimization, abbreviated as SAM) has been shown to help obtain a flatter loss surface by simultaneously minimizing the loss value and the loss sharpness value, thereby improving the generalization ability of the model.

[0005] In the field of binary image classification, when optimizing AUC at the instance level, it is challenging to directly adopt the existing sharpness-aware minimization method, because for the instance level, direct adaptation will lead to a complex minimax-minimax optimization problem that is almost impossible to solve.

[0006] The information of the above cited literature is as follows:

[0007] [1] Ying, Y.; Wen, L.; and Lyu, S. 2016. Stochastic Online AUCMaximization. In Annual Conference on Neural Information Processing Systems, 451–459.

[0008] [2] Keskar, NS; Mudigere, D.; Nocedal, J.; Smelyanskiy, M.; andTang, PTP 2017. On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima. In International Conference on LearningRepresentations, 1–16.

[0009] [3] Foret, P.; Kleiner, A.; Mobahi, H.; and Neyshabur, B. 2021. Sharpness-aware Minimization for Efficiently Improving Generalization. InInternational Conference on Learning Representations, 1–20.

[0010] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. The relevant information is submitted and disclosed together with the present invention solution. If there is no evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art. Summary of the invention

[0011] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide an image classification training method and an image classification method.

[0012] The objective of the present invention is achieved through the following technical solutions:

[0013] According to a first aspect of the present invention, there is provided an image classification training method, comprising: obtaining a binary classification training set of images, which comprises a plurality of samples, wherein the samples comprise sample images and labels, wherein the labels indicate the categories to which the corresponding sample images belong, wherein the label space in the training set is provided with only two categories and the number of samples of the two categories is unbalanced; obtaining samples of a current batch from the training set, inputting the sample images of the current batch into an image classification model, determining an original classification prediction value of the input image using model parameters of the model, and determining a first gradient with respect to the model parameters according to the original classification prediction value, the labels and a preset first total loss function; and determining a first gradient with respect to the model parameters using a preset scrambling hyperparameter. The first gradient of the model parameters is mapped to determine the disturbance corresponding to the model parameters; the model parameters are added to the disturbance to obtain the disturbance parameters of the model; the disturbance parameters of the model are used to determine the classification prediction value after disturbance of the sample images of the current batch, and the gradient of the second gradient of the model parameters and the gradient of the learnable parameters in the second total loss function are determined according to the classification prediction value after disturbance, the label and the preset second total loss function; according to the second gradient of the model parameters and the gradient of the learnable parameters in the second total loss function, the model parameters and the learnable parameters in the second total loss function are updated with the loss value of the second total loss function as the optimization target. The technical solution of this embodiment can at least achieve the following beneficial technical effects: the present invention maps the first gradient by scrambling the hyperparameter to obtain the disturbance corresponding to the model parameters, and the setting of the disturbance can adjust the optimization process of the original sharpness perception minimization technology from the minimax-minimax optimization problem to the minimax optimization problem, and the computational complexity of the minimax optimization problem is much lower than that of the minimax-minimax optimization problem, which effectively reduces the time of model training and reduces energy consumption; at the same time, with the help of the sharpness perception minimization technology, the generalization ability of the model is effectively improved.

[0014] Optionally, the disturbance is obtained by dividing the first gradient with respect to the model by a preset scrambling hyperparameter.

[0015] Optionally, the disturbance is calculated in the following manner:

[0016]

[0017] in, represents disturbance, represents the model parameters, represents the first total loss function, represents the gradient, represents the first gradient of the model, represents the preset scrambling hyperparameters, The technical solution of this embodiment can at least achieve the following beneficial technical effects: Through reasoning, the inventors have proved that the disturbance obtained in this way can be used as a simplified substitute for the disturbance of the original sharpness perception minimization technology, significantly reducing the difficulty of solving the problem.

[0018] Optionally, the first loss function is: .

[0019] Optionally, the second loss function is:

[0020]

[0021] in, represents the number of the sample, Indicates the number of samples in the current batch, , represents the model parameters, , , Representation function The learnable parameters set in , represents a sample image, Indicates the label, Express expectations about the sample; Represents an indicator function. When the condition in the brackets is met, the output of the indicator function is 1, otherwise it is 0; Represents a sample image The corresponding label; Indicates the use of model parameters Determined sample images The technical solution of this embodiment can at least achieve the following beneficial technical effects: the total loss function incorporates the content of classification loss minimization and sharpness perception minimization technology, and the model parameters can be trained from multiple aspects to better improve the classification performance and generalization of the model.

[0022] Optionally, during the training process, model parameters and learnable parameters and learnable parameters Use gradient descent to update the learnable parameters The technical solution of this embodiment can at least achieve the following beneficial technical effects: after adopting the perturbation and total loss function of the present invention, by updating the model parameters, learnable parameters and learnable parameters Use gradient descent to update and learn the parameters Using gradient ascent for updates simplifies the computational process of minimizing classification loss and sharpness perception, allowing the model to converge faster and improving the performance of the trained model.

[0023] According to a second aspect of the present invention, there is provided an image classification method, the method comprising: obtaining a trained image classification model trained according to the method described in the first aspect; obtaining an image to be classified, and classifying the image to be classified using the trained image classification model.

[0024] According to a third aspect of the present invention, there is provided an anomaly detection method, the method comprising: obtaining a trained image classification model trained according to the method described in the first aspect, wherein the trained image classification model is trained by a preset anomaly detection classification training set, and the labels of the samples in the anomaly detection classification training set are divided into two categories, one is a normal category, and the other is an abnormal category; obtaining an image to be classified, and classifying the image to be classified using the trained image classification model to determine whether there is an abnormality.

[0025] According to a fourth aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a memory, wherein the memory is used to store executable instructions; the one or more processors are configured to implement the steps of the method described in the first, second or third aspect by executing the executable instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0027] Figure 1 4 is a flow chart of an image classification training method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0029] As mentioned in the background technology section, in the field of binary image classification, when optimizing AUC for instance-level forms, there are certain challenges in directly using existing sharpness-aware minimization methods. Because for instance-level forms, direct adaptation will lead to a complex minimax-minimax optimization problem that is almost impossible to solve. In this regard, the method of the present invention adjusts the perturbation used in the sharpness-aware minimization technology, maps the first gradient about the model parameters using preset scrambling hyperparameters, and determines the perturbation corresponding to the model parameters, thereby simplifying the minimax-minimax optimization problem to a minimax optimization problem, effectively reducing the model training time and energy consumption; at the same time, with the help of sharpness-aware minimization technology, the generalization ability of the model is effectively improved.

[0030] In order to illustrate the problem faced by the present invention, the inventor first explains the reasons why it is difficult to directly apply the instance-level formal I-AUC to the original sharpness perception minimization technology by combining formulas.

[0031] The expected loss of I-AUC is defined as:

[0032]

[0033] in, represents the expected function, Indicates that each sample The corresponding loss function is, represents the model parameters, , , Represents the learnable parameters set within the loss function.

[0034] The empirical loss of I-AUC is defined as:

[0035]

[0036] in, represents the number of the sample, Indicates the number of samples in a batch, or batch size (BatchSize).

[0037] Directly applying the instance-level formal I-AUC to the original sharpness-aware minimization technique results in the following complex optimization problem:

[0038]

[0039] in, represents the model parameters, represents the perturbation parameter vector set in the original sharpness-aware minimization technique, The elements in each dimension have a mean of 0 and a variance of Gaussian distribution, represents the regularization coefficient.

[0040] It can be seen that this leads to a minimax-minimax optimization problem that is almost impossible to solve. If applied directly, it will make it difficult for the model training process to converge.

[0041] In this regard, the inventor considered performing reasoning optimization to find alternative feasible solutions to achieve similar effects. The reasoning process is as follows:

[0042] make , and then give the following assumptions:

[0043] Assume 1 that For a about of Weakly concave function, that is, for any About A concave function, where .

[0044] Therefore, under the above assumptions, we can directly use the maximum and minimum theorem to simplify the above objectives and obtain:

[0045]

[0046] For the inner maximization problem, the first-order Taylor expansion can be used to approximate the solution :

[0047]

[0048]

[0049]

[0050]

[0051] in, represents the gradient, represents the L2 distance function, Indicates transpose.

[0052] As a result, the optimization process of the original sharpness-aware minimization technology was adjusted from a minimax-minimax optimization problem to a minimax optimization problem, which effectively reduced the model training time and energy consumption. At the same time, with the help of sharpness-aware minimization technology, the generalization ability of the model was effectively improved.

[0053] In order to further understand the principle of the present invention, it is described below in conjunction with various implementation modes.

[0054] Implementation 1:

[0055] This implementation is explained from the perspective of model structure, training data, loss function, training process and model application.

[0056] 1. Model Structure

[0057] According to one embodiment of the present invention, the image classification model includes a feature extractor for extracting image features from an input image and a classification layer for determining a classification prediction value corresponding to the input image (ie, the probability that the input image belongs to a corresponding category) based on the image features.

[0058] Optionally, the image classification model can adopt an existing classification model, for example, a binary image classification model constructed by a ResNet model, an EfficientNet model, a DenseNet model or a VGGNet model. Of course, a model customized by the implementer can also be used, for example, a feature extractor is constructed by a convolutional layer, an attention layer, a residual layer, a linear layer and a classification layer or a combination thereof, and a binary classification layer is constructed by a fully connected layer and a Sigmoid layer. The embodiment of the present invention does not impose any restrictions on this. After obtaining the model, the parameters of the image classification model can be randomly initialized as the parameters of the model used for the first training.

[0059] 2. Training Data

[0060] According to one embodiment of the present invention, samples are randomly sampled from an unbalanced binary classification image data set to form a training set, and then the samples of the training set are randomly divided into multiple batches. The data set of this embodiment is a data set for contraband detection training, specifically collected for detecting contraband in security inspection scenes, and the data set includes multiple samples, the samples include sample images and labels, the labels indicate the category to which the corresponding sample images belong, and the label space in the data set is only provided with two categories and the number of samples of the two categories is unbalanced, wherein one category is a normal class, indicating that there are no contraband in the image; the other category is an abnormal class, indicating that there are contraband in the image. Usually, the samples of the normal class are relatively more, and the samples of the abnormal class are relatively less, so as to form an imbalance in quantity. The data set can adopt an existing data set, or a self-made data set collected and labeled by the implementer, and the present invention does not impose any restrictions on this.

[0061] In addition, in order to facilitate subsequent understanding, some letters are defined in combination with the training set:

[0062] In the image binary classification problem, the training set From the distribution ,in, Indicates input, express dimensional real number space, and Respectively represent the height, width and number of channels of the image; Indicates the label, represents the 0th category, Indicates that in class 1, the number of samples in the two classes is unbalanced; represents the sample size. and Represents the number of samples in class 0 and class 1 respectively. The score function is defined as ,in, Represents model parameters, scores express The probability of belonging to class 0.

[0063] 3. Loss Function

[0064] In the method of the present invention, two total loss functions are provided, wherein the first total loss function is only used to obtain the first gradient with respect to the model parameters and will not actually be used to guide the updating of the model parameters, which is guided by the second total loss function.

[0065] According to one embodiment of the present invention, the first total loss function is:

[0066] ,

[0067] in, represents the number of the sample, Indicates the number of samples in the current batch, , represents the model parameters, , , Representation function The learnable parameters set in , represents a sample image, Indicates the label, Express expectations about the sample; Represents an indicator function. When the condition in the brackets is met, the output of the indicator function is 1, otherwise it is 0. ,For example, Indicates that when the label value of the sample is 1 ,otherwise ; Represents a sample image The corresponding label, Indicates the use of model parameters Determined sample images The confidence level of a class. In a binary classification scenario, assuming there are two classes, class 0 and class 1, we can The result of is set as the confidence that the image belongs to class 0, then the confidence of the other class can be calculated from 1- get.

[0068] According to one embodiment of the present invention, the second total loss function is:

[0069]

[0070] in, represents the disturbance of the method of the present invention, represents the preset scrambling hyperparameters, represents the L2 distance function, Represents the regularization coefficient. The explanation of the remaining parameters in the second total loss function can refer to the introduction of the first total loss function, which will not be repeated here.

[0071] Preferably, the perturbation used in the second total loss function is calculated as follows:

[0072]

[0073] in, represents disturbance, represents the model parameters, represents the first total loss function, represents the gradient, represents the preset scrambling hyperparameters, . Combined with the previous reasoning process, it can be known that the inventors have proved that the perturbation obtained in this way can be used as a simplified substitute for the perturbation of the original sharpness-aware minimization technique. The setting of this perturbation can adjust the optimization process combining the instance-level formal I-AUC and the sharpness-aware minimization technique from a minimax-minimax optimization problem to a minimax optimization problem, significantly reducing the difficulty of solving it.

[0074] 4. Training Process

[0075] According to one embodiment of the present invention, referring to Figure 1 , an image classification training method is provided, comprising steps S1-S7. In order to better understand the present invention, each step is described in detail below in conjunction with a specific embodiment.

[0076] Step S1: Obtain a binary classification training set of images, which includes multiple samples. The samples include sample images and labels. The labels indicate the category to which the corresponding sample images belong. In the training set, the label space has only two categories and the number of samples in the two categories is unbalanced.

[0077] According to one embodiment of the present invention, the binary classification training set can be obtained from the data set in the second part. Thus, a training set for detecting contraband in security inspection scenes can be obtained. Among them, one category is the normal class, indicating that there are no contraband in the image; the other category is the abnormal class, indicating that there are contraband in the image; and the samples of the normal class are more than the samples of the abnormal class. It should be understood that it is also feasible to set more samples of the abnormal class, which only requires selecting more samples of the abnormal class to be added to the training set when sampling from the data set.

[0078] Step S2: Obtain samples of the current batch from the training set, input the sample images of the current batch into the image classification model, use the model parameters of the model to determine the original classification prediction value of the input image, and determine the first gradient of the model parameters based on the original classification prediction value, label and a preset first total loss function.

[0079] According to an embodiment of the present invention, a batch of samples can be obtained from the training set each time, and the samples in the batch can be used to complete a model training process (i.e., a process of updating model parameters). For example, assuming that there are 100 samples in the training set, and one epoch is divided into 10 batches, then each batch is divided into 10 samples, and the model is trained using the samples in each batch.

[0080] During training, each sample image of each batch is input into the image classification model, the image features are extracted using the feature extractor, and the original classification prediction value corresponding to the sample image is obtained using the classifier based on the image features. ) Calculate the loss value according to the original classification prediction value and label corresponding to the sample images of this batch; and perform the first back propagation based on the loss value to obtain the gradient of the model parameters (to distinguish, it is called the first gradient of the model parameters), which can be expressed as:

[0081]

[0082] in, Represents the first total loss function.

[0083] Step S3: Map the first gradient about the model parameter using the preset scrambling hyperparameter to determine the disturbance corresponding to the model parameter.

[0084] According to one embodiment of the present invention, the perturbation of this step is obtained by dividing the first gradient of the model by a preset scrambling hyperparameter, which is expressed as:

[0085]

[0086] in, represents disturbance, represents the model parameters, represents the first total loss function, represents the gradient, represents the preset scrambling hyperparameters, . The specific value of can be set by the implementer when training the model based on experience or a better value selected from multiple actual training sessions.

[0087] Step S4: Add the model parameters to the disturbance to obtain the disturbance parameters of the model.

[0088] According to one embodiment of the present invention, the disturbance parameter of the model is: It should be understood that in the field of neural networks, model parameters are multidimensional, with multiple components, not scalars. Correspondingly, perturbations are also multidimensional, with the same dimensions as model parameters. This step is equivalent to adding the corresponding perturbation component in the perturbation to each parameter in the model parameters, thereby obtaining the perturbation parameters of the model.

[0089] Step S5: Determine the post-perturbation classification prediction value of the sample images of the current batch using the perturbation parameters of the model, and determine the second gradient of the model parameters and the gradient of the learnable parameters in the second total loss function based on the post-perturbation classification prediction value, the label and the preset second total loss function.

[0090] According to one embodiment of the present invention, after the model parameters are perturbed, a prediction value (i.e., a classification prediction value after perturbation) is obtained through a forward propagation based on the perturbation parameters of the model, and the loss value is calculated together with the label through the second total loss function. The second gradient of the model refers to the gradient of the second total loss function calculated based on the sample images and labels of the current batch relative to each model parameter. The second gradient can be calculated as follows:

[0091]

[0092] in, Represents the second total loss function.

[0093] Step S6: According to the second gradient of the model parameters and the gradient of the learnable parameters in the second total loss function, the model parameters and the learnable parameters in the second total loss function are updated with the goal of minimizing the loss value of the second total loss function.

[0094] According to one embodiment of the present invention, during the training process, the model parameters and the learnable parameters are and learnable parameters Use gradient descent to update the learnable parameters Update using gradient ascent.

[0095] In schematic form, the model parameters are iteratively updated in the following manner:

[0096]

[0097] in, Indicates The model parameters obtained by batch updating, Indicates The model parameters used before batch update, represents the learning rate, Represents the second gradient. For example, it can be set to 0.01, 0.001 or 0.0001, etc. The specific value can be set according to the needs of the implementer.

[0098] Schematic, learnable parameters Update as follows:

[0099]

[0100] in, Indicates Learnable parameters obtained by batch update , Indicates Learnable parameters used before batch update , represents the learning rate, Represents learnable parameters gradient.

[0101] Schematic, learnable parameters Update as follows:

[0102]

[0103] in, Indicates Learnable parameters obtained by batch update , Indicates Learnable parameters used before batch update , represents the learning rate, Represents learnable parameters gradient.

[0104] Schematic, learnable parameters Update as follows:

[0105]

[0106] in, Indicates Learnable parameters obtained by batch update , Indicates Learnable parameters used before batch update , represents the learning rate, Represents learnable parameters gradient.

[0107] Step S7: Repeat steps S2-S6 until the preset training requirements are met to obtain a trained image classification model.

[0108] According to one embodiment of the present invention, in step S7, check whether the preset training requirements are met. If not, enter the next batch and return to step S2; if so, stop training to obtain a trained model. The preset training requirements are, for example, the requirements for the total training batches or rounds, such as training 10,000 batches or 20 rounds, etc.; alternatively, the preset training requirements can be set to reduce the loss value to a preset threshold range and maintain a preset number of batches within the threshold range. Of course, other training requirements set by the implementer can also be adopted, and the present invention is not limited to this.

[0109] 5. Model Application

[0110] According to an embodiment of the present invention, a method for detecting anomalies is provided, specifically a method for detecting contraband, the method comprising:

[0111] A1. Obtain a trained image classification model trained according to the method of the above embodiment, wherein the trained image classification model is trained by a training set for detecting contraband in security inspection scenarios;

[0112] A2. Obtain an image to be classified, and classify the image to be classified using the trained image classification model to determine whether there are contraband items.

[0113] Implementation Method 2

[0114] The difference between this embodiment and embodiment 1 is that the data set of this embodiment is a data set for identifying a certain disease. For example, it is a data set for identifying whether there is a lesion in the fundus, wherein the sample image is a fundus image taken by a fundus camera, the normal class in the label represents the absence of fundus lesions, and the abnormal class represents the presence of fundus lesions (such as leopard-shaped fundus, retinal detachment, retinal holes, macular lesions, etc.).

[0115] Correspondingly, by using the training set to train the model, a trained image classification model for determining whether fundus disease exists can be obtained.

[0116] Implementation 3

[0117] The difference between this embodiment and embodiments 1 and 2 is that the data set of this embodiment is a data set for identifying whether a certain device (or a component of a device) has an abnormality. The sample image is an image of the device (or a component of a device) taken by a camera, the normal class in the label represents that the device (or a component of a device) is normal, and the abnormal class represents that an abnormal working condition exists (such as a broken fixture, a broken prop, a protective cover not closed, a loose bolt, etc.).

[0118] Correspondingly, by using the training set to train the model, a trained image classification model can be obtained for determining whether a certain device (or a component of a device) has an abnormality.

[0119] In order to verify the effect of the present invention, the inventors also conducted the following comparative experiments:

[0120] Existing methods for comparison:

[0121] CE, stands for Cross-Entropy, which is the cross entropy loss function;

[0122] Focal, which means Focal loss, i.e. focal loss function;

[0123] CB, stands for Class-Balanced loss, that is, class-balanced loss function;

[0124] LDAM+DRW means the combination of LDAM and DRW technologies, where LDAM stands for Label-distribution-aware margin loss, i.e., the marginal loss function of label distribution awareness; DRW stands for Deferred re-weighting, i.e., the loss function of the delayed re-weighting method;

[0125] LA stands for Logit adjustment, which is the loss function of the model output adjustment method (or score adjustment loss function). It is a loss function that calculates the cross entropy after adjusting the model output logical value based on the class distribution;

[0126] VS, stands for Vector-scaling loss, that is, vector scaling loss function;

[0127] DDC stands for Data dependent contraction loss, that is, data related compression loss function.

[0128] P-AUC, represents the existing pairwise AUC;

[0129] I-AUC, represents the AUC of the existing instance-level form;

[0130] AUC+CE, which means the combination of AUC and CE techniques;

[0131] AUC-Warm, which means AUC Warmup, i.e. AUC loss with warm-up strategy;

[0132] AUC-M, stands for AUC Margin, i.e. AUC marginal loss;

[0133] CTAUC stands for Compositional-training AUC, which means the AUC loss using combined training.

[0134] Experimental training data:

[0135] Since the collection of the data sets of the above embodiments requires a lot of cost, in order to verify the effectiveness of the method of the present invention at a low cost, Cat&Dog, CIFAR-10, CIFAR-100, STL-10, and PatchCamelyon are selected as an alternative to conduct binary classification training experiments. For the training set, the multi-category classification data set is divided into two categories (the first half of the categories are marked as 0, and the second half of the categories are marked as 1). Assuming that the positive class is the minority class, the imbalance ratio is set to 0.01. Then, according to the specified imbalance ratio, the samples in the positive class are randomly deleted to obtain an unbalanced binary classification training set.

[0136] Experimental training process:

[0137] First, for each batch of data, the predicted value is obtained through the network forward propagation, and the loss value is calculated together with the label through the first total loss function;

[0138] Then, the loss value is used for the first back propagation to obtain the gradient of the model parameters, and then the approximate solution of the perturbation parameters is calculated, that is, ;

[0139] Secondly, the model parameters are perturbed, and the predicted values ​​are obtained through another forward propagation, and the loss value is calculated together with the label through the second loss function;

[0140] Next, the parameters are calculated by back propagation again The gradient of the parameter Use the gradient ascent algorithm (Stochastic Gradient Descent-Ascent, SGDA algorithm for short) for updating, and use the gradient descent method to update other parameters.

[0141] The experimental results are shown in Table 1:

[0142] Table 1

[0143]

[0144] In Table 1, the best results are marked in gray, and the suboptimal results are underlined. It can be seen that the method of the present invention (ISA-AUC) has achieved the best performance on all data sets. Therefore, the method of the present invention can not only reduce the difficulty of solving the model training process and make the model converge faster, but also effectively enhance the generalization ability of the model.

[0145] In general, the inventors of the present invention have found that the instance-level formal framework of AUC optimization exhibits poor generalization ability due to the sharp loss surface. In accuracy-oriented learning, sharpness-aware minimization methods are a commonly used technique that improves generalization ability by achieving a flat loss surface. Therefore, smoothing the loss surface in AUC optimization becomes a natural choice. However, directly applying I-AUC to the original sharpness-aware minimization method will lead to complex optimization problems. In order to solve this problem, the present invention extends the sharpness-aware minimization method and proposes a minimax sharpness-aware minimization method (i.e., the image classification training method of the present invention) based on regularization problems and the minimax theorem to adapt to the complexity of I-AUC optimization. This method can obtain a flatter loss surface, which significantly improves the generalization ability of the model while reducing the difficulty of training optimization.

[0146] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0147] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0148] A computer-readable storage medium may be a tangible device that holds and stores instructions used by an instruction execution device. Computer-readable storage media may include, for example, but are not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0149] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. An image classification training method, characterized in that: include: Obtain a binary classification training set of images, which includes multiple samples, where the samples include sample images and labels, and the labels indicate the categories to which the corresponding sample images belong. In the training set, there are only two categories in the label space and the number of samples of the two categories is unbalanced; Obtain samples of the current batch from the training set, input the sample images of the current batch into the image classification model, determine the original classification prediction value of the input image using the model parameters of the model, and determine the first gradient of the model parameters according to the original classification prediction value, the label and the preset first total loss function; Mapping the first gradient about the model parameter using the preset scrambling hyperparameter to determine the disturbance corresponding to the model parameter; Add the model parameters to the disturbance to obtain the disturbance parameters of the model; Determine the perturbed classification prediction value of the sample images of the current batch using the perturbation parameters of the model, and determine the second gradient of the model parameters and the gradient of the learnable parameters in the second total loss function according to the perturbation classification prediction value, the label and the preset second total loss function; According to the second gradient about the model parameters and the gradient of the learnable parameters in the second total loss function, the model parameters and the learnable parameters in the second total loss function are updated with minimizing the loss value of the second total loss function as the optimization objective.

2. The method according to claim 1, characterized in that The perturbation is obtained by dividing the first gradient with respect to the model by a preset scrambling hyperparameter.

3. The method according to claim 1 or 2, characterized in that: The disturbance is calculated as follows: in, represents disturbance, represents the model parameters, represents the first total loss function, represents the gradient, represents the first gradient of the model, represents the preset scrambling hyperparameters, .

4. The method according to claim 3, characterized in that The first loss function is: The second loss function is: in, represents the number of the sample, Indicates the number of samples in the current batch, , represents the model parameters, , , Representation function The learnable parameters set in , represents a sample image, Indicates the label, Express expectations about the sample; Represents an indicator function. When the condition in the brackets is met, the output of the indicator function is 1, otherwise it is 0; Represents a sample image The corresponding label; Indicates the use of model parameters Determined sample images Confidence in belonging to a class.

5. The method according to claim 4, characterized in that During the training process, model parameters and learnable parameters are and learnable parameters Use gradient descent to update the learnable parameters Update using gradient ascent.

6. An image classification method, characterized in that: The method comprises: Obtain a trained image classification model trained according to the method of any one of claims 1 to 5; An image to be classified is obtained, and the image to be classified is classified using the trained image classification model.

7. An anomaly detection method, characterized in that: The method comprises: Obtaining a trained image classification model trained according to the method of any one of claims 1 to 5, wherein the trained image classification model is trained by a preset anomaly detection classification training set, and labels of samples in the anomaly detection classification training set are divided into two categories, one is a normal category, and the other is an abnormal category; An image to be classified is acquired, and the image to be classified is classified using the trained image classification model to determine whether there is an abnormality.

8. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: one or more processors; as well as A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-7 by executing the executable instructions.

Images