Image classification model training method, image classification method, device and computer equipment

Abstract

The application relates to an image classification model training method, an image classification method, an image classification device and computer equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: training an adversarial image generation model to be trained according to a first image classification model and sample images used for training the first image classification model, so as to obtain a target adversarial image generation model; generating a first adversarial image with a deviation from the sample images less than a preset deviation based on the target adversarial image generation model; retraining the first image classification model according to the first adversarial image, so as to obtain a second image classification model; and combining the target adversarial image generation model and the second image classification model, so as to obtain a target image classification model. The method can improve the robustness of the image classification model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an image classification model training method, an image classification method, an apparatus, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the advancement of artificial intelligence, AI-based image classification has been widely applied in various fields. Among related technologies, image classification models are typically based on neural networks, such as DNN (Deep Nueral Network) models. However, research has found that even minute image interference, imperceptible to the human eye, can severely impact image classification models, meaning that the robustness of image classification models in these technologies is relatively weak. Summary of the Invention

[0003] Therefore, it is necessary to address the technical problem of weak robustness of the aforementioned image classification models by providing an image classification model training method, image classification method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the robustness of image classification models.

[0004] Firstly, this application provides a method for training an image classification model, including:

[0005] Based on the first image classification model and the sample images used to train the first image classification model, the adversarial image generation model to be trained is trained to obtain the target adversarial image generation model.

[0006] Based on the target adversarial image generation model, a first adversarial image with a deviation less than a preset deviation from the sample image is generated;

[0007] Based on the first adversarial image, the first image classification model is retrained to obtain a second image classification model.

[0008] The target adversarial image generation model and the second image classification model are combined to obtain the target image classification model.

[0009] In one embodiment, the step of training an adversarial image generation model to obtain a target adversarial image generation model based on a first image classification model and sample images used to train the first image classification model includes:

[0010] The sample image is input into the adversarial image generation model to be trained to obtain classification prediction information of the sample image corresponding to different preset categories, and to generate a second adversarial image with a deviation from the sample image that is smaller than the preset deviation.

[0011] The second adversarial image is input into the first image classification model to obtain classification prediction information of the second adversarial image corresponding to the different preset categories;

[0012] Based on the classification prediction information of the sample image and the classification prediction information of the second adversarial image, the first loss of the adversarial image generation model to be trained is determined, and based on the classification prediction information of the sample image and the true category, the second loss of the adversarial image generation model to be trained is determined.

[0013] Based on the first loss and the second loss of the adversarial image generation model to be trained, the adversarial image generation model to be trained is iteratively trained to obtain the target adversarial image generation model.

[0014] In one embodiment, the step of retraining the first image classification model based on the first adversarial image to obtain a second image classification model includes:

[0015] A perturbation layer is added to the first image classification model to obtain a third image classification model; the perturbation layer is used to inject perturbation information into the classification prediction information generated by the third image classification model.

[0016] Based on the first adversarial image, the third image classification model is iteratively trained to obtain the second image classification model; the classification result output by the second image classification model based on the classification prediction information injected with the perturbation information is the same as the classification result output by the first image classification model.

[0017] In one embodiment, the second image classification model is obtained in the following manner:

[0018] The first parameter of the third image classification model is fixed, and the second parameter of the third image classification model is updated to obtain the updated second parameter of the third image classification model; the first parameter is the parameter of the perturbation layer in the third image classification model, and the second parameter is the parameter of the first image classification model in the third image classification model;

[0019] The updated second parameter of the third image classification model is fixed, and the first parameter of the third image classification model is updated to obtain the updated first parameter of the third image classification model.

[0020] The second image classification model is obtained based on the third image classification model under the updated first parameters and the updated second parameters.

[0021] Secondly, this application also provides a method for training an image classification model, including:

[0022] An image to be classified is acquired and input into a pre-trained target image classification model; the target image classification model includes at least an adversarial image generation model and an image classification model; the target image classification model is trained according to an image classification model training method.

[0023] The adversarial image generation model generates an adversarial image whose deviation from the image to be classified is less than a preset deviation.

[0024] The classification result of the adversarial image is determined by the image classification model, and the classification result of the image to be classified is obtained.

[0025] Thirdly, this application also provides an image classification model training device, comprising:

[0026] The first training module is used to train the adversarial image generation model to be trained based on the first image classification model and the sample images used to obtain the first image classification model, so as to obtain the target adversarial image generation model.

[0027] The image generation module is used to generate a first adversarial image with a deviation less than a preset deviation from the sample image, based on the target adversarial image generation model.

[0028] The second training module is used to retrain the first image classification model based on the first adversarial image to obtain a second image classification model.

[0029] The model combination module is used to combine the target adversarial image generation model and the second image classification model to obtain the target image classification model.

[0030] Fourthly, this application also provides an image classification model training device, comprising:

[0031] An image acquisition module is used to acquire an image to be classified and input the image to be classified into a pre-trained target image classification model; the target image classification model includes at least an adversarial image generation model and an image classification model; the target image classification model is trained according to the above-described image classification model training method.

[0032] The image generation module is used to generate an adversarial image with a deviation of less than a preset deviation from the image to be classified by the adversarial image generation model.

[0033] The image classification module is used to determine the classification result of the adversarial image through the image classification model, and obtain the classification result of the image to be classified.

[0034] Fifthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0035] Based on the first image classification model and the sample images used to obtain the first image classification model, train the adversarial image generation model to be trained to obtain the target adversarial image generation model.

[0036] Based on the target adversarial image generation model, a first adversarial image with a deviation less than a preset deviation from the sample image is generated;

[0037] Based on the first adversarial image, the first image classification model is retrained to obtain a second image classification model.

[0038] The target adversarial image generation model and the second image classification model are combined to obtain the target image classification model.

[0039] Sixthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0040] Based on the first image classification model and the sample images used to obtain the first image classification model, train the adversarial image generation model to be trained to obtain the target adversarial image generation model.

[0041] Based on the target adversarial image generation model, a first adversarial image with a deviation less than a preset deviation from the sample image is generated;

[0042] Based on the first adversarial image, the first image classification model is retrained to obtain a second image classification model.

[0043] The target adversarial image generation model and the second image classification model are combined to obtain the target image classification model.

[0044] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0045] Based on the first image classification model and the sample images used to obtain the first image classification model, train the adversarial image generation model to be trained to obtain the target adversarial image generation model.

[0046] Based on the target adversarial image generation model, a first adversarial image with a deviation less than a preset deviation from the sample image is generated;

[0047] Based on the first adversarial image, the first image classification model is retrained to obtain a second image classification model.

[0048] The target adversarial image generation model and the second image classification model are combined to obtain the target image classification model.

[0049] The aforementioned image classification model training method, image classification method, apparatus, computer device, computer-readable storage medium, and computer program product, through training a target adversarial image generation model using a first image classification model and sample images, enable the target adversarial image generation model to inherit the knowledge of the first image classification model, thereby generating a first adversarial image with a deviation from the sample image less than a preset deviation. By retraining the first image classification model using the first adversarial image, the first image classification model learns adversarial features from the target adversarial image generation model during training, thus obtaining a more robust second image classification model. By combining the target adversarial image generation model and the second image classification model, a target image classification model with even greater robustness can be obtained. Therefore, the image classification model training method based on the above process can improve the robustness of the image classification model. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating an image classification model training method in one embodiment;

[0052] Figure 2 This is a schematic diagram of the structure of a target image classification model in one embodiment;

[0053] Figure 3 This is a flowchart illustrating the steps of training an adversarial image generation model to obtain a target adversarial image generation model based on a first image classification model and sample images used to train the first image classification model, in one embodiment.

[0054] Figure 4 This is a schematic diagram of the target image classification model in another embodiment;

[0055] Figure 5 This is a flowchart illustrating an image classification method in one embodiment;

[0056] Figure 6This is a flowchart illustrating the adversarial defense training method and the image classification method for an image classification neural network in one embodiment.

[0057] Figure 7 This is a flowchart illustrating the adversarial defense training method and the image classification method for an image classification neural network in another embodiment.

[0058] Figure 8 This is a structural block diagram of an image classification model training device in one embodiment;

[0059] Figure 9 This is a structural block diagram of an image classification device in one embodiment;

[0060] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0063] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0064] In one exemplary embodiment, such as Figure 1As shown, an image classification model training method is provided. This embodiment illustrates the application of this method to a server. It is understood that this method can also be applied to terminals, and to systems including servers and terminals, and is implemented through interaction between the server and the terminal. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle devices, projection devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. In this embodiment, the method includes the following steps S102 to S108:

[0065] Step S102: Based on the first image classification model and the sample images used to train the first image classification model, train the adversarial image generation model to be trained, and obtain the target adversarial image generation model.

[0066] The first image classification model is a trained image classification model. In specific applications, the first image classification model is a neural network model, preferably a DNN (Deep Neural Network) model.

[0067] The first image classification model was trained based on a sample dataset; the sample dataset includes multiple images, each labeled with its corresponding true category.

[0068] The sample images are any number of images in the sample dataset.

[0069] The target adversarial image generation model is used to handle the same image classification task as the first image classification model, and to generate adversarial images. In specific applications, the number of parameters in the adversarial image generation model to be trained is less than the number of parameters in the first image classification model. In specific applications, the adversarial image generation model to be trained is an AET (Adversarial Example Transformer) model.

[0070] Specifically, the server performs distillation training on the adversarial image generation model to be trained based on the first image classification model and the sample images used to train the first image classification model, so that the adversarial image generation model to be trained inherits the knowledge of the first image classification model and obtains the target adversarial image generation model.

[0071] Step S104: Based on the target adversarial image generation model, generate a first adversarial image whose deviation from the sample image is less than a preset deviation.

[0072] Specifically, the server inputs the sample image into the target adversarial image generation model, and generates a first adversarial image of the sample image through the target adversarial image generation model; the deviation between the first adversarial image and the sample image is less than a preset deviation.

[0073] Step S106: Based on the first adversarial image, the first image classification model is retrained to obtain the second image classification model.

[0074] The second image classification model is used to handle the same image classification task as the first image classification model.

[0075] Specifically, the server uses the first adversarial image as input information and the category of the first adversarial image as supervision information to retrain the first image classification model, enabling it to learn the adversarial features from the target adversarial image generation model, thereby obtaining a second image classification model that can accurately classify the first adversarial image.

[0076] The category of the first adversarial image can be either the classification result of the first adversarial image under the target adversarial image generation model, or the true category obtained by manually annotating the first adversarial image.

[0077] Step S108: Combine the target adversarial image generation model and the second image classification model to obtain the target image classification model.

[0078] The target image classification model includes at least a target adversarial image generation model and a second image classification model.

[0079] Specifically, the server connects the target adversarial image generation model before the second image classification model to obtain the target image classification model. For example... Figure 2 The diagram shown is a schematic representation of the target image classification model in this embodiment.

[0080] In practical applications, the server performs image classification using a target image classification model.

[0081] In the above image classification model training method, the server trains a target adversarial image generation model using a first image classification model and sample images. This allows the target adversarial image generation model to inherit the knowledge of the first image classification model, thereby generating a first adversarial image with a deviation from the sample image that is less than a preset deviation. The server then retrains the first image classification model using the first adversarial image, enabling the first image classification model to learn adversarial features from the target adversarial image generation model during training, thus obtaining a more robust second image classification model. By combining the target adversarial image generation model and the second image classification model, the server can obtain a target image classification model with even greater robustness. Therefore, based on the above process, the image classification model training method can improve the robustness of the image classification model.

[0082] In one exemplary embodiment, such as Figure 3 As shown, step S102 above, which trains the adversarial image generation model to be trained based on the first image classification model and the sample images used to train the first image classification model, to obtain the target adversarial image generation model, specifically includes the following steps:

[0083] Step S302: Input the sample image into the adversarial image generation model to be trained to obtain the classification prediction information of the sample image corresponding to different preset categories, and generate a second adversarial image with a deviation from the sample image that is less than a preset deviation.

[0084] Step S304: Input the second adversarial image into the first image classification model to obtain classification prediction information of the second adversarial image corresponding to different preset categories.

[0085] Step S306: Determine the first loss of the adversarial image generation model to be trained based on the classification prediction information of the sample image and the classification prediction information of the second adversarial image; determine the second loss of the adversarial image generation model to be trained based on the classification prediction information of the sample image and the true category.

[0086] Step S308: Based on the first loss and the second loss of the adversarial image generation model to be trained, iteratively train the adversarial image generation model to be trained to obtain the target adversarial image generation model.

[0087] Among them, the different preset categories are multiple preset categories corresponding to the same image classification task processed by the target adversarial image generation model, the first image classification model, and the second image classification model.

[0088] The classification prediction information is used to characterize the image classification result. Specifically, it can be the image's original score (logit) for each category, the image's probability distribution for each category, or the image's classification result. The original score, probability distribution, and classification result are all generated and output by the corresponding models (including an adversarial image generation model, a first image classification model, and a second image classification model). In the classification task, the neural network model first generates the original score (logit) for each category to characterize the confidence level of each category. Then, it processes the original score (logit) for each category through an activation function to obtain the probability distribution for each category, and finally obtains the classification result based on the probability distribution for each category.

[0089] Preferably, the original score logit is used as the classification prediction information in this application.

[0090] Specifically, the server first inputs the sample image into the adversarial image generation model to be trained, and classifies the sample image through the adversarial image generation model to obtain the original score logit of the sample image corresponding to each preset category, and generates a second adversarial image of the sample image, the deviation between the second adversarial image and the sample image being less than a preset deviation; then, the server inputs the second adversarial image into the first image classification model, and classifies the second adversarial image through the first image classification model to obtain the original score logit of the second adversarial image corresponding to each preset category; next, the server determines the mean square error between the original score logit of the sample image and the original score logit of the second adversarial image, obtaining the first loss of the adversarial image generation model to be trained, and determines the cross-entropy loss between the original score logit of the sample image and the true category of the sample image, obtaining the second loss of the adversarial image generation model to be trained; finally, the server uses the first loss and the second loss of the adversarial image generation model to be trained as the total loss of the adversarial image generation model to be trained, and iteratively trains the adversarial image generation model to be trained until the total loss of the adversarial image generation model to be trained is less than the preset loss, thus obtaining the target adversarial image generation model.

[0091] In practical applications, the process by which the adversarial image generation model to be trained generates adversarial images (including the first adversarial image and the second adversarial image) of the sample images is shown in Equation 1:

[0092] (Formula 1)

[0093] in, For the adversarial image generation model to be trained; For sample images, For the true category of the sample image, To counteract images; The perturbation range hyperparameter is used to constrain the deviation between the sample image and the adversarial image, i.e. This is the preset deviation; This is the cross-entropy loss function.

[0094] The server calculates the total loss of the adversarial image generation model to be trained based on the following formula 2:

[0095] (Formula 2)

[0096] in, The total loss of the adversarial image generation model to be trained; This is the first image classification model; Let be the mean square error function. This is the cross-entropy loss function.

[0097] Further, in step S302, the sample image is input into the adversarial image generation model to be trained to obtain classification prediction information of the sample image corresponding to different preset categories. Before generating a second adversarial image with a deviation less than a preset deviation from the sample image, the server can pre-train the adversarial image generation model to be trained to have an initial parameter distribution that is compatible with the first image classification model for handling the same classification task. Specifically, the server pre-trains the adversarial image generation model to be trained based on the sample image. During pre-training, the server optimizes the parameters of the adversarial image generation model to be trained based on the following formula 3:

[0098] (Formula 3)

[0099] in, These are the parameters of the adversarial image generation model to be trained. These are the optimized parameters for the adversarial image generation model to be trained.

[0100] In this embodiment, a dual loss function design using mean squared error loss and cross-entropy loss ensures that the adversarial image generation model under training can generate adversarial examples with a certain degree of offensiveness while strictly constraining the perturbation amplitude of the adversarial examples to achieve stealth. It also learns the discriminative ability of the first image classification model, thus obtaining a target adversarial image generation model that can both generate adversarial examples and accurately classify sample images. Furthermore, using the original score logit instead of probability distributions or classification results for loss calculation allows for loss calculation based on richer information, enabling more refined iterative optimization of the model.

[0101] In an exemplary embodiment, step S106 above, which involves retraining the first image classification model based on the first adversarial image to obtain a second image classification model, specifically includes the following: adding a perturbation layer to the first image classification model to obtain a third image classification model; and iteratively training the third image classification model based on the first adversarial image to obtain the second image classification model.

[0102] The third image classification model includes at least the first image classification model and a perturbation layer.

[0103] The perturbation layer is used to inject perturbation information into the classification prediction information generated by the third image classification model. In specific applications, the perturbation information is random noise, preferably random noise from a Gaussian distribution. In specific applications, the perturbation layer is an RNDL (Random Noise Disturbs Layer). Figure 4 The diagram shown is a schematic representation of the target image classification model in this embodiment.

[0104] For the same image, the classification result output by the second image classification model based on the classification prediction information injected with perturbation information through the perturbation layer is the same as the classification result output by the first image classification model. That is, one of the purposes of training the third image classification model is to keep the TOP-1 prediction accuracy of the model after adding the perturbation layer unchanged.

[0105] Specifically, the server adds a perturbation layer to the first image classification model to inject perturbation information into the classification prediction information, thus obtaining a third image classification model. Then, the server trains the third image classification model with the first adversarial image as input information and the category of the first adversarial image as supervision information, so that it learns the adversarial features from the target adversarial image generation model, thus obtaining a third image classification model that can accurately classify the first adversarial image.

[0106] In practical applications, the perturbation layer generates perturbation information based on a Gaussian distribution as shown in Equation 4:

[0107] (Formula 4)

[0108] in, It follows a Gaussian distribution; For the parameters of the perturbation layer; in Formula 4, the parameters are... The output size of the normal distribution has been scaled.

[0109] See Figure 4 In the target image classification model, the final output vector is obtained by adding the vector output by the fully connected layer of the second image classification model to the noise output by the perturbation layer. The final output vector can be expressed as Equation 5:

[0110] (Formula 5)

[0111] in, This is the second image classification model; These are the parameters of the fully connected layer in the second image classification model; that is, the vector ultimately output by the second image classification model. Satisfies Gaussian distribution This prompts the probability vector output by the second image classification model to be modeled as a probability value, rather than a deterministic function.

[0112] In this embodiment, by adding a perturbation layer, the server can randomly inject noise into the output of the second image classification model, making it difficult for attackers to construct effective attack strategies by analyzing the output probability distribution of the second image classification model, thereby improving the information security of the second image classification model.

[0113] In an exemplary embodiment, the second image classification model is obtained by fixing the first parameters of the third image classification model and updating the second parameters of the third image classification model to obtain the updated second parameters of the third image classification model; fixing the updated second parameters of the third image classification model and updating the first parameters of the third image classification model to obtain the updated first parameters of the third image classification model; and obtaining the second image classification model based on the updated first parameters and the updated second parameters.

[0114] The first parameter is the parameter of the perturbation layer in the third image classification model, and the second parameter is the parameter of the first image classification model in the third image classification model.

[0115] Because of the added perturbation layer, the third image classification model has two sets of parameters: one set from the first image classification model and the other from the perturbation layer. Therefore, the server uses an alternating training method to train both sets of parameters separately to ensure stable convergence. Specifically, when training the parameters of the first image classification model, the parameters of the perturbation layer are fixed to guarantee the Top-1 accuracy of the first model. Conversely, when training the parameters of the perturbation layer, the parameters of the first model are fixed to ensure that the perturbation layer injects perturbation information into the first model and causes its output probability distribution to change randomly. Specifically, the server first fixes the parameters of the perturbation layer and updates the parameters of the first image classification model to obtain the updated second parameters. Then, the server fixes the updated second parameters and updates the parameters of the perturbation layer to obtain the updated first parameters. Finally, the server uses the third image classification model with the updated first and second parameters as the second image classification model.

[0116] In practical applications, the server updates the parameters of the first image classification model, i.e., the second parameter, according to Formula 6:

[0117] (Formula 6)

[0118] in, For the second parameter, This is the second parameter after the update.

[0119] In this embodiment, as shown in Equation 6, the parameters of the first image classification model are updated by gradient descent to learn the data distribution of the second adversarial image generated by the target adversarial image generation model, and to learn to correctly classify the image when perturbation information is injected into the perturbation layer.

[0120] The server updates the parameters of the perturbation layer, i.e., the first parameter, according to Formula 7: (Formula 7)

[0121] in, As the first parameter, The first parameter after the update; For the current training round, This is the maximum number of training rounds.

[0122] In this embodiment, Equation 6 forces the parameters of the perturbation layer to converge to zero, thereby eliminating the uncertainty caused by random noise perturbation to the first image classification model. To ensure that the perturbation layer can properly inject perturbation information, this embodiment adds parameters to the optimization function of the perturbation layer. Forced addition of regularization terms To prevent parameters The convergence of the parameters tends to zero; according to Equation 7, the parameters of the perturbation layer... By updating the probability vector through gradient descent, the probability vector output by the first image classification model is disturbed, thereby misleading attackers to obtain incorrect probability distribution information and effectively improving the model's security.

[0123] In this embodiment, by training the first image classification model and the perturbation layer, the robustness of the first image classification model against adversarial attacks is improved, and the ability of the perturbation layer to inject perturbation information is enhanced.

[0124] Based on the same inventive concept, this application also provides an image classification method. The specific limitations of one or more image classification method embodiments provided below can be found in the limitations of the image classification model training method above, and will not be repeated here.

[0125] In one exemplary embodiment, such as Figure 5As shown, an image classification model training method is provided. This embodiment illustrates the application of this method to a server. It is understood that this method can also be applied to a terminal, or to a system including both a server and a terminal, and is implemented through interaction between the server and the terminal. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. In this embodiment, the method includes the following steps, S102 to S108:

[0126] Step S502: Obtain the image to be classified and input the image to be classified into the pre-trained target image classification model.

[0127] The target image classification model includes at least an adversarial image generation model and an image classification model; the target image classification model is trained using the image classification model training method described above. In specific applications, the adversarial image generation model in the target image classification model is the target adversarial image generation model in the image classification model training method, and the image classification model in the target image classification model is the second image classification model.

[0128] Specifically, the server acquires the image to be classified and inputs it into a pre-trained target image classification model.

[0129] Step S504: Generate an adversarial image with a deviation less than a preset deviation from the image to be classified by using an adversarial image generation model.

[0130] Specifically, the image to be classified enters the adversarial image generation model, which generates an adversarial image of the image to be classified. The deviation between the adversarial image and the image to be classified is less than a preset deviation.

[0131] Step S506: Using an image classification model, determine the classification result of the adversarial image to obtain the classification result of the image to be classified.

[0132] Specifically, the adversarial image enters the image classification model, the image classification model performs image classification processing on the adversarial image, outputs the classification result of the adversarial image, and the server uses the classification result of the adversarial image as the classification result of the image to be classified.

[0133] In this embodiment, adversarial images are first generated by an adversarial image generation model, and then the adversarial images are classified by an image classification model. Since the image classification model has fully learned the adversarial features during the training process, it can resist the interference of adversarial features and output accurate classification results.

[0134] In one exemplary embodiment, the target image classification model further includes a perturbation layer for injecting perturbation information into the classification prediction information generated by the image classification model to randomly disrupt the entire probability distribution and ensure the information security of the target image classification model.

[0135] To more clearly illustrate the image classification model training and image classification model method provided in the embodiments of this application, a specific embodiment is used below to specifically describe the image classification model training and image classification model method provided in the embodiments of this application. However, it should be understood that the embodiments of this application are not limited thereto. Figure 6 As shown, in one exemplary embodiment, this application also provides an adversarial defense training method and an image classification method for an image classification neural network, specifically including the following steps:

[0136] Step S602: Based on the first image classification model and the sample images used to train the first image classification model, train the adversarial image generation model to be trained, and obtain the target adversarial image generation model.

[0137] Step S604: Based on the target adversarial image generation model, generate a first adversarial image whose deviation from the sample image is less than a preset deviation.

[0138] The image classification model is implemented using a DNN model, and the adversarial image generation model is implemented using an AET model.

[0139] like Figure 7 As another schematic diagram of this embodiment, the above steps S602 to S604 can also be implemented by step 1.

[0140] Step 1: Distillation Training of the Adversarial Example Transformer (AET Model). First, the AET model is pre-trained using the training dataset of a deep neural network model (DNN model), giving it an initial parameter distribution similar to that of the DNN model when handling the same classification task. Then, the decision boundary information of the DNN—the raw logits—is used to guide the training of the AET model, allowing it to inherit the knowledge of the DNN model and generate more complex adversarial images. Subsequent training follows with a deep neural network-random noise perturbation model (DNN-RNDL model).

[0141] Step S606: Add a perturbation layer to the first image classification model to obtain the third image classification model.

[0142] Step S608: Based on the first adversarial image, iteratively train the third image classification model: fix the first parameter of the perturbation layer in the third image classification model, update the second parameter of the first image classification model in the third image classification model to obtain the updated second parameter; fix the updated second parameter, update the first parameter to obtain the updated first parameter; use the third image classification model with the updated first parameter and the updated second parameter as the second image classification model.

[0143] The perturbation layer is implemented using the RNDL module (random noise perturbation layer).

[0144] See Figure 7 The steps S606 to S608 described above can also be achieved through step 2.

[0145] Step 2: DNN-RNDL Model Training. An RNDL module is added to the DNN model. The RNDL module is designed to randomly perturb the last fully connected layer of the DNN model while maintaining its Top-1 accuracy. This improves the model's safety by disrupting the output probability distribution of the DNN model. In this step, alternating training is used to train the parameters of the DNN model and the RNDL module separately to ensure stable convergence of both sets of parameters. Specifically, when training the DNN parameters, the RNDL parameters are fixed to guarantee the Top-1 accuracy of the DNN model; when training the RNDL parameters, the DNN parameters are fixed to ensure that the RNDL module injects random noise into the DNN model and causes its output probability distribution to change randomly.

[0146] Step S610: Combine the target adversarial image generation model and the second image classification model to obtain the target image classification model.

[0147] See Figure 7 The above step S610 can also be achieved through the following step 3.

[0148] Step 3: Integrate the defense model. Integrate the trained AET model and DNN-RNDL model to form the final defense model.

[0149] Step S612: Obtain the image to be classified and input the image to be classified into the pre-trained target image classification model.

[0150] Step S614: Generate an adversarial image with a deviation less than a preset deviation from the image to be classified by using the adversarial image generation model in the target image classification model. Then, determine the classification result of the adversarial image by using the second image classification model in the target image classification model, and use it as the classification result of the image to be classified.

[0151] In this embodiment, firstly, an AET (Advanced Execution Technology) and DNN (Digital Neural Network) approach is combined, and an RNDL (Recurrent Neural Network DL) module is integrated to form a multi-layered defense system. AET inherits knowledge from the DNN through knowledge distillation to generate complex adversarial images for subsequent training, thereby improving the model's robustness. RNDL, on the other hand, injects Gaussian-distributed random noise to perturb the output probability distribution of the DNN, enhancing the model's defense against attackers. Secondly, by alternately training the parameters of the DNN and RNDL, it is ensured that both converge to their respective target states, thus forming a highly robust integrated defense model under adversarial attacks. Based on the above process, this embodiment ultimately constructs a highly robust and secure neural network defense system.

[0152] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0153] Based on the same inventive concept, this application also provides an image classification model training apparatus for implementing the image classification model training method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the image classification model training apparatus provided below can be found in the limitations of the image classification model training method described above, and will not be repeated here.

[0154] In one exemplary embodiment, such as Figure 8 As shown, an image classification model training device is provided, comprising: a first training module 802, an image generation module 804, a second training module 806, and a model combination module 808, wherein:

[0155] The first training module 802 is used to train the adversarial image generation model to be trained based on the first image classification model and the sample images used to train the first image classification model, so as to obtain the target adversarial image generation model.

[0156] The image generation module 804 is used to generate a first adversarial image with a deviation less than a preset deviation from the sample image based on the target adversarial image generation model.

[0157] The second training module 806 is used to retrain the first image classification model based on the first adversarial image to obtain the second image classification model.

[0158] The model combination module 808 is used to combine the target adversarial image generation model and the second image classification model to obtain the target image classification model.

[0159] In an exemplary embodiment, the first training module 802 is further configured to: input sample images into the adversarial image generation model to be trained to obtain classification prediction information of the sample images corresponding to different preset categories; and generate a second adversarial image whose deviation from the sample images is less than a preset deviation; input the second adversarial image into the first image classification model to obtain classification prediction information of the second adversarial image corresponding to different preset categories; determine a first loss of the adversarial image generation model to be trained based on the classification prediction information of the sample images and the classification prediction information of the second adversarial image; determine a second loss of the adversarial image generation model to be trained based on the classification prediction information of the sample images and the true categories; and iteratively train the adversarial image generation model to be trained based on the first loss and the second loss of the adversarial image generation model to obtain a target adversarial image generation model.

[0160] In an exemplary embodiment, the second training module 806 is further configured to add a perturbation layer to the first image classification model to obtain a third image classification model; the perturbation layer is configured to inject perturbation information into the classification prediction information generated by the third image classification model; the third image classification model is iteratively trained according to the first adversarial image to obtain a second image classification model; for the same image, the classification result output by the second image classification model is the same as the classification result output by the first image classification model.

[0161] In an exemplary embodiment, the second training module 806 is further configured to fix the first parameters of the third image classification model, update the second parameters of the third image classification model, and obtain the updated second parameters of the third image classification model; the first parameters are the parameters of the perturbation layer in the third image classification model, and the second parameters are the parameters of the first image classification model in the third image classification model; fix the updated second parameters of the third image classification model, update the first parameters of the third image classification model, and obtain the updated first parameters of the third image classification model; and obtain the second image classification model based on the updated first parameters and the updated second parameters of the third image classification model.

[0162] Based on the same inventive concept, this application also provides an image classification apparatus for implementing the image classification method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image classification apparatus embodiments provided below can be found in the limitations of the image classification method described above, and will not be repeated here.

[0163] In one exemplary embodiment, such as Figure 9 As shown, an image classification device is provided, including: an image acquisition module 902, an image generation module 904, and an image classification module 906, wherein:

[0164] The image acquisition module 902 is used to acquire the image to be classified and input the image to be classified into a pre-trained target image classification model; the target image classification model includes at least an adversarial image generation model and an image classification model; the target image classification model is trained according to an image classification model training method.

[0165] Image generation module 904 is used to generate adversarial images with a deviation from the image to be classified that is less than a preset deviation by using an adversarial image generation model.

[0166] The image classification module 906 is used to determine the classification result of the adversarial image through the image classification model, and obtain the classification result of the image to be classified.

[0167] The modules in the aforementioned image classification model training device and image classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0168] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores sample image data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements an image classification model training method and an image classification method.

[0169] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0170] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0171] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0172] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0173] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0175] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for training an image classification model, characterized in that, The method includes: Based on the first image classification model and the sample images used to train the first image classification model, the adversarial image generation model to be trained is trained to obtain the target adversarial image generation model. Based on the target adversarial image generation model, a first adversarial image with a deviation less than a preset deviation from the sample image is generated; A perturbation layer is added to the first image classification model to obtain a third image classification model; the perturbation layer is used to inject perturbation information into the classification prediction information generated by the third image classification model. Based on the first adversarial image, the third image classification model is iteratively trained to obtain a second image classification model; for the same image, the classification result output by the second image classification model based on the classification prediction information injected with the perturbation information is the same as the classification result output by the first image classification model; further comprising: fixing the first parameter of the third image classification model, updating the second parameter of the third image classification model to obtain the updated second parameter of the third image classification model; the first parameter is the parameter of the perturbation layer in the third image classification model, and the second parameter is the parameter of the first image classification model in the third image classification model; fixing the updated second parameter of the third image classification model, updating the first parameter of the third image classification model to obtain the updated first parameter of the third image classification model; and obtaining the second image classification model based on the third image classification model under the updated first parameter and the updated second parameter. The target adversarial image generation model and the second image classification model are combined to obtain the target image classification model.

2. The method according to claim 1, characterized in that, The step of training an adversarial image generation model to obtain a target adversarial image generation model based on a first image classification model and sample images used to train the first image classification model includes: The sample image is input into the adversarial image generation model to be trained to obtain classification prediction information of the sample image corresponding to different preset categories, and to generate a second adversarial image with a deviation from the sample image that is smaller than the preset deviation. The second adversarial image is input into the first image classification model to obtain classification prediction information of the second adversarial image corresponding to the different preset categories; Based on the classification prediction information of the sample image and the classification prediction information of the second adversarial image, the first loss of the adversarial image generation model to be trained is determined, and based on the classification prediction information of the sample image and the true category, the second loss of the adversarial image generation model to be trained is determined. Based on the first loss and the second loss of the adversarial image generation model to be trained, the adversarial image generation model to be trained is iteratively trained to obtain the target adversarial image generation model.

3. The method according to claim 1, characterized in that, The first image classification model is trained based on a sample dataset; the sample dataset includes multiple images, each labeled with a corresponding true category.

4. An image classification method, characterized in that, The method includes: A target image to be classified is obtained, and the target image to be classified is input into a pre-trained target image classification model; the target image classification model includes at least an adversarial image generation model and an image classification model; the target image classification model is trained according to the image classification model training method according to any one of claims 1 to 3. The adversarial image generation model generates an adversarial image whose deviation from the image to be classified is less than a preset deviation. The classification result of the adversarial image is determined by the image classification model, and the classification result of the image to be classified is obtained.

5. An image classification model training device, characterized in that, The device includes: The first training module is used to train the adversarial image generation model to be trained based on the first image classification model and the sample images used to train the first image classification model, so as to obtain the target adversarial image generation model. The image generation module is used to generate a first adversarial image with a deviation less than a preset deviation from the sample image, based on the target adversarial image generation model. The second training module is used to add a perturbation layer to the first image classification model to obtain a third image classification model; the perturbation layer is used to inject perturbation information into the classification prediction information generated by the third image classification model; the third image classification model is iteratively trained according to the first adversarial image to obtain a second image classification model; for the same image, the classification result output by the second image classification model based on the classification prediction information with injected perturbation information is the same as the classification result output by the first image classification model; the second training module is also used to fix the first parameter of the third image classification model and update the second parameter of the third image classification model to obtain the updated second parameter of the third image classification model; the first parameter is the parameter of the perturbation layer in the third image classification model, and the second parameter is the parameter of the first image classification model in the third image classification model; the updated second parameter of the third image classification model is fixed, and the first parameter of the third image classification model is updated to obtain the updated first parameter of the third image classification model; the second image classification model is obtained according to the third image classification model under the updated first parameter and the updated second parameter; The model combination module is used to combine the target adversarial image generation model and the second image classification model to obtain the target image classification model.

6. The apparatus according to claim 5, characterized in that, The first training module is further configured to input the sample image into the adversarial image generation model to be trained, obtain classification prediction information of the sample image corresponding to different preset categories, and generate a second adversarial image with a deviation from the sample image that is less than the preset deviation; The second adversarial image is input into the first image classification model to obtain classification prediction information of the second adversarial image corresponding to the different preset categories; Based on the classification prediction information of the sample image and the classification prediction information of the second adversarial image, the first loss of the adversarial image generation model to be trained is determined, and based on the classification prediction information of the sample image and the true category, the second loss of the adversarial image generation model to be trained is determined. Based on the first loss and the second loss of the adversarial image generation model to be trained, the adversarial image generation model to be trained is iteratively trained to obtain the target adversarial image generation model.

7. An image classification device, characterized in that, The device includes: An image acquisition module is used to acquire an image to be classified and input the image to be classified into a pre-trained target image classification model; the target image classification model includes at least an adversarial image generation model and an image classification model; the target image classification model is trained according to the image classification model training method according to any one of claims 1 to 3. The image generation module is used to generate an adversarial image with a deviation of less than a preset deviation from the image to be classified by the adversarial image generation model. The image classification module is used to determine the classification result of the adversarial image through the image classification model, and obtain the classification result of the image to be classified.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the image classification model training method according to any one of claims 1 to 3 or the image classification method according to claim 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the image classification model training method according to any one of claims 1 to 3 or the image classification method according to claim 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the image classification model training method according to any one of claims 1 to 3 or the image classification method according to claim 4.

Images