Image classification model training method and device, image classification method and device and computer equipment

Abstract

The invention relates to an image classification model training method and device, an image classification method and device and computer equipment, and relates to the technical field of artificial intelligence. The method comprises the steps of training a to-be-trained confrontation image generation model according to a first image classification model and a sample image used for training the first image classification model, and obtaining a target confrontation image generation model; based on a target confrontation image generation model, generating a first confrontation image whose deviation with the sample image is smaller than a preset deviation; according to the first confrontation image, training the first image classification model again to obtain a second image classification model; and combining the target confrontation image generation model and the second image classification model to obtain a target image classification model. By adopting the method, the robustness of the image classification model can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an image classification model training method, an image classification method, an image classification device, a computer device, a computer readable storage medium and a computer program product. BACKGROUND

[0002] With the progress of artificial intelligence, image classification based on artificial intelligence has been widely applied in various fields. In the related art, an image classification model based on a neural network, such as a DNN (Deep Neural Network) model, is usually used for image classification. However, research has found that even a slight image disturbance that is difficult for the human eye to detect can have a serious impact on the image classification model, that is, the robustness of the image classification model in the related art is weak. SUMMARY

[0003] Therefore, it is necessary to provide an image classification model training method, an image classification method, an image classification device, a computer device, a computer readable storage medium and a computer program product capable of improving the robustness of an image classification model to solve the technical problem of weak robustness of the image classification model.

[0004] In a first aspect, the present application provides an image classification model training method, comprising:

[0005] training a to-be-trained adversarial image generation model according to a first image classification model and a sample image used for training the first image classification model, to obtain a target adversarial image generation model;

[0006] generating a first adversarial image with a deviation from the sample image less than a preset deviation based on the target adversarial image generation model;

[0007] retraining the first image classification model according to the first adversarial image to obtain a second image classification model;

[0008] combining the target adversarial image generation model and the second image classification model to obtain a target image classification model.

[0009] In one embodiment, the training of the to-be-trained adversarial image generation model according to the first image classification model and the sample image used for training the first image classification model to obtain the target adversarial image generation model comprises:

[0010] inputting the sample image into the to-be-trained adversarial image generation model to obtain classification prediction information corresponding to different preset categories of the sample image, and generating a second adversarial image with a deviation from the sample image less than the preset deviation;

[0011] inputting the second adversarial image into the first image classification model to obtain classification prediction information corresponding to the different preset category of the second adversarial image;

[0012] According to the classification prediction information of the sample image and the classification prediction information of the second adversarial image, a first loss of the adversarial image generation model to be trained is determined, and according to the classification prediction information of the sample image and the real category, a second loss of the adversarial image generation model to be trained is determined.

[0013] According to the first loss and the second loss of the adversarial image generation model to be trained, the adversarial image generation model to be trained is iteratively trained to obtain the target adversarial image generation model.

[0014] In one of the embodiments, the retraining of the first image classification model according to the first adversarial image to obtain a second image classification model comprises:

[0015] A perturbation layer is added to the first image classification model to obtain a third image classification model; the perturbation layer is used to inject perturbation information into the classification prediction information generated by the third image classification model;

[0016] The third image classification model is iteratively trained according to the first adversarial image to obtain the second image classification model; the classification result output by the second image classification model based on the classification prediction information injected with the perturbation information is the same as the classification result output by the first image classification model.

[0017] In one of the embodiments, the second image classification model is obtained in the following manner:

[0018] The first parameters of the third image classification model are fixed, and the second parameters of the third image classification model are updated to obtain updated second parameters of the third image classification model; the first parameters are the parameters of the perturbation layer in the third image classification model, and the second parameters are the parameters of the first image classification model in the third image classification model;

[0019] The updated second parameters of the third image classification model are fixed, and the first parameters of the third image classification model are updated to obtain updated first parameters of the third image classification model;

[0020] The second image classification model is obtained according to the third image classification model under the updated first parameters and the updated second parameters.

[0021] In a second aspect, the present application further provides an image classification model training method, comprising:

[0022] obtaining an image to be classified, inputting the image to be classified into a target image classification model trained in advance; the target image classification model at least comprises an adversarial image generation model and an image classification model; the target image classification model is obtained according to an image classification model training method;

[0023] generating, by the adversarial image generation model, an adversarial image with a deviation less than a preset deviation from the image to be classified;

[0024] determining, by the image classification model, a classification result of the adversarial image, and obtaining a classification result of the image to be classified.

[0025] In a third aspect, the present application further provides an image classification model training device, comprising:

[0026] a first training module configured to train an adversarial image generation model to be trained according to a first image classification model and sample images used to obtain the first image classification model, and obtain a target adversarial image generation model;

[0027] an image generation module configured to generate, based on the target adversarial image generation model, a first adversarial image with a deviation less than a preset deviation from the sample image;

[0028] a second training module configured to retrain the first image classification model according to the first adversarial image, and obtain a second image classification model;

[0029] a model combination module configured to combine the target adversarial image generation model and the second image classification model, and obtain a target image classification model.

[0030] In a fourth aspect, the present application further provides an image classification model training device, comprising:

[0031] an image acquisition module configured to obtain an image to be classified, and input the image to be classified into a target image classification model trained in advance; the target image classification model at least comprises an adversarial image generation model and an image classification model; the target image classification model is obtained according to the image classification model training method;

[0032] an image generation module configured to generate, by the adversarial image generation model, an adversarial image with a deviation less than a preset deviation from the image to be classified;

[0033] an image classification module configured to determine, by the image classification model, a classification result of the adversarial image, and obtain a classification result of the image to be classified.

[0034] In a fifth aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0035] training the adversarial image generation model to be trained according to the first image classification model and sample images used to obtain the first image classification model, to obtain a target adversarial image generation model;

[0036] generating, based on the target adversarial image generation model, a first adversarial image with a deviation from the sample image less than a preset deviation;

[0037] retraining the first image classification model according to the first adversarial image, to obtain a second image classification model;

[0038] combining the target adversarial image generation model and the second image classification model, to obtain a target image classification model.

[0039] In a sixth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0040] training the adversarial image generation model to be trained according to the first image classification model and sample images used to obtain the first image classification model, to obtain a target adversarial image generation model;

[0041] generating, based on the target adversarial image generation model, a first adversarial image with a deviation from the sample image less than a preset deviation;

[0042] retraining the first image classification model according to the first adversarial image, to obtain a second image classification model;

[0043] combining the target adversarial image generation model and the second image classification model, to obtain a target image classification model.

[0044] In a seventh aspect, the present application provides a computer program product comprising a computer program, and the computer program is executed by a processor to implement the following steps:

[0045] training the adversarial image generation model to be trained according to the first image classification model and sample images used to obtain the first image classification model, to obtain a target adversarial image generation model;

[0046] generating, based on the target adversarial image generation model, a first adversarial image with a deviation from the sample image less than a preset deviation;

[0047] retrain the first image classification model according to the first adversarial image to obtain a second image classification model;

[0048] combine the target adversarial image generation model and the second image classification model to obtain a target image classification model.

[0049] The image classification model training method, the image classification method, the device, the computer device, the computer readable storage medium and the computer program product can train a target adversarial image generation model through a first image classification model and a sample image, so that the target adversarial image generation model can inherit the knowledge of the first image classification model, thereby generating a first adversarial image with a deviation from the sample image less than a preset deviation; retraining the first image classification model through the first adversarial image can enable the first image classification model to learn adversarial features from the target adversarial image generation model in the training process, thereby obtaining a second image classification model with stronger robustness; combining the target adversarial image generation model and the second image classification model can obtain a target image classification model with stronger robustness; therefore, the image classification model training method based on the above process can improve the robustness of the image classification model. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other related drawings without creative labor on the basis of these drawings.

[0051] Figure 1 a flowchart of an image classification model training method in an embodiment;

[0052] Figure 2 a structural diagram of a target image classification model in an embodiment;

[0053] Figure 3 a flowchart of a step of training a target adversarial image generation model according to a first image classification model and a sample image used to train the first image classification model in an embodiment;

[0054] Figure 4 a structural diagram of a target image classification model in another embodiment;

[0055] Figure 5 a flowchart of an image classification method in an embodiment;

[0056] Figure 6A flowchart of an adversarial defense training method and an image classification method for an image classification neural network in one embodiment;

[0057] Figure 7 A flowchart of an adversarial defense training method and an image classification method for an image classification neural network in another embodiment;

[0058] Figure 8 A structural block diagram of an image classification model training apparatus in one embodiment;

[0059] Figure 9 A structural block diagram of an image classification apparatus in one embodiment;

[0060] Figure 10 An internal structural diagram of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0062] It should be noted that the terms "first", "second", etc. used in the present application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "a plurality of" used in the present application refers to two or more. The term "and / or" used in the present application refers to one of the options or any combination of multiple options.

[0063] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0064] In one exemplary embodiment, as Figure 1As shown, an image classification model training method is provided. In this embodiment, the method is applied to a server for illustration. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a server and a terminal, and can be implemented through the interaction of the server and the terminal. The server can be a standalone physical server, a server cluster or a distributed system formed by multiple physical servers, or a cloud server providing cloud computing services; the terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, unmanned aerial vehicles, low-altitude aircraft, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. In this embodiment, the method includes the following steps S102 to S108:

[0065] In step S102, an adversarial image generation model to be trained is trained according to a first image classification model and sample images used for training the first image classification model, to obtain a target adversarial image generation model.

[0066] The first image classification model is a trained image classification model. In specific applications, the first image classification model is a neural network model, and preferably, the first image classification model is a DNN (Deep Neural Network) model.

[0067] The first image classification model is trained based on a sample data set; the sample data set includes a plurality of images, and each image is labeled with a corresponding true class.

[0068] The sample images are any number of images in the sample data set.

[0069] The target adversarial image generation model is used to process the same image classification task as the first image classification model, and generate adversarial images. In specific applications, the parameter amount of the adversarial image generation model to be trained is less than the parameter amount of the first image classification model. In specific applications, the adversarial image generation model to be trained is an AET (Adversarial Example Transformer) model.

[0070] Specifically, the server performs distillation training on the to-be-trained adversarial image generation model according to the first image classification model and the sample image used for training the first image classification model, so that the to-be-trained adversarial image generation model inherits the knowledge of the first image classification model, and a target adversarial image generation model is obtained.

[0071] In step S104, a first adversarial image with a deviation from the sample image less than a preset deviation is generated based on the target adversarial image generation model.

[0072] Specifically, the server inputs the sample image into the target adversarial image generation model, and generates a first adversarial image of the sample image through the target adversarial image generation model; the deviation between the first adversarial image and the sample image is less than the preset deviation.

[0073] In step S106, the first image classification model is retrained according to the first adversarial image, and a second image classification model is obtained.

[0074] The second image classification model is used to process the same image classification task as the first image classification model.

[0075] Specifically, the server re-trains the first image classification model with the first adversarial image as input information and the class of the first adversarial image as supervision information, so that the first image classification model learns the adversarial features from the target adversarial image generation model, and a second image classification model capable of accurately classifying the first adversarial image is obtained.

[0076] The class of the first adversarial image can be a classification result of the first adversarial image under the target adversarial image generation model, or a real class labeled for the first adversarial image by a human.

[0077] In step S108, the target adversarial image generation model and the second image classification model are combined to obtain a target image classification model.

[0078] The target image classification model at least includes the target adversarial image generation model and the second image classification model.

[0079] Specifically, the server connects the target adversarial image generation model in front of the second image classification model to obtain the target image classification model. As shown in Figure 2 FIG. 1 shows a structure diagram of the target image classification model in the embodiment.

[0080] In a specific application, the server performs image classification through the target image classification model.

[0081] In the image classification model training method, the server trains the target adversarial image generation model through the first image classification model and the sample image, so that the target adversarial image generation model can inherit the knowledge of the first image classification model, thereby generating the first adversarial image with a deviation from the sample image less than a preset deviation; the server re-trains the first image classification model through the first adversarial image, so that the first image classification model learns the adversarial features from the target adversarial image generation model in the training process, thereby obtaining a second image classification model with stronger robustness; and the server combines the target adversarial image generation model and the second image classification model, thereby obtaining a target image classification model with stronger robustness. Therefore, based on the above process, the image classification model training method can improve the robustness of the image classification model.

[0082] In one exemplary embodiment, as shown in FIG. 1, Figure 3 The step S102 includes the following steps:

[0083] The step S302 includes the following steps:

[0084] The step S304 includes the following steps:

[0085] The step S306 includes the following steps:

[0086] The step S308 includes the following steps:

[0087] The different preset categories are a plurality of preset categories corresponding to the same image classification task processed by the target adversarial image generation model, the first image classification model, and the second image classification model.

[0088] The classification prediction information is used to represent the classification result of the image, which can be the original score logit of the image under each category, the distribution probability of the image under each category, or the classification result of the image. The original score, the distribution probability, and the classification result are all generated and output by the corresponding model (including the target adversarial image generation model, the first image classification model, and the second image classification model). In the classification task, the neural network model first generates the original score logit corresponding to each category, which is used to represent the confidence of each category. Then, the original score logit corresponding to each category is processed by the activation function to obtain the distribution probability corresponding to each category, and the final classification result is obtained based on the distribution probability corresponding to each category.

[0089] Preferably, the original score logit is used as the classification prediction information in the present application.

[0090] Specifically, the server first inputs the sample image into the adversarial image generation model to be trained, and processes the sample image through the adversarial image generation model to be trained to obtain the original score logit of the sample image corresponding to each preset category, and generates a second adversarial image of the sample image, the deviation between the second adversarial image and the sample image being less than a preset deviation. Then, the server inputs the second adversarial image into the first image classification model, and processes the second adversarial image through the first image classification model to obtain the original score logit of the second adversarial image corresponding to each preset category. Next, the server determines the mean square error between the original score logit of the sample image and the original score logit of the second adversarial image to obtain the first loss of the adversarial image generation model to be trained, and determines the cross-entropy loss between the original score logit of the sample image and the real category of the sample image according to the original score logit of the sample image and the real category of the sample image to obtain the second loss of the adversarial image generation model to be trained. Finally, the server takes the first loss and the second loss of the adversarial image generation model to be trained as the total loss of the adversarial image generation model to be trained, and iteratively trains the adversarial image generation model to be trained until the total loss of the adversarial image generation model to be trained is less than a preset loss, thereby obtaining the target adversarial image generation model.

[0091] In a specific application, the process of generating the adversarial image (including the first adversarial image and the second adversarial image) of the sample image by the adversarial image generation model to be trained is shown in formula 1:

[0092] (Formula 1)

[0093] wherein, is the adversarial image generation model to be trained; is the sample image, a true class of the sample image, an adversarial image; a perturbation range hyperparameter used to constrain the deviation between the sample image and the adversarial image, i.e. a preset deviation; a cross-entropy loss function.

[0094] The server calculates the total loss of the adversarial image generation model to be trained based on the following formula 2:

[0095] (Formula 2)

[0096] wherein, the total loss of the adversarial image generation model to be trained; the first image classification model; a mean square error function, a cross-entropy loss function.

[0097] Further, in step S302, the sample image is input into the adversarial image generation model to be trained to obtain classification prediction information corresponding to different preset classes of the sample image, and before generating a second adversarial image with a deviation from the sample image less than a preset deviation, the server can pre-train the adversarial image generation model to be trained to have an initial parameter distribution for processing the same classification task as the first image classification model; specifically, the server pre-trains the adversarial image generation model to be trained according to the sample image, and when pre-training, the server optimizes the parameters of the adversarial image generation model to be trained based on the following formula 3:

[0098] (Formula 3)

[0099] wherein, the parameters of the adversarial image generation model to be trained, the optimized parameters of the adversarial image generation model to be trained.

[0100] In this embodiment, through the double-loss function design of the mean square error loss and the cross-entropy loss, it is ensured that the adversarial image generation model to be trained can generate adversarial samples with certain attack while strictly constraining the perturbation amplitude of the adversarial samples, realize the concealment, and learn the discrimination ability of the first image classification model, so as to obtain a target adversarial image generation model which can both generate adversarial samples and accurately classify sample images. In addition, using the original score logit instead of the probability distribution or the classification result for loss calculation can perform loss calculation based on more abundant information and perform more fine iterative optimization on the model.

[0101] In an exemplary embodiment, step S106 above, which involves retraining the first image classification model based on the first adversarial image to obtain a second image classification model, specifically includes the following: adding a perturbation layer to the first image classification model to obtain a third image classification model; and iteratively training the third image classification model based on the first adversarial image to obtain the second image classification model.

[0102] The third image classification model includes at least the first image classification model and a perturbation layer.

[0103] The perturbation layer is used to inject perturbation information into the classification prediction information generated by the third image classification model. In specific applications, the perturbation information is random noise, preferably random noise from a Gaussian distribution. In specific applications, the perturbation layer is an RNDL (Random Noise Disturbs Layer). Figure 4 The diagram shown is a schematic representation of the target image classification model in this embodiment.

[0104] For the same image, the classification result output by the second image classification model based on the classification prediction information injected with perturbation information through the perturbation layer is the same as the classification result output by the first image classification model. That is, one of the purposes of training the third image classification model is to keep the TOP-1 prediction accuracy of the model after adding the perturbation layer unchanged.

[0105] Specifically, the server adds a perturbation layer to the first image classification model to inject perturbation information into the classification prediction information, thus obtaining a third image classification model. Then, the server trains the third image classification model with the first adversarial image as input information and the category of the first adversarial image as supervision information, so that it learns the adversarial features from the target adversarial image generation model, thus obtaining a third image classification model that can accurately classify the first adversarial image.

[0106] In practical applications, the perturbation layer generates perturbation information based on a Gaussian distribution as shown in Equation 4:

[0107] (Formula 4)

[0108] in, It follows a Gaussian distribution; For the parameters of the perturbation layer; in Formula 4, the parameters are... The output size of the normal distribution has been scaled.

[0109] See Figure 4 In the target image classification model, the final output vector is obtained by adding the vector output by the fully connected layer of the second image classification model to the noise output by the perturbation layer. The final output vector can be expressed as Equation 5:

[0110] (Formula 5)

[0111] wherein, is a second image classification model; is a parameter of a full connection layer of the second image classification model; that is, a vector output by the second image classification model satisfies a Gaussian distribution This causes the probability vector output by the second image classification model to be modeled as a probability value rather than a deterministic function.

[0112] In this embodiment, the server can randomly inject noise in the output of the second image classification model by adding the disturbance layer, so that it is difficult for an attacker to construct an effective attack strategy by analyzing the output probability distribution of the second image classification model, thereby improving the information security of the second image classification model.

[0113] In an exemplary embodiment, the second image classification model is obtained by: fixing the first parameter of the third image classification model, updating the second parameter of the third image classification model to obtain the updated second parameter of the third image classification model; fixing the updated second parameter of the third image classification model, updating the first parameter of the third image classification model to obtain the updated first parameter of the third image classification model; and obtaining the second image classification model according to the third image classification model under the updated first parameter and the updated second parameter.

[0114] wherein the first parameter is a parameter of the disturbance layer in the third image classification model, and the second parameter is a parameter of the first image classification model in the third image classification model.

[0115] Since the disturbance layer is added, there are two groups of parameters in the third image classification model, one group being the parameters of the first image classification model and the other group being the parameters of the disturbance layer. Therefore, the server uses an alternating training manner to train the two groups of parameters respectively to ensure that the two groups of parameters converge stably, that is, when the parameters of the first image classification model are trained, the parameters of the disturbance layer are fixed to ensure the Top-1 accuracy of the first image classification model, and when the parameters of the disturbance layer are trained, the parameters of the first image classification model are fixed to ensure that the disturbance layer injects disturbance information into the first image classification model and causes the probability distribution output by the first image classification model to change randomly. Specifically, the server first fixes the parameters of the disturbance layer, updates the parameters of the first image classification model to obtain the updated second parameter; then, the server fixes the updated second parameter, updates the parameters of the disturbance layer to obtain the updated first parameter; and finally, the server takes the third image classification model under the updated first parameter and the updated second parameter as the second image classification model.

[0116] In a specific application, the server updates the parameters of the first image classification model, i.e., the second parameters, according to Formula 6:

[0117] (Formula 6)

[0118] wherein, is the second parameter, is the updated second parameter.

[0119] In this embodiment, as shown in Formula 6, the parameters of the first image classification model are updated by gradient descent to learn the data distribution of the second adversarial image generated from the target adversarial image generation model and to learn to correctly classify images in the case of injecting perturbation information in the perturbation layer.

[0120] The server updates the parameters of the perturbation layer, i.e., the first parameters, according to Formula 7: (Formula 7)

[0121] wherein, is the first parameter, is the updated first parameter; is the current training round, is the maximum training round.

[0122] In this embodiment, Formula 6 forces the parameters of the perturbation layer to converge to zero, thereby eliminating the uncertainty of the first image classification model caused by random noise perturbation. In order to enable the perturbation layer to normally implement the injection of perturbation information, this embodiment adds a regularization term that forces the parameter to increase to the optimization function of the perturbation layer to prevent the parameter from converging to zero; through Formula 7, the parameters of the perturbation layer are updated by gradient descent to disturb the probability vector output by the first image classification model, to mislead the attacker to obtain incorrect probability distribution information and effectively improve the security of the model.

[0123] In this embodiment, through the training of the first image classification model and the perturbation layer, the robustness of the first image classification model when facing adversarial attacks is improved, and the ability of the perturbation layer to inject perturbation information is improved.

[0124] Based on the same inventive concept, the embodiments of the present application also provide an image classification method. The specific limitations in one or more image classification method embodiments provided below can be referred to the limitations of the image classification model training method provided above, which will not be repeated here.

[0125] In an exemplary embodiment, as Figure 5As shown, an image classification model training method is provided, and the embodiment is exemplified by applying the method to a server. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a server and a terminal, and is implemented through the interaction of the server and the terminal. The server can be a stand-alone physical server, a server cluster or a distributed system formed by multiple physical servers, or a cloud server providing cloud computing services; the terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, unmanned aerial vehicles, low-altitude aircraft, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. In the embodiment, the method includes the following steps S502 to S108:

[0126] In step S502, a to-be-classified image is obtained, and the to-be-classified image is input into a target image classification model pre-trained.

[0127] The target image classification model at least includes an adversarial image generation model and an image classification model, and is trained by the image classification model training method. In a specific application, the adversarial image generation model in the target image classification model is the target adversarial image generation model in the image classification model training method, and the image classification model in the target image classification model is the second image classification model.

[0128] Specifically, the server obtains a to-be-classified image, and inputs the to-be-classified image into a target image classification model pre-trained.

[0129] In step S504, an adversarial image with a deviation from the to-be-classified image less than a preset deviation is generated by the adversarial image generation model.

[0130] Specifically, the to-be-classified image enters the adversarial image generation model, the adversarial image generation model generates an adversarial image of the to-be-classified image, and the deviation between the adversarial image and the to-be-classified image is less than the preset deviation.

[0131] In step S506, a classification result of the adversarial image is determined by the image classification model, and a classification result of the to-be-classified image is obtained.

[0132] Specifically, the adversarial image enters the image classification model, the image classification model performs image classification processing on the adversarial image, outputs a classification result of the adversarial image, and the server takes the classification result of the adversarial image as the classification result of the to-be-classified image.

[0133] In this embodiment, first, the adversarial image is generated by the adversarial image generation model, and then the adversarial image is classified by the image classification model. Since the image classification model has learned the adversarial features sufficiently in the training process, the image classification model can resist the interference of the adversarial features and output accurate classification results.

[0134] In one exemplary embodiment, the target image classification model further comprises a perturbation layer for injecting perturbation information into the classification prediction information generated by the image classification model to randomly disturb the entire probability distribution and ensure the information security of the target image classification model.

[0135] In order to more clearly illustrate the image classification model training and image classification model method provided by the embodiments of the present application, the image classification model training and image classification model method provided by the embodiments of the present application are specifically described below with one specific embodiment, but it should be understood that the embodiments of the present application are not limited thereto. As shown in Figure 6 In one exemplary embodiment, the present application also provides an adversarial defense training method for an image classification neural network and an image classification method, which specifically comprises the following steps:

[0136] Step S602, training the adversarial image generation model to be trained according to the first image classification model and the sample image used to train the first image classification model, to obtain a target adversarial image generation model.

[0137] Step S604, generating a first adversarial image with a deviation from the sample image less than a preset deviation based on the target adversarial image generation model.

[0138] The image classification model is implemented by a DNN model, and the adversarial image generation model is implemented by an AET model.

[0139] As shown in Figure 7 For another schematic diagram of the present embodiment, the above steps S602 to S604 can also be implemented by the following step 1.

[0140] Step 1: Distillation training of the adversarial sample converter (AET model). First, the AET model is pre-trained using the training data set of the deep neural network model (DNN model) to have an initial parameter distribution for the same classification task as the DNN model, and then the decision boundary information of the DNN-raw logits is used to guide the training of the AET model, so that the AET model can inherit the knowledge of the DNN model and promote it to generate more complex adversarial images for subsequent training of the deep neural network-random noise perturbation model (DNN-RNDL model).

[0141] Step S606, a disturbance layer is added in the first image classification model to obtain a third image classification model.

[0142] Step S608, the third image classification model is iteratively trained according to the first adversarial image: the first parameter of the disturbance layer in the third image classification model is fixed, the second parameter of the first image classification model in the third image classification model is updated to obtain an updated second parameter; the updated first parameter is updated while the updated second parameter is fixed to obtain an updated first parameter; the third image classification model under the updated first parameter and the updated second parameter is taken as the second image classification model.

[0143] The disturbance layer is implemented by using an RNDL module (random noise disturbance layer).

[0144] Referring to Figure 7 The steps S606 to S608 can also be implemented by the following step 2.

[0145] Step 2: DNN-RNDL model training. An RNDL module is added to the DNN model, and the RNDL module is designed to randomly disturb the last fully connected layer of the DNN model while maintaining the Top-1 accuracy of the DNN model, so as to improve the security of the model by disturbing the output probability distribution of the DNN model. In this step, the parameters of the DNN model and the RNDL module are trained in an alternating manner to ensure stable convergence of the two sets of parameters. Specifically, when training the DNN parameters, the RNDL parameters are fixed to ensure the Top-1 accuracy of the DNN model; when training the RNDL parameters, the DNN parameters are fixed to ensure that the RNDL module injects random noise into the DNN model and randomly changes the output probability distribution of the DNN model.

[0146] Step S610, the target adversarial image generation model and the second image classification model are combined to obtain a target image classification model.

[0147] Referring to Figure 7 The step S610 can also be implemented by the following step 3.

[0148] Step 3: integrated defense model. The trained AET model and DNN-RNDL model are integrated to form a defense model.

[0149] Step S612, a to-be-classified image is obtained, and the to-be-classified image is input into the target image classification model trained in advance.

[0150] Step S614, generating, by the target adversarial image generation model in the target image classification model, an adversarial image with a deviation less than a preset deviation from the image to be classified, and determining, by the second image classification model in the target image classification model, a classification result of the adversarial image as the classification result of the image to be classified.

[0151] In the embodiment, first, the AET and the DNN are combined, and the RNDL module is integrated to form a multi-level defense system. The AET inherits the knowledge of the DNN through knowledge distillation to generate complex adversarial images for subsequent training, thereby improving the robustness of the model. The RNDL disturbs the output probability distribution of the DNN by injecting random noise with a Gaussian distribution, thereby improving the defense capability of the model against attackers. Second, the parameters of the DNN and the RNDL are alternately trained to ensure that they converge to their respective target states, thereby forming an integrated defense model with high robustness against adversarial attacks. Based on the above process, the embodiment ultimately constructs a neural network defense system with high robustness and high security.

[0152] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by the combination are within the scope of protection of the present application.

[0153] Based on the same inventive concept, the embodiments of the present application also provide an image classification model training device for implementing the above-mentioned image classification model training method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more image classification model training device embodiments provided below can refer to the limitations of the image classification model training method in the above text, which will not be repeated here.

[0154] In one exemplary embodiment, as shown in Figure 8 An image classification model training device is provided, including a first training module 802, an image generation module 804, a second training module 806, and a model combination module 808, wherein:

[0155] The first training module 802 is configured to train the adversarial image generation model to be trained according to the first image classification model and sample images used for training the first image classification model, to obtain a target adversarial image generation model.

[0156] The image generation module 804 is configured to generate a first adversarial image with a deviation from the sample image less than a preset deviation based on the target adversarial image generation model.

[0157] The second training module 806 is configured to retrain the first image classification model according to the first adversarial image, to obtain a second image classification model.

[0158] The model combination module 808 is configured to combine the target adversarial image generation model and the second image classification model, to obtain a target image classification model.

[0159] In an example embodiment, the first training module 802 is further configured to input the sample image into the adversarial image generation model to be trained, to obtain classification prediction information of the sample image corresponding to different preset categories, and generate a second adversarial image with a deviation from the sample image less than a preset deviation; input the second adversarial image into the first image classification model, to obtain classification prediction information of the second adversarial image corresponding to different preset categories; determine a first loss of the adversarial image generation model to be trained according to the classification prediction information of the sample image and the classification prediction information of the second adversarial image, and determine a second loss of the adversarial image generation model to be trained according to the classification prediction information of the sample image and the real category; iteratively train the adversarial image generation model to be trained according to the first loss and the second loss of the adversarial image generation model to be trained, to obtain the target adversarial image generation model.

[0160] In an example embodiment, the second training module 806 is further configured to add a disturbance layer to the first image classification model, to obtain a third image classification model; the disturbance layer is configured to inject disturbance information into classification prediction information generated by the third image classification model; iteratively train the third image classification model according to the first adversarial image, to obtain the second image classification model; and for the same image, the classification result output by the second image classification model is the same as the classification result output by the first image classification model.

[0161] In an example embodiment, the second training module 806 is further configured to fix first parameters of the third image classification model, update second parameters of the third image classification model to obtain updated second parameters of the third image classification model, the first parameters are parameters of a perturbation layer in the third image classification model, and the second parameters are parameters of the first image classification model in the third image classification model; fix the updated second parameters of the third image classification model, update the first parameters of the third image classification model to obtain updated first parameters of the third image classification model; and obtain the second image classification model according to the third image classification model with the updated first parameters and the updated second parameters.

[0162] Based on the same inventive concept, the embodiments of the present application also provide an image classification device for implementing the above-mentioned image classification method. The implementation scheme of the device for solving the problem is similar to the implementation scheme described in the above method, so the specific limitations in one or more image classification device embodiments provided below can refer to the limitations of the image classification method in the above text, which will not be described here again.

[0163] In an example embodiment, as shown in Figure 9 An image classification device is provided, which includes an image acquisition module 902, an image generation module 904, and an image classification module 906.

[0164] The image acquisition module 902 is configured to acquire an image to be classified and input the image to be classified into a target image classification model trained in advance, the target image classification model at least including an adversarial image generation model and an image classification model, and the target image classification model being trained according to an image classification model training method.

[0165] The image generation module 904 is configured to generate, by the adversarial image generation model, an adversarial image with a deviation from the image to be classified being less than a preset deviation.

[0166] The image classification module 906 is configured to determine a classification result of the adversarial image by the image classification model to obtain a classification result of the image to be classified.

[0167] The above-mentioned image classification model training device and image classification device can be all or partially implemented by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform the operations corresponding to the above-mentioned modules.

[0168] In an example embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 10As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store sample image data. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the terminal outside through the network connection. The computer program is executed by the processor to realize an image classification model training method and an image classification method.

[0169] Those skilled in the art can understand that, Figure 10 The skilled in the art can understand that,

[0170] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps in the above method embodiments.

[0171] In an exemplary embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by the processor to realize the steps in the above method embodiments.

[0172] In an exemplary embodiment, a computer program product is provided, including a computer program, and the computer program is executed by the processor to realize the steps in the above method embodiments.

[0173] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0174] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0175] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for training an image classification model, characterized in that, The method includes: Based on the first image classification model and the sample images used to train the first image classification model, the adversarial image generation model to be trained is trained to obtain the target adversarial image generation model. Based on the target adversarial image generation model, a first adversarial image with a deviation less than a preset deviation from the sample image is generated; Based on the first adversarial image, the first image classification model is retrained to obtain a second image classification model. The target adversarial image generation model and the second image classification model are combined to obtain the target image classification model.

2. The method according to claim 1, characterized in that, The step of training an adversarial image generation model to obtain a target adversarial image generation model based on a first image classification model and sample images used to train the first image classification model includes: The sample image is input into the adversarial image generation model to be trained to obtain classification prediction information of the sample image corresponding to different preset categories, and to generate a second adversarial image with a deviation from the sample image that is smaller than the preset deviation. The second adversarial image is input into the first image classification model to obtain classification prediction information of the second adversarial image corresponding to the different preset categories; Based on the classification prediction information of the sample image and the classification prediction information of the second adversarial image, the first loss of the adversarial image generation model to be trained is determined, and based on the classification prediction information of the sample image and the true category, the second loss of the adversarial image generation model to be trained is determined. Based on the first loss and the second loss of the adversarial image generation model to be trained, the adversarial image generation model to be trained is iteratively trained to obtain the target adversarial image generation model.

3. The method according to claim 1 or 2, characterized in that, The step of retraining the first image classification model based on the first adversarial image to obtain a second image classification model includes: A perturbation layer is added to the first image classification model to obtain a third image classification model; the perturbation layer is used to inject perturbation information into the classification prediction information generated by the third image classification model. Based on the first adversarial image, the third image classification model is iteratively trained to obtain the second image classification model; for the same image, the classification result output by the second image classification model based on the classification prediction information injected with the perturbation information is the same as the classification result output by the first image classification model.

4. The method according to claim 3, characterized in that, The second image classification model is obtained in the following way: The first parameter of the third image classification model is fixed, and the second parameter of the third image classification model is updated to obtain the updated second parameter of the third image classification model. The first parameter is the parameter of the perturbation layer in the third image classification model, and the second parameter is the parameter of the first image classification model in the third image classification model; The updated second parameter of the third image classification model is fixed, and the first parameter of the third image classification model is updated to obtain the updated first parameter of the third image classification model. The second image classification model is obtained based on the third image classification model under the updated first parameters and the updated second parameters.

5. An image classification method, characterized in that, The method includes: A target image to be classified is obtained, and the target image to be classified is input into a pre-trained target image classification model; the target image classification model includes at least an adversarial image generation model and an image classification model; the target image classification model is trained according to the image classification model training method according to any one of claims 1 to 4. The adversarial image generation model generates an adversarial image whose deviation from the image to be classified is less than a preset deviation. The classification result of the adversarial image is determined by the image classification model, and the classification result of the image to be classified is obtained.

6. An image classification model training device, characterized in that, The device includes: The first training module is used to train the adversarial image generation model to be trained based on the first image classification model and the sample images used to train the first image classification model, so as to obtain the target adversarial image generation model. The image generation module is used to generate a first adversarial image with a deviation less than a preset deviation from the sample image, based on the target adversarial image generation model. The second training module is used to retrain the first image classification model based on the first adversarial image to obtain a second image classification model. The model combination module is used to combine the target adversarial image generation model and the second image classification model to obtain the target image classification model.

7. An image classification device, characterized in that, The device includes: An image acquisition module is used to acquire an image to be classified and input the image to be classified into a pre-trained target image classification model; the target image classification model includes at least an adversarial image generation model and an image classification model; the target image classification model is trained according to the image classification model training method according to any one of claims 1 to 4. The image generation module is used to generate an adversarial image with a deviation of less than a preset deviation from the image to be classified by the adversarial image generation model. The image classification module is used to determine the classification result of the adversarial image through the image classification model, and obtain the classification result of the image to be classified.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the image classification model training method according to any one of claims 1 to 4 or the image classification method according to claim 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the image classification model training method according to any one of claims 1 to 4 or the image classification method according to claim 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the image classification model training method according to any one of claims 1 to 4 or the image classification method according to claim 5.

Images