Robust image classification method and device based on causal diffusion model

Abstract

The invention provides a robust image classification method based on a causal diffusion model. The method comprises the following steps: modeling a structural causal model through a conditional diffusion model; collecting a sample image serving as a training sample from the image data set; extracting causal features and non-causal features of the training sample through a structural causal model, calculating a causal information bottleneck, maximizing the causal information bottleneck, and updating the structural causal model to obtain an intermediate model; generating causal features of the training sample by using the intermediate model, guiding the intermediate model to predict a prediction category of the training sample, constructing a loss function according to the prediction category and a category label of the training sample, and training the intermediate model to obtain an image classification model; acquiring an adversarial sample image marked with a target category, and performing purification processing on the received adversarial sample by using the image classification model to obtain a purified image; and extracting causal features of the purified image by using the image classification model, and obtaining an image classification result of the purified image based on the causal features of the purified image.

Description

Technical Field

[0001] The present invention relates to the field of image classification task adversarial defense technology, and in particular to a method and device for image classification that designs a diffusion model embedded with a causal mechanism to achieve adversarial robustness. Background Art

[0002] In the field of artificial intelligence, especially in scenarios using deep neural networks, models are often threatened by various adversarial attacks, which mislead models into making incorrect judgments by making tiny, imperceptible modifications to input data. Although there are many methods for defending against adversarial attacks, such as adversarial training and data purification, these methods usually face problems such as being unable to generalize to unseen attack methods and limited robustness.

[0003] Most methods focus on defending against known attacks, but lack adaptability and generalization capabilities for attacks not seen during training. Existing methods such as adversarial training can enhance the model's defense capabilities against specific attacks, but are not effective against unseen attacks. For adversarial purification methods, the purification amplitude is strongly dependent on the attack intensity, and the performance of attacks with varying amplitudes is significantly impaired. In summary, existing methods such as adversarial training and purification can improve the model's defense capabilities against specific attacks, but are not effective when facing unseen attacks or when the attack intensity varies. Summary of the invention

[0004] The technical problem to be solved by this application proposal is how to overcome the problem that the existing adversarial training and adversarial purification methods have poor defense effects when facing unseen attacks or changes in attack intensity. To this end, the present invention proposes a new defense framework, the Causal Diffusion Model (CausalDiff), which combines the Structural Causal Model (SCM) and the Conditional Diffusion Model (DDPM). This method effectively improves the robustness of the model against unknown attacks and enhances its adaptability and generalization in a dynamic adversarial environment by identifying and utilizing key causal features in the data generation process. In addition, the present invention also provides a complete system implementation plan, including detailed steps of data processing, model training and inference process, to ensure the effectiveness and practicality of the method.

[0005] Specifically, in order to achieve the above technical effects, the present invention proposes a method as follows Figure 4 The robust image classification method based on the causal diffusion model shown in FIG. 1 includes:

[0006] The construction step is to model the structural causal model through the conditional diffusion model, and the causal features and the non-causal features are used as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories;

[0007] The training step includes obtaining an image data set, wherein the image data set includes a plurality of sample images with annotated category labels; collecting sample images as training samples from the image data set; extracting the causal features and non-causal features of the training samples through the structural causal model, and calculating the causal information bottleneck of the causal features and the non-causal features, and updating the structural causal model by maximizing the causal information bottleneck to obtain an intermediate model; using the intermediate model to generate the causal features of the training samples, guiding the intermediate model to predict the predicted category of the training samples, and constructing a loss function according to the predicted category of the training samples and the category label, and training the intermediate model to obtain an image classification model;

[0008] A purification step, obtaining an adversarial sample image of a marked target category, and applying the image classification model to purify the received adversarial sample. The image classification model removes adversarial perturbations in the adversarial sample image by maximizing the likelihood of the adversarial sample image to obtain a purified image;

[0009] The extraction step uses the image classification model to extract the causal features of the purified image, and obtains an image classification result of the purified image based on the causal features of the purified image.

[0010] In the robust image classification method based on the causal diffusion model, the adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

[0011] The robust image classification method based on the causal diffusion model, wherein the purification step comprises:

[0012] The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood to the adversarial sample image is calculated, the adversarial sample image is gradient-ascended, and the purified image is obtained.

[0013] The robust image classification method based on the causal diffusion model, wherein the causal information bottleneck is:

[0014] CIB(X,Y,S,Z)=I(X;S,Z)+I(Y;S)-I(S;Z)-λI(X;S,Z)

[0015] Where I(·; ·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label;

[0016] The causal information bottleneck is maximized by increasing I(X; S, Z). At the same time, the structural causal model can predict Y based on S, decouple S and Z, and S and Z only match key semantic information.

[0017] like Figure 5As shown, the present invention also proposes a robust image classification device based on a causal diffusion model, which includes:

[0018] The construction module models the structural causal model through the conditional diffusion model, and the causal features and non-causal features are used as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories;

[0019] A training module obtains an image data set, wherein the image data set has a plurality of sample images with annotated category labels; collects sample images as training samples from the image data set; extracts causal features and non-causal features of the training samples through the structural causal model, and calculates the causal information bottleneck of the causal features and the non-causal features, and updates the structural causal model by maximizing the causal information bottleneck to obtain an intermediate model; uses the intermediate model to generate the causal features of the training samples, guides the intermediate model to predict the predicted category of the training samples, and constructs a loss function based on the predicted category of the training samples and the category label, trains the intermediate model, and obtains an image classification model;

[0020] A purification module obtains an adversarial sample image of a marked target category and applies the image classification model to purify the received adversarial sample. The image classification model removes adversarial perturbations in the adversarial sample image by maximizing the likelihood of the adversarial sample image to obtain a purified image.

[0021] an extraction module, which extracts causal features of the purified image using the image classification model, and obtains an image classification result of the purified image based on the causal features of the purified image;

[0022] The adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

[0023] The robust image classification device based on the causal diffusion model, wherein the purification module includes:

[0024] The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood to the adversarial sample image is calculated, the adversarial sample image is gradient-ascended, and the purified image is obtained.

[0025] The robust image classification device based on the causal diffusion model, wherein the causal information bottleneck is:

[0026] CIB(X,Y,S,Z)=I(X;S,Z)+I(Y;S)-I(S;Z)-λI(X;S,Z)

[0027] Where I(·; ·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label;

[0028] The causal information bottleneck is maximized by increasing I(X; S, Z). At the same time, the structural causal model can predict Y based on S, decouple S and Z, and S and Z only match key semantic information.

[0029] The present invention also proposes an electronic device, which includes the above-mentioned robust image classification device. The electronic device may be connected to an information display device, and the information display device is used to display the image classification result according to display parameters and attributes set by the user or through an artificial intelligence model.

[0030] The present invention also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the robust image classification methods are implemented.

[0031] The present invention also proposes a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the robust image classification methods are implemented.

[0032] It can be seen from the above scheme that the advantages of the present invention are:

[0033] The causal diffusion model proposed in this invention provides an innovative adversarial attack defense mechanism by combining the structural causal model (SCM) and the conditional diffusion model, significantly enhancing the defense capability against unknown attacks and improving the robustness and generalization capability of the model. The causal diffusion model includes two steps of causal feature decoupling and purification during reasoning, and we also tested the effect of each module when used alone. Compared with the existing technology, this method can effectively reduce the dependence on large-scale data sets, optimize the adversarial purification process, and provide higher security and reliability in high-risk application scenarios, especially showing excellent performance when facing complex and changing attack environments.

[0034] Table 1 shows a series of adversarial test results using AutoAttack (AA) on the CIFAR-10 dataset, which evaluate the robustness of different models in the face of advanced adversarial attacks. AutoAttack is an automated adversarial attack tool that integrates multiple attack methods to comprehensively test the defense of the model.

[0035] The attack methods listed in the table include:

[0036] AA Linf (perturbation range is 8 / 255): AutoAttack using the L-infinity norm. This attack limits the maximum perturbation allowed for each pixel to no more than 8 / 255, thereby testing the model’s sensitivity to extreme pixel perturbations.

[0037] AA L2 (perturbation range is 0.5): AutoAttack using the L2 norm, which evaluates the sensitivity to global image perturbations (the Euclidean distance of all pixel perturbations does not exceed 0.5).

[0038] StAdv (perturbation range is 8 / 255): Spatially Transformed Adversarial Attack (StAdv), which misleads the model by performing subtle geometric deformation on the image and tests the robustness of the model to spatial transformation.

[0039] The table also lists the accuracy of each model when it is not subjected to any attack and the performance under various attacks. Finally, an average robustness score is provided to reflect the overall defense ability of the model in all considered attack methods. The model includes the standard causal diffusion model and a feature decoupling module and the individual defense effects of the purification module.

[0040] These results clearly demonstrate the potential of causal diffusion techniques to improve model performance in adversarial environments, especially how they help models maintain high accuracy and robustness in the face of diverse adversarial attacks. This comprehensive evaluation highlights the defensive performance of causal diffusion models against unseen attack methods. It is worth mentioning that since the causal diffusion model is not trained with any adversarial examples, any attack method used during testing is unseen for the causal diffusion model.

[0041] Table 1: Performance test results of the causal diffusion model on the CIFAR10 dataset

[0042]

[0043] Table 2: Performance test results of the causal diffusion model on the GTSRB dataset

[0044]

[0045] Table 2 shows a series of adversarial test results using AutoAttack (AA) on the GTSRB dataset, which evaluates the robustness of the causal diffusion model in the face of advanced adversarial attacks. AutoAttack is an automated adversarial attack tool that integrates multiple attack methods to comprehensively test the model's defense capabilities.

[0046] The attack methods listed in the table include:

[0047] AA Linf (perturbation range is 8 / 255): AutoAttack using the L-infinity norm. This attack limits the maximum perturbation allowed for each pixel to no more than 8 / 255, thereby testing the model’s sensitivity to extreme pixel perturbations.

[0048] AA L2 (perturbation range is 0.5): AutoAttack using the L2 norm, which evaluates the sensitivity to global image perturbations (the Euclidean distance of all pixel perturbations does not exceed 0.5).

[0049] Fog: This test simulates the impact of foggy environment on images and evaluates the robustness of the model under visually obstructed conditions.

[0050] Table 2 also shows the high accuracy of the causal diffusion model when it is not subjected to any attack (97.85%) and the performance under various attacks, with an average robustness of 82.62%, to reflect the overall defense capability of the model in all considered attack methods. These results clearly show the potential of causal diffusion techniques to improve the performance of models in adversarial environments, especially how it helps the model maintain high accuracy and robustness in the face of diverse adversarial attacks. This comprehensive evaluation highlights the defense performance of the causal diffusion model against unseen attack methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the structural causal model;

[0052] Figure 2 A diagram of the training method for the causal diffusion model;

[0053] Figure 3 Flowchart of the robust inference method for the causal diffusion model;

[0054] Figure 4 is a flow chart of the method of the present invention;

[0055] Figure 5 It is a module diagram of the device of the present invention;

[0056] Figure 6 This is a schematic diagram of the structure of a first electronic device of the present invention;

[0057] Figure 7 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0058] Figure 8 It is a schematic diagram of the structure of a second electronic device of the present invention.

[0059] Reference numerals:

[0060] A-First electronic device;

[0061] B-Robust image classification device;

[0062] C-data acquisition equipment;

[0063] D-information display device;

[0064] 1000 - second electronic device;

[0065] Ⅰ-computational unit;

[0066] II-ROM;

[0067] III-RAM;

[0068] IV-bus;

[0069] V-interface;

[0070] VI - input unit;

[0071] VII-output unit;

[0072] VIII- Storage medium;

[0073] Ⅸ-Communication unit. DETAILED DESCRIPTION

[0074] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0075] Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0076] The processor described in the present invention is the control center of the electronic device, which can be a processor or a general term for multiple processing elements. For example, it can be one or more central processing units (CPUs), or application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0077] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0078] In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0079] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0080] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0081] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0082] It should also be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0083] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0084] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0085] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0086] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0087] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0088] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0089] During the research process, the inventors found that although existing methods such as adversarial training and purification can improve the defense capabilities of the model against specific attacks, they are not effective when facing unseen attacks or changes in attack intensity. To this end, the inventors proposed a causal diffusion model (CausalDiff), which is a new framework that combines the structural causal model (SCM) and the conditional diffusion model. By identifying and utilizing key causal features from the data generation process, it effectively improves the robustness of the model to various unseen attacks. This method not only reduces the impact of adversarial attacks, but also improves the adaptability and generalization of the model in a dynamic attack environment. In order to achieve the above technical effects, the present invention proposes the following key technical points:

[0090] Key point 1, causal feature recognition framework based on diffusion model. Model the structural causal model through the conditional diffusion model; perform model initialization and data processing; randomly extract small batches of samples from the large-scale original data set; use the causal diffusion model to calculate the causal information bottleneck for the small batch; optimize and update the causal diffusion model parameters through loss back propagation; the causal diffusion model obtained by this method can identify causal and non-causal features for the input samples.

[0091] Key point 2: A device that uses the causal diffusion model to improve the adversarial robustness of image classification tasks. In the model inference stage, for the input image, the causal diffusion model is first used to purify the sample; after obtaining the purified sample, the causal diffusion model identifies the causal features and non-causal features; based on the causal features, the causal diffusion model classifies the causal features to obtain the predicted category; this device enables the model to maintain a high level of classification accuracy and robustness even in the face of changing attack intensity when performing image classification tasks in an adversarial environment.

[0092] In order to make the above features and effects of the present invention more clearly and understandably described, embodiments are given below and described in detail with reference to the accompanying drawings. This specification discloses one or more embodiments that include the features of the present invention. The disclosed embodiments are only for illustration. The scope of protection of the present invention is not limited to the disclosed embodiments, and the present invention is defined by the attached claims.

[0093] Figure 1 It is a structural causal model, where nodes represent variables and arrows represent causal relationships, in which causal features work together with non-causal features to determine image samples, while only causal features determine category labels.

[0094] Figure 2 It is a training method for the causal diffusion model. The method is implemented through a specific flowchart. The main purpose is to use clean data samples to train the causal diffusion model to learn the data generation mechanism to achieve robust image classification. The following are the detailed steps of the process:

[0095] Step 1: Modeling structural causal model through conditional diffusion model (S101)

[0096] The causal model generates images and their corresponding categories based on causal features and non-causal features. Specifically, causal features and non-causal features jointly generate images, and causal features generate categories. The present invention uses the conditional diffusion model to model Figure 1 The structural causal model shown in Figure 1. In the conditional diffusion model, causal features and non-causal features serve as conditional variables to guide the image generation process of the diffusion model, that is, modeling Figure 1 The generation process of causal features and non-causal features to images is shown. The present invention uses a neural network to model the classification process of causal features to category labels. This classifier also serves as a part of the causal diffusion model.

[0097] Among them, the difference between the causal diffusion model adopted by the present invention and the traditional diffusion model is that the conditional diffusion model adds a feature vector as a condition on the basis of the diffusion model. The present invention adopts the conditional diffusion model to model the conditional probability p(x|s,z), and uses the conditional diffusion model that can obtain the conditional probability p(x|s,z) as the causal diffusion model; wherein x is an image, s is a causal feature, and z is a non-causal feature. Usually, the diffusion model models p(x), so it can generate an image by maximizing p(x). The conditional diffusion model generates an image of corresponding content based on the input conditions, namely the s and z feature vectors, thereby using the feature vector as a condition to control the image generation process. The causal diffusion model of the present invention predicts the category by connecting a discriminator to the feature vector s.

[0098] Step 2: Randomly initialize the causal diffusion model (S102)

[0099] The causal diffusion model is randomly initialized and the initial parameters are set. This includes the initial random assignment of weights and biases to set the starting point for the model training process. This step is crucial to the subsequent learning efficiency and the final performance of the model.

[0100] Among them, weights and biases are both neural network model parameters, which are the parameters of the causal diffusion model. The training process is the process of updating these parameters. The initialization setting is to randomly initialize the parameters, that is, to sample values ​​from the Gaussian distribution as the initial values ​​of the parameters.

[0101] Step 3: Randomly extract small batch samples from the large-scale original dataset (S103)

[0102] Randomly extract small batches of samples from a large-scale original data set. This process involves randomly selecting data from the entire data set according to certain rules for single-iteration training. Small batch training helps improve the generalization ability of neural network models and reduces memory consumption during training.

[0103] The dataset contains a certain number of images and corresponding category labels. For example, the CIFAR-10 dataset contains 50,000 training sample images and category labels. The category labels are divided into ten categories: airplane, car, bird, cat, deer, dog, frog, horse, ship, and truck.

[0104] Step 4: Calculate the causal information bottleneck lower bound (S104)

[0105] The extracted small batch samples are input into the Causal Information Bottleneck (CIB) calculation module. The CIB module is a training target derived from the structural causal model. The structural causal model is shown in the attached figure. Figure 1 As shown in Figure 2, this CIB is a training target derived from the structural causal model and is a module independent of the structural causal model. In this module, maximizing the causal information bottleneck optimizes the model parameters so that it can remove irrelevant non-causal information while retaining important causal features. The causal information bottleneck is defined as

[0106] CIB(X,Y,S,Z)=I(X;S,Z)+I(Y;S)-I(S;Z)-λI(X;S,Z)

[0107] Where I(·;·) represents the mutual information between the two, S represents the causal features extracted by the structural causal model, Z represents the non-causal features extracted by the structural causal model, X represents the image, and Y represents the correct category label of the sample. The meaning of the causal information bottleneck is to make the mutual information between the causal feature S and the non-causal feature Z and X as large as possible, and at the same time S can predict Y, and S and Z are as decoupled as possible, that is, S and Z should not match all the detailed information in X, so that S and Z only learn the abstract semantic features in X, and then only match the key semantic information.

[0108] Step 5: Loss back propagation and model parameter update (S105)

[0109] The calculated loss is used to update the parameters of the causal diffusion model through the back-propagation algorithm. This step is a standard process in machine learning, which optimizes the model after multiple iterations by adjusting the parameters in the model to minimize the loss function.

[0110] Figure 3 This is a flowchart of a robust reasoning method for a causal diffusion model, which aims to predict the category of input samples through a causal diffusion model, especially in the face of adversarial environments or noise interference. The following is a specific implementation method described in detail according to the flowchart:

[0111] Step 1: Configure initial conditions and model parameters (S201)

[0112] Before model training, first configure Figure 2 The causal diffusion model is trained through the training process shown, which includes setting model parameters such as weights, biases, and network structure to ensure that the model is optimized and ready to receive new input data.

[0113] Step 2: Accept adversarial sample image input (S202)

[0114] The system accepts adversarial sample images as external input. These images are usually designed to mislead the model to make wrong predictions or classifications. These samples are received in order to test and use the causal diffusion model to identify and correct the perturbations introduced by the adversarial attack.

[0115] Among them, adversarial samples are those in which the attacker adds adversarial perturbations to the original samples to cause the model to misclassify. To humans, adversarial perturbations appear as imperceptible noise.

[0116] Step 3: Purification using the causal diffusion model (S203)

[0117] The causal diffusion model is applied to purify the received adversarial samples. This process removes or mitigates the adversarial perturbations in the sample by maximizing the sample likelihood through the diffusion model, thereby restoring the original or true characteristics of the sample. The purified sample is expected to be closer to the state before the attack.

[0118] Among them, the way to maximize the sample likelihood is: use the diffusion model to estimate the sample likelihood, which is an existing method; then back-propagate the sample likelihood, calculate the gradient of the sample likelihood to the image sample, and perform gradient ascent on the image to maximize the sample likelihood of the image.

[0119] The diffusion model calculates likelihood by estimating the image likelihood through the lower bound of evidence, which is a common method used by diffusion models. Since adversarial samples are generally less likely than clean samples, the sample likelihood means the probability of the diffusion model generating the image. If the probability of generating the image is high, that is, the likelihood is high, then the sample is more likely to be a clean sample rather than an adversarial sample. Therefore, adversarial sample purification can be achieved by maximizing the sample likelihood. The result of purification is the purified image sample, not the feature.

[0120] Step 4: Purify samples using causal diffusion model reasoning (S204)

[0121] Further reasoning analysis is performed on the purified samples to extract key causal and non-causal features using the causal diffusion model. This step is performed through the causal reasoning ability of the model, aiming to clarify which features are the key factors in determining the sample category.

[0122] Step 5: Use the causal diffusion model to make category predictions based on causal features (S205)

[0123] Based on the causal features extracted from the previous step, the causal diffusion model is used to make the final category prediction. This step uses causal features for category prediction and ignores the influence of non-causal features, so that the adversarial perturbations that may remain in S203 will also be regarded as non-causal features and will not affect the category prediction results, thereby improving the accuracy of the prediction and the model's ability to resist adversarial attacks.

[0124] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0125] like Figure 5 As shown, the present invention also proposes a robust image classification device based on a causal diffusion model, which includes:

[0126] The construction module models the structural causal model through the conditional diffusion model, and the causal features and non-causal features are used as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories;

[0127] A training module obtains an image data set, wherein the image data set has a plurality of sample images with annotated category labels; collects sample images as training samples from the image data set; extracts causal features and non-causal features of the training samples through the structural causal model, and calculates the causal information bottleneck of the causal features and the non-causal features, and updates the structural causal model by maximizing the causal information bottleneck to obtain an intermediate model; uses the intermediate model to generate the causal features of the training samples, guides the intermediate model to predict the predicted category of the training samples, and constructs a loss function based on the predicted category of the training samples and the category label, trains the intermediate model, and obtains an image classification model;

[0128] A purification module obtains an adversarial sample image of a marked target category and applies the image classification model to purify the received adversarial sample. The image classification model removes adversarial perturbations in the adversarial sample image by maximizing the likelihood of the adversarial sample image to obtain a purified image.

[0129] an extraction module, which extracts causal features of the purified image using the image classification model, and obtains an image classification result of the purified image based on the causal features of the purified image;

[0130] The adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

[0131] The robust image classification device based on the causal diffusion model, wherein the purification module includes:

[0132] The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood to the adversarial sample image is calculated, the adversarial sample image is gradient-ascended, and the purified image is obtained.

[0133] The robust image classification device based on the causal diffusion model, wherein the causal information bottleneck is:

[0134] CIB(X,Y,S,Z)=I(X;S,Z)+I(Y;S)-I(S;Z)-λI(X;S,Z)

[0135] Where I(·; ·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label;

[0136] The causal information bottleneck is maximized by increasing I(X; S, Z). At the same time, the structural causal model can predict Y based on S, decouple S and Z, and S and Z only match key semantic information.

[0137] like Figure 6 As shown, the present invention further proposes a first electronic device A in another embodiment, including the above-mentioned robust image classification device, and the electronic device may be connected to an information display device.

[0138] like Figure 7 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to collect and obtain images to be identified and classified, and the information display device D is used to display the classification results obtained by the analysis of the present invention.

[0139] The information display device D can sort and process the data output by the first electronic device A based on the information display mechanism to improve the readability of the data output by the first electronic device A. The information display mechanism can be manually preset, for example, the data output by the first electronic device A is visually displayed, which can be based on the display parameters and / or attributes set by the user. The display parameters can be, for example, the display data range, and the display attributes can be, for example, the display font, color, whether to scroll and play, etc. The user is presented with the key information specified by the user, for example, the categories of special concern are displayed in bold, etc. The user can understand this information more timely without having to access the secondary page or scroll the page, saving the user's operation. Or the information display mechanism can be an artificial intelligence AI display model, which can learn the user's key information based on the user's previous usage habits, such as viewing time, number of clicks, number of edits, etc., and then automatically present rich and necessary key information to the user.

[0140] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored in a readable storage medium. When the computer program is executed by a processor, the computer can execute the robust image classification method provided by the above methods.

[0141] The present invention also proposes a storage medium VIII in another embodiment for storing a computer program for executing the robust image classification method. It should be understood that the storage medium in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0142] Figure 8 A schematic block diagram of a second electronic device 1000 that can be used to implement an embodiment of the present invention is shown. The second electronic device 1000 electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein. The second electronic device 1000 may be the same or different from the first electronic device A.

[0143] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from a storage medium VIII into a random access memory (RAM) III. In RAM III, various programs and data required for the operation of the device 1000 can also be stored. The computing unit I, ROM II, and RAM III are connected to each other via a bus IV. An input / output (I / O) interface V is also connected to the bus IV.

[0144] A plurality of components in the second electronic device 1000 are connected to the I / O interface V, including: an input unit VI, such as a keyboard, a mouse, etc.; an output unit VII, such as various types of displays, speakers, etc.; a storage medium VIII, such as a disk, an optical disk, etc.; and a communication unit IX, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0145] The computing unit I may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S4. For example, in some embodiments, the method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage medium VIII. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via ROM II and / or a communication unit IX. When the computer program is loaded into RAM III and executed by the computing unit I, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit I may be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0146] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A robust image classification method based on causal diffusion model, characterized in that: include: The construction step is to model the structural causal model through the conditional diffusion model, and the causal features and the non-causal features are used as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories; The training step includes obtaining an image data set, wherein the image data set includes a plurality of sample images with annotated category labels; collecting sample images as training samples from the image data set; extracting the causal features and non-causal features of the training samples through the structural causal model, and calculating the causal information bottleneck of the causal features and the non-causal features, and updating the structural causal model by maximizing the causal information bottleneck to obtain an intermediate model; using the intermediate model to generate the causal features of the training samples, guiding the intermediate model to predict the predicted category of the training samples, and constructing a loss function according to the predicted category of the training samples and the category label, and training the intermediate model to obtain an image classification model; A purification step, obtaining an adversarial sample image of a marked target category, and applying the image classification model to purify the received adversarial sample. The image classification model removes adversarial perturbations in the adversarial sample image by maximizing the likelihood of the adversarial sample image to obtain a purified image; The extraction step uses the image classification model to extract the causal features of the purified image, and obtains an image classification result of the purified image based on the causal features of the purified image.

2. The robust image classification method based on the causal diffusion model as claimed in claim 1, characterized in that: The adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

3. The robust image classification method based on the causal diffusion model as claimed in claim 1, characterized in that: The purification steps include: The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood to the adversarial sample image is calculated, the adversarial sample image is gradient-ascended, and the purified image is obtained.

4. The robust image classification method based on the causal diffusion model as claimed in claim 1, characterized in that: The causal information bottleneck is: Where I(·;·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label; Maximizing this causal information bottleneck is done by increasing At the same time, the structural causal model can predict Y based on S, decouple S and Z, and S and Z only match key semantic information.

5. A robust image classification device based on a causal diffusion model, characterized in that: include: The construction module models the structural causal model through the conditional diffusion model, and the causal features and non-causal features are used as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories; A training module obtains an image data set, wherein the image data set has a plurality of sample images with annotated category labels; collects sample images as training samples from the image data set; extracts causal features and non-causal features of the training samples through the structural causal model, and calculates the causal information bottleneck of the causal features and the non-causal features, and updates the structural causal model by maximizing the causal information bottleneck to obtain an intermediate model; uses the intermediate model to generate the causal features of the training samples, guides the intermediate model to predict the predicted category of the training samples, and constructs a loss function based on the predicted category of the training samples and the category label, trains the intermediate model, and obtains an image classification model; A purification module obtains an adversarial sample image of a marked target category and applies the image classification model to purify the received adversarial sample. The image classification model removes adversarial perturbations in the adversarial sample image by maximizing the likelihood of the adversarial sample image to obtain a purified image. an extraction module, which extracts causal features of the purified image using the image classification model, and obtains an image classification result of the purified image based on the causal features of the purified image; The adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

6. The robust image classification device based on the causal diffusion model according to claim 5, characterized in that: The purification module includes: The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood to the adversarial sample image is calculated, the adversarial sample image is gradient-ascended, and the purified image is obtained.

7. The robust image classification device based on the causal diffusion model according to claim 5, characterized in that: The causal information bottleneck is: Where I(·;·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label; Maximizing this causal information bottleneck is done by increasing At the same time, the structural causal model can predict Y based on S, decouple S and Z, and S and Z only match key semantic information.

8. An electronic device, characterized in that: It comprises a robust image classification device as described in claims 5-7, wherein the electronic device is connected to an information display device, and the information display device is used to display the image classification result according to display parameters and attributes set by the user or through an artificial intelligence model.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the robust image classification method according to any one of claims 1 to 4 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the robust image classification method described in any one of claims 1 to 4 are implemented.

Images