Robust image classification method and device based on causal diffusion model

Abstract

The present invention proposes a robust image classification method based on a causal diffusion model, comprising: modeling a structural causal model through a conditional diffusion model; collecting sample images as training samples from an image data set; extracting causal features and non-causal features of the training samples through the structural causal model and calculating the causal information bottleneck, maximizing the causal information bottleneck, updating the structural causal model, and obtaining an intermediate model; using the intermediate model to generate causal features of the training samples, guiding the intermediate model to predict the predicted category of the training samples, and constructing a loss function according to the predicted category and category label of the training samples, training the intermediate model, and obtaining an image classification model; obtaining adversarial sample images of marked target categories, applying the image classification model to purify the received adversarial samples, and obtaining purified images; using the image classification model to extract causal features of the purified images, and obtaining image classification results of the purified images based on the causal features of the purified images.

Description

Technical Field

[0001] The present invention relates to the field of adversarial defense technology for image classification tasks, and in particular to a method and device for implementing adversarial robust image classification by designing a diffusion model embedded with a causal mechanism. Background Art

[0002] In the field of artificial intelligence, especially in scenarios using deep neural networks, models are often threatened by various adversarial attacks. These attacks mislead the model into making incorrect judgments by modifying input data in subtle, imperceptible ways. While various methods for defending against adversarial attacks exist, such as adversarial training and data cleansing, these methods often suffer from problems such as inability to generalize to unseen attack methods and limited robustness.

[0003] Most methods focus on defending against known attacks, lacking adaptability and generalization capabilities for attacks not seen during training. Existing methods, such as adversarial training, can enhance a model's defense against specific attacks but are less effective against unseen attacks. Adversarial purification methods, however, rely heavily on attack strength, significantly impairing performance against attacks with varying strengths. In summary, while existing methods such as adversarial training and purification can improve a model's defense against specific attacks, they are less effective against unseen attacks or those with varying strengths. Summary of the Invention

[0004] The technical problem to be solved by this application proposal is how to overcome the problem that the existing adversarial training and adversarial purification methods have poor defense effects when facing unseen attacks or changes in attack intensity. To this end, the present invention proposes a new defense framework, the Causal Diffusion Model (CausalDiff), which combines the Structural Causal Model (SCM) and the Conditional Diffusion Model (DDPM). This method effectively improves the robustness of the model against unknown attacks and enhances its adaptability and generalization in dynamic adversarial environments by identifying and utilizing key causal features in the data generation process. In addition, the present invention also provides a complete system implementation plan, including detailed steps of data processing, model training and inference process, to ensure the effectiveness and practicality of the method.

[0005] Specifically, in order to achieve the above technical effects, the present invention proposes a method as follows Figure 4 The robust image classification method based on the causal diffusion model shown in FIG. 1 includes:

[0006] The construction step is to model a structural causal model through a conditional diffusion model, wherein causal features and non-causal features are used as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories;

[0007] A training step includes obtaining an image dataset having a plurality of sample images with annotated category labels; collecting sample images as training samples from the image dataset; extracting causal features and non-causal features of the training samples through the structural causal model, and calculating the causal information bottleneck of the causal features and the non-causal features, and updating the structural causal model by maximizing the causal information bottleneck to obtain an intermediate model; using the intermediate model to generate the causal features of the training samples, guiding the intermediate model to predict the predicted category of the training samples, and constructing a loss function based on the predicted category and the category label of the training samples, training the intermediate model, and obtaining an image classification model;

[0008] The purification step obtains an adversarial sample image with a labeled target category and applies the image classification model to purify the received adversarial sample. The image classification model removes the adversarial perturbation in the adversarial sample image by maximizing the likelihood of the adversarial sample image to obtain a purified image.

[0009] The extraction step uses the image classification model to extract the causal features of the purified image, and obtains an image classification result of the purified image based on the causal features of the purified image.

[0010] In the robust image classification method based on the causal diffusion model, the adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

[0011] The robust image classification method based on the causal diffusion model, wherein the purification step includes:

[0012] The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood with respect to the adversarial sample image is calculated, and the adversarial sample image is gradient-ascended to obtain the purified image.

[0013] In the robust image classification method based on the causal diffusion model, the causal information bottleneck is:

[0014] CIB(X,Y,S,Z)=I(X;S,Z)+I(Y;S)-I(S;Z)-λI(X;S,Z)

[0015] Where I(·;·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label;

[0016] The causal information bottleneck is maximized by increasing I(X; S, Z). At the same time, the structural causal model can predict Y based on S, decoupling S and Z, and S and Z only match key semantic information.

[0017] like Figure 5As shown, the present invention also proposes a robust image classification device based on a causal diffusion model, which includes:

[0018] The construction module models the structural causal model through the conditional diffusion model, with causal features and non-causal features as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories;

[0019] A training module obtains an image dataset having multiple sample images with annotated category labels; collects sample images from the image dataset as training samples; extracts causal features and non-causal features of the training samples using the structural causal model, calculates the causal information bottleneck of the causal features and the non-causal features, updates the structural causal model by maximizing the causal information bottleneck, and obtains an intermediate model; uses the intermediate model to generate causal features of the training samples, guides the intermediate model to predict the predicted category of the training samples, constructs a loss function based on the predicted category of the training samples and the category label, trains the intermediate model, and obtains an image classification model;

[0020] A purification module obtains adversarial sample images with labeled target categories and applies the image classification model to purify the received adversarial samples. The image classification model removes adversarial perturbations from the adversarial sample images by maximizing the likelihood of the adversarial sample images to obtain a purified image.

[0021] an extraction module, which extracts causal features of the purified image using the image classification model, and obtains an image classification result of the purified image based on the causal features of the purified image;

[0022] The adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

[0023] The robust image classification device based on the causal diffusion model, wherein the purification module includes:

[0024] The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood with respect to the adversarial sample image is calculated, and the adversarial sample image is gradient-ascended to obtain the purified image.

[0025] In the robust image classification device based on the causal diffusion model, the causal information bottleneck is:

[0026] CIB(X,Y,S,Z)=I(X;S,Z)+I(Y;S)-I(S;Z)-λI(X;S,Z)

[0027] Where I(·;·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label;

[0028] The causal information bottleneck is maximized by increasing I(X; S, Z). At the same time, the structural causal model can predict Y based on S, decoupling S and Z, and S and Z only match key semantic information.

[0029] The present invention also proposes an electronic device, which includes the aforementioned robust image classification device. The electronic device may be connected to an information display device, which is used to display the image classification results using display parameters and attributes set by the user or through an artificial intelligence model.

[0030] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the robust image classification methods when executed by a processor.

[0031] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the robust image classification methods are implemented.

[0032] From the above scheme, it can be seen that the advantages of the present invention are:

[0033] The causal diffusion model proposed in this invention provides an innovative adversarial attack defense mechanism by combining the structural causal model (SCM) and the conditional diffusion model, significantly enhancing the defense against unknown attacks and improving the robustness and generalization ability of the model. The causal diffusion model includes two steps during inference: causal feature decoupling and purification. We also tested the effectiveness of each module when used alone. Compared with existing technologies, this method can effectively reduce dependence on large-scale data sets, optimize the adversarial purification process, and provide higher security and reliability in high-risk application scenarios, especially showing excellent performance when facing complex and changing attack environments.

[0034] Table 1 shows a series of adversarial test results using AutoAttack (AA) on the CIFAR-10 dataset, which evaluate the robustness of different models in the face of advanced adversarial attacks. AutoAttack is an automated adversarial attack tool that integrates multiple attack methods to comprehensively test the model's defense capabilities.

[0035] The attack methods listed in the table include:

[0036] AA Linf (perturbation range is 8 / 255): AutoAttack using the L-infinity norm. This attack limits the maximum perturbation allowed for each pixel to no more than 8 / 255, thereby testing the model's sensitivity to extreme pixel perturbations.

[0037] AA L2 (perturbation range is 0.5): AutoAttack using the L2 norm, which evaluates the sensitivity to global image perturbations (the Euclidean distance of all pixel perturbations does not exceed 0.5).

[0038] StAdv (perturbation range is 8 / 255): Spatially Transformed Adversarial Attack (StAdv) misleads the model by performing subtle geometric deformation on the image to test the model's robustness to spatial transformation.

[0039] The table also lists each model's accuracy when unattacked, its performance under various attacks, and finally provides an average robustness score to reflect the model's overall defense capabilities against all considered attack methods. The model includes a standard causal diffusion model and the individual defense effects of a feature decoupling module and a purification module.

[0040] These results clearly demonstrate the potential of causal diffusion techniques to improve model performance in adversarial environments, particularly their ability to maintain high accuracy and robustness against diverse adversarial attacks. This comprehensive evaluation highlights the causal diffusion model's ability to defend against unseen attack methods. Notably, since the causal diffusion model was trained without any adversarial examples, any attack method tested against it was considered unseen.

[0041] Table 1: Performance test results of the causal diffusion model on the CIFAR10 dataset

[0042]

[0043] Table 2: Performance test results of the causal diffusion model on the GTSRB dataset

[0044]

[0045] Table 2 shows a series of adversarial test results using AutoAttack (AA) on the GTSRB dataset. These results evaluate the robustness of the causal diffusion model in the face of advanced adversarial attacks. AutoAttack is an automated adversarial attack tool that integrates multiple attack methods to comprehensively test the model's defense capabilities.

[0046] The attack methods listed in the table include:

[0047] AA Linf (perturbation range is 8 / 255): AutoAttack using the L-infinity norm. This attack limits the maximum perturbation allowed for each pixel to no more than 8 / 255, thereby testing the model's sensitivity to extreme pixel perturbations.

[0048] AA L2 (perturbation range is 0.5): AutoAttack using the L2 norm, which evaluates the sensitivity to global image perturbations (the Euclidean distance of all pixel perturbations does not exceed 0.5).

[0049] Fog: This test simulates the impact of foggy environments on images and evaluates the robustness of the model under visually obstructed conditions.

[0050] Table 2 also shows the causal diffusion model's high accuracy (97.85%) when not subject to any attack, as well as its performance under various attacks, with an average robustness of 82.62%, reflecting the model's overall defense capabilities across all considered attack methods. These results clearly demonstrate the potential of causal diffusion techniques to improve model performance in adversarial settings, particularly how they help models maintain high accuracy and robustness in the face of diverse adversarial attacks. This comprehensive evaluation emphasizes the causal diffusion model's defense capabilities against unseen attack methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the structural causal model;

[0052] Figure 2 A diagram of the training method for the causal diffusion model;

[0053] Figure 3 Flowchart of the robust inference method for the causal diffusion model;

[0054] Figure 4 Flow chart of the method of the present invention;

[0055] Figure 5 This is a module diagram of the device of the present invention;

[0056] Figure 6 This is a schematic structural diagram of a first electronic device of the present invention;

[0057] Figure 7 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0058] Figure 8 This is a schematic structural diagram of a second electronic device according to the present invention.

[0059] Reference numerals:

[0060] A-First electronic device;

[0061] B-Robust image classification device;

[0062] C-data acquisition equipment;

[0063] D-information display device;

[0064] 1000- second electronic device;

[0065] Ⅰ-computing unit;

[0066] II-ROM;

[0067] III-RAM;

[0068] IV-bus;

[0069] V-interface;

[0070] VI-input unit;

[0071] VII-output unit;

[0072] VIII-Storage medium;

[0073] IX-Communication unit. DETAILED DESCRIPTION

[0074] It should be noted that, in this application, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0075] Without further constraints, an element defined by the phrase "comprises a..." does not preclude the existence of additional identical elements in the process, method, article or apparatus that includes the element.

[0076] The processor described in the present invention is the control center of an electronic device and can be a single processor or a collective term for multiple processing elements. For example, it can be one or more central processing units (CPUs), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0077] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0078] In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smartphones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0079] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0080] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0081] The above embodiments can be implemented in whole or in part through software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired method (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, or magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0082] It should also be understood that the term "and / or" in this document simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " in this document generally indicates an "or" relationship between the related objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0083] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0084] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0085] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0086] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0087] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0088] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0089] During the research process, the inventors found that although existing methods such as adversarial training and purification can improve the defense capabilities of the model against specific attacks, they are not effective when facing unseen attacks or changes in attack intensity. To this end, the inventors proposed a causal diffusion model (CausalDiff), which is a new framework that combines the structural causal model (SCM) and the conditional diffusion model. By identifying and utilizing key causal features from the data generation process, it effectively improves the robustness of the model to various unseen attacks. This method not only reduces the impact of adversarial attacks, but also improves the adaptability and generalization of the model in a dynamic attack environment. In order to achieve the above technical effects, the present invention proposes the following key technical points:

[0090] Key Point 1: A causal feature recognition framework based on a diffusion model. This framework uses a conditional diffusion model to model a structural causal model; performs model initialization and data processing; randomly extracts small batches of samples from a large-scale original dataset; calculates the causal information bottleneck for these small batches using the causal diffusion model; and optimizes and updates the causal diffusion model parameters through loss backpropagation. The resulting causal diffusion model can identify both causal and non-causal features of the input samples.

[0091] Key Point 2: A device that uses the causal diffusion model to improve the adversarial robustness of image classification tasks. During the model inference phase, the causal diffusion model is first used to purify the input image sample. After obtaining the purified sample, the causal diffusion model identifies causal and non-causal features. Based on the causal features, the causal diffusion model classifies them to obtain a predicted category. This device enables the model to maintain high classification accuracy and robustness even in the face of varying attack intensities when performing image classification tasks in adversarial environments.

[0092] To illustrate the above-mentioned features and effects of the present invention more clearly and easily, the following embodiments are specifically described below with reference to the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are for illustrative purposes only. The scope of protection of the present invention is not limited to the disclosed embodiments; the present invention is defined by the appended claims.

[0093] Figure 1 It is a structural causal model, where nodes represent variables and arrows represent causal relationships, in which causal features work together with non-causal features to determine image samples, while only causal features determine the category label.

[0094] Figure 2 This is a training method for the causal diffusion model. This method is implemented through a specific flowchart. The main purpose is to use clean data samples to train the causal diffusion model to learn the data generation mechanism and thus achieve robust image classification. The following are the detailed steps of the process:

[0095] Step 1: Modeling a structural causal model through a conditional diffusion model (S101)

[0096] The causal model generates images and their corresponding categories based on causal features and non-causal features. Specifically, causal features and non-causal features jointly generate images, and causal features generate categories. This invention uses the conditional diffusion model to model Figure 1 The structural causal model shown in Figure 2. In the conditional diffusion model, causal features and non-causal features serve as conditional variables to guide the image generation process of the diffusion model, i.e., modeling Figure 1 The process of generating causal features and non-causal features into images is shown. The present invention uses a neural network to model the classification process of causal features into category labels. This classifier also serves as part of the causal diffusion model.

[0097] Among them, the difference between the causal diffusion model adopted by the present invention and the traditional diffusion model is that the conditional diffusion model adds a feature vector as a condition on the basis of the diffusion model. The present invention adopts the conditional diffusion model to model the conditional probability p(x|s,z), and uses the conditional diffusion model that can obtain the conditional probability p(x|s,z) as the causal diffusion model; wherein x is an image, s is a causal feature, and z is a non-causal feature. Usually, the diffusion model models p(x), so it can generate an image by maximizing p(x). The conditional diffusion model generates an image of corresponding content based on the input conditions, namely the s and z feature vectors, thereby using the feature vector as a condition to control the image generation process. The causal diffusion model of the present invention predicts the category by connecting a discriminator to the feature vector s.

[0098] Step 2: Randomly initialize the causal diffusion model (S102)

[0099] The causal diffusion model is randomly initialized to set initial parameters. This includes the initial random assignment of weights and biases, setting a starting point for the model training process. This step is crucial for subsequent learning efficiency and the model's ultimate performance.

[0100] The weights and biases are neural network model parameters, which are used to model the causal diffusion model. The training process is the process of updating these parameters. Initialization is to randomly initialize the parameters, that is, sampling values ​​from a Gaussian distribution as the initial values ​​of the parameters.

[0101] Step 3: Randomly extract small batch samples from the large-scale original dataset (S103)

[0102] Mini-batches are randomly sampled from a large dataset. This involves randomly selecting data from the entire dataset according to a set of rules for use in a single training iteration. Mini-batch training helps improve the generalization ability of neural network models and reduces memory consumption during training.

[0103] A dataset consists of a certain number of images and their corresponding category labels. For example, the CIFAR-10 dataset contains 50,000 training sample images and their category labels. The category labels are divided into ten categories: airplane, car, bird, cat, deer, dog, frog, horse, ship, and truck.

[0104] Step 4: Calculate the causal information bottleneck lower bound (S104)

[0105] The extracted small batch samples are input into the Causal Information Bottleneck (CIB) calculation module. The CIB module is a training target derived from the structural causal model. The structural causal model is shown in the attached Figure 1 As shown in Figure 2, this CIB is a training target derived from the structural causal model and is a module independent of the structural causal model. In this module, maximizing the causal information bottleneck optimizes the model parameters so that it can remove irrelevant non-causal information while retaining important causal features. The causal information bottleneck is defined as

[0106] CIB(X,Y,S,Z)=I(X;S,Z)+I(Y;S)-I(S;Z)-λI(X;S,Z)

[0107] Where I(·;·) represents the mutual information between the two, S represents the causal features extracted by the structural causal model, Z represents the non-causal features extracted by the structural causal model, X represents the image, and Y represents the correct category label of the sample. The causal information bottleneck aims to maximize the mutual information between the causal feature S and the non-causal feature Z with X, while ensuring that S can predict Y and that S and Z are decoupled as much as possible. Specifically, S and Z should not match all the detailed information in X. Instead, they learn only the abstract semantic features of X and, consequently, only match the key semantic information.

[0108] Step 5: Loss back propagation and model parameter update (S105)

[0109] The calculated loss is used to update the parameters of the causal diffusion model through the backpropagation algorithm. This step is a standard process in machine learning, and the model is optimized after multiple iterations by adjusting the parameters in the model to minimize the loss function.

[0110] Figure 3 This is a flowchart of a robust inference method for the causal diffusion model, which aims to predict the category of input samples using the causal diffusion model, especially in the face of adversarial environments or noise interference. The following is a detailed description of the specific implementation method based on the flowchart:

[0111] Step 1: Configure initial conditions and model parameters (S201)

[0112] Before model training, first configure Figure 2 The training process shown here trains the causal diffusion model, which includes setting model parameters such as weights, biases, and network structure to ensure that the model is optimized and ready to receive new input data.

[0113] Step 2: Accept adversarial sample image input (S202)

[0114] The system accepts adversarial examples as input. These images are typically designed to mislead the model into making incorrect predictions or classifications. These examples are received to test and use the causal diffusion model to identify and correct perturbations introduced by adversarial attacks.

[0115] Among them, adversarial samples are those in which the attacker adds adversarial perturbations to the original samples, causing the model to misclassify. To humans, the adversarial perturbations appear as imperceptible noise.

[0116] Step 3: Purification using the causal diffusion model (S203)

[0117] A causal diffusion model is applied to the received adversarial examples to perform purification. This process removes or mitigates adversarial perturbations in the sample by maximizing the sample likelihood through the diffusion model, thereby restoring the original or true characteristics of the sample. The purified sample is expected to be closer to its pre-attack state.

[0118] Among them, the way to maximize the sample likelihood is: use the diffusion model to estimate the sample likelihood, which is an existing method; then backpropagate the sample likelihood, calculate the gradient of the sample likelihood with respect to the image sample, and perform gradient ascent on the image to maximize the sample likelihood of the image.

[0119] The diffusion model calculates likelihood by estimating the image likelihood using a lower bound on the evidence, a common approach used by diffusion models. Since adversarial examples generally have lower likelihoods than clean examples, the sample likelihood represents the probability that the diffusion model generated the image. A higher probability, or likelihood, indicates that the image is more likely to be a clean sample than an adversarial one. Therefore, adversarial examples can be purified by maximizing the sample likelihood. The result of purification is a purified image sample, not a feature.

[0120] Step 4: Purify samples using causal diffusion model reasoning (S204)

[0121] Further reasoning analysis is performed on the cleansed samples, using the causal diffusion model to extract key causal and non-causal features. This step is performed using the model's causal reasoning capabilities to identify which features are key factors in determining the sample's category.

[0122] Step 5: Use the causal diffusion model to make category predictions based on causal features (S205)

[0123] Based on the causal features extracted from the previous step, the causal diffusion model is used to make the final category prediction. This step uses causal features for category prediction while ignoring the influence of non-causal features. As a result, any adversarial perturbations that may remain in S203 will also be treated as non-causal features and will not affect the category prediction results, thereby improving the accuracy of the prediction and the model's ability to resist adversarial attacks.

[0124] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiment.

[0125] like Figure 5 As shown, the present invention also proposes a robust image classification device based on a causal diffusion model, which includes:

[0126] The construction module models the structural causal model through the conditional diffusion model, with causal features and non-causal features as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories;

[0127] A training module obtains an image dataset having multiple sample images with annotated category labels; collects sample images from the image dataset as training samples; extracts causal features and non-causal features of the training samples using the structural causal model, calculates the causal information bottleneck of the causal features and the non-causal features, updates the structural causal model by maximizing the causal information bottleneck, and obtains an intermediate model; uses the intermediate model to generate causal features of the training samples, guides the intermediate model to predict the predicted category of the training samples, constructs a loss function based on the predicted category of the training samples and the category label, trains the intermediate model, and obtains an image classification model;

[0128] A purification module obtains adversarial sample images with labeled target categories and applies the image classification model to purify the received adversarial samples. The image classification model removes adversarial perturbations from the adversarial sample images by maximizing the likelihood of the adversarial sample images to obtain a purified image.

[0129] an extraction module, which extracts causal features of the purified image using the image classification model, and obtains an image classification result of the purified image based on the causal features of the purified image;

[0130] The adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

[0131] The robust image classification device based on the causal diffusion model, wherein the purification module includes:

[0132] The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood with respect to the adversarial sample image is calculated, and the adversarial sample image is gradient-ascended to obtain the purified image.

[0133] In the robust image classification device based on the causal diffusion model, the causal information bottleneck is:

[0134] CIB(X,Y,S,Z)=I(X;S,Z)+I(Y;S)-I(S;Z)-λI(X;S,Z)

[0135] Where I(·;·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label;

[0136] The causal information bottleneck is maximized by increasing I(X; S, Z). At the same time, the structural causal model can predict Y based on S, decoupling S and Z, and S and Z only match key semantic information.

[0137] like Figure 6 As shown, the present invention further proposes a first electronic device A in another embodiment, including the above-mentioned robust image classification device, and the electronic device may be connected to an information display device.

[0138] like Figure 7 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to collect and obtain images to be identified and classified, and the information display device D is used to display the classification results obtained by the analysis of the present invention.

[0139] The information display device D can organize and process the data output by the first electronic device A based on the information display mechanism to improve the readability of the data output by the first electronic device A. The information display mechanism can be manually preset, for example, the data output by the first electronic device A is visually displayed, which can be based on the display parameters and / or attributes set by the user. The display parameters can be, for example, the display data range, and the display attributes can be, for example, the display font, color, whether to scroll, etc. The user is presented with the key information specified by the user, for example, the categories of special attention are displayed in bold, etc. The user can understand this information more promptly without having to visit the secondary page or scroll the page, saving the user's operation. Or the information display mechanism can be an artificial intelligence AI display model, which can learn the user's key information based on the user's previous usage habits, such as viewing time, number of clicks, number of edits, etc., and then automatically present the user with rich and necessary key information.

[0140] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a readable storage medium. When the computer program is executed by a processor, the computer can execute the robust image classification method provided by the above methods.

[0141] In another embodiment, the present invention further proposes a storage medium VIII for storing a computer program for executing the robust image classification method. It should be understood that the storage medium in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0142] Figure 8 A schematic block diagram of a second electronic device 1000 that can be used to implement an embodiment of the present invention is shown. The second electronic device 1000 electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein. The second electronic device 1000 may be the same as or different from the first electronic device A.

[0143] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from a storage medium VIII into a random access memory (RAM) III. Various programs and data required for the operation of the device 1000 can also be stored in the RAM III. The computing unit I, ROM II, and RAM III are connected to each other via a bus IV. An input / output (I / O) interface V is also connected to the bus IV.

[0144] Multiple components in the second electronic device 1000 are connected to the I / O interface V, including: an input unit VI, such as a keyboard and mouse; an output unit VII, such as various types of displays and speakers; a storage medium VIII, such as a magnetic disk and optical disk; and a communication unit IX, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit IX allows the second electronic device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0145] Computing unit I can be various general and / or special processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. Computing unit I performs the various methods and processes described above, such as method steps S1-S4. For example, in some embodiments, the method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage medium VIII. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 1000 via ROM II and / or communication unit IX. When the computer program is loaded into RAM III and executed by computing unit I, one or more steps of the method described above can be performed. Alternatively, in other embodiments, computing unit I can be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0146] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the description and implementation methods. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A robust image classification method based on causal diffusion model, characterized in that: include: The construction step is to model a structural causal model through a conditional diffusion model, wherein causal features and non-causal features are used as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories; A training step includes obtaining an image dataset having a plurality of sample images with annotated category labels; collecting sample images as training samples from the image dataset; extracting causal features and non-causal features of the training samples through the structural causal model, and calculating the causal information bottleneck of the causal features and the non-causal features, and updating the structural causal model by maximizing the causal information bottleneck to obtain an intermediate model; using the intermediate model to generate the causal features of the training samples, guiding the intermediate model to predict the predicted category of the training samples, and constructing a loss function based on the predicted category and the category label of the training samples, training the intermediate model, and obtaining an image classification model; The purification step obtains an adversarial sample image with a labeled target category and applies the image classification model to purify the received adversarial sample. The image classification model removes the adversarial perturbation in the adversarial sample image by maximizing the likelihood of the adversarial sample image to obtain a purified image. The extraction step uses the image classification model to extract the causal features of the purified image, and obtains an image classification result of the purified image based on the causal features of the purified image.

2. The robust image classification method based on the causal diffusion model according to claim 1, characterized in that: The adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

3. The robust image classification method based on the causal diffusion model according to claim 1, characterized in that: The purification steps include: The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood with respect to the adversarial sample image is calculated, and the adversarial sample image is gradient-ascended to obtain the purified image.

4. The robust image classification method based on the causal diffusion model according to claim 1, wherein: The causal information bottleneck is: Where I(·;·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label; Maximizing this causal information bottleneck is achieved by increasing At the same time, the structural causal model can predict Y based on S, decouple S and Z, and S and Z only match key semantic information.

5. A robust image classification device based on a causal diffusion model, characterized in that: include: The construction module models the structural causal model through the conditional diffusion model, with causal features and non-causal features as conditional variables to guide the structural causal model to generate images, and the causal features guide the structural causal model to generate image categories; A training module obtains an image dataset having multiple sample images with annotated category labels; collects sample images from the image dataset as training samples; extracts causal features and non-causal features of the training samples using the structural causal model, calculates the causal information bottleneck of the causal features and the non-causal features, updates the structural causal model by maximizing the causal information bottleneck, and obtains an intermediate model; uses the intermediate model to generate causal features of the training samples, guides the intermediate model to predict the predicted category of the training samples, constructs a loss function based on the predicted category of the training samples and the category label, trains the intermediate model, and obtains an image classification model; A purification module obtains adversarial sample images with labeled target categories and applies the image classification model to purify the received adversarial samples. The image classification model removes adversarial perturbations from the adversarial sample images by maximizing the likelihood of the adversarial sample images to obtain a purified image. an extraction module, which extracts causal features of the purified image using the image classification model, and obtains an image classification result of the purified image based on the causal features of the purified image; The adversarial sample is obtained by adding adversarial perturbation noise to the sample image.

6. The robust image classification device based on the causal diffusion model according to claim 5, characterized in that: The purification module includes: The sample likelihood of the adversarial sample image is obtained through the diffusion model of the image classification model, the sample likelihood is back-propagated, the gradient of the sample likelihood with respect to the adversarial sample image is calculated, and the adversarial sample image is gradient-ascended to obtain the purified image.

7. The robust image classification device based on the causal diffusion model according to claim 5, characterized in that: The causal information bottleneck is: Where I(·;·) represents the mutual information between the two, S represents the causal feature, Z represents the non-causal feature, X represents the training sample, and Y represents the category label; Maximizing this causal information bottleneck is achieved by increasing At the same time, the structural causal model can predict Y based on S, decouple S and Z, and S and Z only match key semantic information.

8. An electronic device, characterized in that: It includes a robust image classification device as described in claims 5-7, and the electronic device is connected to an information display device, which is used to display the image classification results using display parameters and attributes set by the user or through an artificial intelligence model.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the robust image classification method according to any one of claims 1 to 4 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the robust image classification method according to any one of claims 1 to 4 are implemented.

Images