A method, apparatus, electronic device, and medium for image classification adversarial example defense.

Abstract

This invention discloses a method, apparatus, electronic device, and medium for image classification adversarial example defense. The method includes the following steps: acquiring an input image; constructing a multi-scale latent noise image based on the input image; constructing and training an image reconstruction model; inputting the multi-scale latent noise image into the trained image reconstruction model to obtain a clean image; constructing a fine-tuning dataset based on the clean image; fine-tuning the classifier based on the fine-tuning dataset; and using the fine-tuned classifier to perform classification. This invention proposes a multi-scale latent noise image construction strategy that can effectively destroy the adversarial perturbation from a multi-scale perspective using traditional methods before the adversarial example is input into the reconstruction model, solving the problem that relying solely on deep neural network models cannot effectively destroy adversarial perturbations. This invention can be widely applied in the fields of autonomous driving, face recognition, and medical image analysis.

Description

Technical Field

[0001] This invention relates to adversarial sample defense technology, and more particularly to an image classification adversarial sample defense method, apparatus, electronic device, and medium. Background Technology

[0002] Despite the success of deep neural networks in various pattern recognition tasks, they exhibit vulnerability to adversarial examples. Attackers can easily mislead trained deep neural network models into making incorrect decisions by introducing carefully crafted, imperceptible adversarial perturbations into clean images. The existence of adversarial examples poses a significant threat to the deployment of deep neural networks in security-critical domains such as autonomous driving, facial recognition, and medical image analysis. Therefore, a high-performance adversarial defense method is urgently needed to ensure the security and robustness of neural network models.

[0003] Existing adversarial example defense methods can be broadly categorized into three types: adversarial robustness, adversarial detection, and adversarial elimination. Adversarial robustness improves model robustness through strategies such as adversarial training, adversarial distillation, and optimization of model structure and parameters. However, these methods are not effective against large-scale data (such as ImageNet) and face high deployment costs. Adversarial detection methods attempt to identify adversarial examples by training a detector. However, these methods often struggle to generalize to unknown attacks, and in some scenarios (such as autonomous driving), simply detecting adversarial examples is insufficient; adversarial perturbations must also be eliminated to enable the model to make correct decisions.

[0004] Compared to the previous two types of methods, adversarial elimination eliminates perturbations through transformations or deep neural network models, without requiring model retraining or optimization, thus offering better scalability and application potential. However, existing adversarial elimination methods suffer from two problems: 1) a trade-off between adversarial robustness and clean accuracy. Existing methods face a fundamental dilemma: effectively eliminating adversarial perturbations often impacts performance on clean images; conversely, maintaining the quality of clean images often preserves more adversarial perturbations. 2) poor defensive robustness. Existing methods experience a 40%-100% performance drop when facing defense-aware attacks (i.e., possessing all knowledge of the target model and its defense mechanisms). Therefore, achieving superior adversarial robustness while maintaining excellent clean accuracy remains an unsolved problem in the field of adversarial defense. Summary of the Invention

[0005] In order to at least partially solve one of the technical problems existing in the prior art, the present invention aims to provide an image classification adversarial sample defense method, device, electronic device and medium based on latent noise image reconstruction.

[0006] The first technical solution adopted in this invention is:

[0007] An image classification adversarial example defense method includes the following steps:

[0008] Acquire the input image and construct a multi-scale latent noise image based on the input image;

[0009] A model for image reconstruction is constructed and trained. Multi-scale noisy images are input into the trained model to obtain clean images.

[0010] A fine-tuned dataset is constructed based on clean images. The classifier is then fine-tuned based on the fine-tuned dataset, and the fine-tuned classifier is used to perform classification.

[0011] Further, the step of acquiring the input image and constructing a multi-scale latent noise image based on the input image includes:

[0012] The input image is transformed to three different scales through resizing operations to obtain large-scale, medium-scale, and small-scale images.

[0013] Large-scale, medium-scale, and small-scale images are divided into image patches of the same size and flattened into D-dimensional column vectors, denoted as x. p p = l, m, s;

[0014] Based on the image patches segmented from large-scale, medium-scale, and small-scale images, three PCA bases are calculated, denoted as V. p p = l, m, s;

[0015] Patch-level PCA is used to map image patches to the latent space, Gaussian noise is introduced into the latent space, and these noisy latent space features are mapped back to the pixel space through inverse PCA to construct a multi-scale latent noise image.

[0016] Furthermore, the expression for the multi-scale latent noise image is as follows:

[0017]

[0018] In the formula, σ represents the noise scale, ∈ represents Gaussian noise, and T represents the transpose.

[0019] Furthermore, the image reconstruction model is a reconstruction model with cross-scale feature interaction, and the image reconstruction model includes a cross-scale feature fusion encoder and a decoder;

[0020] The cross-scale feature fusion encoder is used to extract features from images at different scales and fuse the extracted features to obtain fused features.

[0021] The decoder is used to reconstruct a high-quality, clean image based on the fusion features;

[0022] The clean image retains the essential structural information (robust features) of the image, while discarding some non-robust features that are sensitive to perturbations (including adversarial perturbations).

[0023] Furthermore, the expression for the cross-scale feature interaction is:

[0024]

[0025]

[0026] In the formula, k∈{0,1,2,3}, and Layer represents large-scale, medium-scale, and small-scale embeddings, respectively. k This represents the k-th Swing Transformer block in the cross-scale feature fusion encoder.

[0027] Furthermore, the decoder consists of a 1×1 convolutional layer and a pixel shuffle layer.

[0028] Furthermore, the image reconstruction model also includes a random layer, which is set after the decoder. The random layer is used to resize the clean image reconstructed by the decoder and obtain an image of a preset size by padding.

[0029] The second technical solution adopted in this invention is:

[0030] An image classification adversarial example defense device, comprising:

[0031] The image noise-adding module is used to acquire the input image and construct a multi-scale latent noise image based on the input image;

[0032] The image reconstruction module is used to build and train an image reconstruction model. It inputs multi-scale noisy images into the trained image reconstruction model to obtain clean images.

[0033] The classifier fine-tuning module is used to construct a fine-tuning dataset based on clean images, fine-tune the classifier based on the fine-tuning dataset, and use the fine-tuned classifier to perform classification.

[0034] The third technical solution adopted in this invention is:

[0035] An electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement an image classification adversarial example defense method as described above.

[0036] The fourth technical solution adopted in this invention is:

[0037] A computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement an image classification adversarial example defense method as described above.

[0038] The fifth technical solution adopted in this invention is:

[0039] A computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions to cause the computer device to perform the method described above.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] (1) This invention proposes a multi-scale hidden noise image construction strategy, which can effectively destroy the adversarial perturbation from a multi-scale perspective using traditional methods before the adversarial sample is input into the reconstruction model, thus solving the problem that deep neural network models alone cannot effectively destroy adversarial perturbations.

[0042] (2) This invention proposes a deep neural network reconstruction model with cross-scale feature interaction, which makes the reconstruction model have stronger modeling ability and can better reconstruct clean images from multi-scale hidden noise images. Combining multi-scale hidden noise image construction strategy can significantly improve the robustness of adversarial defense.

[0043] (3) A method is proposed to fine-tune the classifier using the clean dataset reconstructed from the latent noise image, so that the classifier can better adapt to the data distribution after reconstruction, while improving adversarial robustness and clean accuracy. This solves the trade-off between adversarial robustness and clean accuracy faced by previous adversarial elimination-based methods. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of the image classification adversarial sample defense method based on latent noise image reconstruction proposed in this embodiment of the invention;

[0046] Figure 2 These are clean samples and adversarial samples and their corresponding hidden noise images in the embodiments of this invention;

[0047] Figure 3 This is a schematic diagram illustrating the principle of the multi-scale latent noise image construction strategy in this embodiment of the invention.

[0048] Figure 4 This is a structural diagram of the cross-scale feature fusion encoder in the reconstruction model of this invention embodiment;

[0049] Figure 5 This is a structural diagram of the decoder in the reconstruction model of this invention embodiment;

[0050] Figure 6 This is a schematic diagram of random layer processing in an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of the fine-tuning strategy in an embodiment of the present invention. Detailed Implementation

[0052] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0053] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0054] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0055] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0056] To address the existing technical problems, this invention proposes an image classification adversarial example defense scheme based on latent noise image reconstruction. Combining traditional principal component analysis and deep learning methods, it eliminates the effect of adversarial attacks from a multi-scale perspective, restores adversarial examples to a clean data distribution, and solves the trade-off between adversarial robustness and clean accuracy through a simple and effective fine-tuning strategy.

[0057] Example 1

[0058] like Figure 1 As shown, this embodiment provides an image classification adversarial example defense method based on latent noise image reconstruction. Essentially, it is an adversarial example defense method for image classification that combines traditional methods and deep learning methods. The specific steps are as follows:

[0059] S1. Obtain the input image (clean image / adversarial image) and construct a multi-scale hidden noise image from the input image.

[0060] In some embodiments, step S1 designs a multi-scale latent noise image construction strategy, using the plot-level principal component analysis (PCA) method to construct a multi-scale latent noise image from the input image (clean image / adversarial image).

[0061] In particular, the latent noise images constructed from clean samples and adversarial samples in step S1 have similar characteristics, such as... Figure 2 As shown, Figure 2 In the middle (a), there is a clean sample and its corresponding hidden noise image. Figure 2 In (b), we see adversarial samples and their corresponding latent noise images. This allows us to construct latent noise images using only clean samples and train the reconstruction model in step S2 using a self-supervised method (the reconstruction model is essentially a denoising autoencoder).

[0062] S2. Construct and train an image reconstruction model. Input multi-scale hidden noise images into the trained image reconstruction model to obtain clean images.

[0063] In some embodiments, step S2 designs a deep neural network image reconstruction model with cross-scale feature interaction. Using the trained image reconstruction model, the multi-scale latent noise image obtained in step S1 is reconstructed to restore the image to the distribution of clean data.

[0064] S3. Construct a fine-tuned dataset based on clean images, fine-tune the classifier based on the fine-tuned dataset, and use the fine-tuned classifier to perform classification.

[0065] In some embodiments, step S3 proposes a simple and effective fine-tuning strategy based on steps S1 and S2. Steps S1 and S2 are used to perform latent noise image reconstruction on the clean training set to obtain a fine-tuning dataset for fine-tuning the classifier.

[0066] The multi-scale latent noise image construction strategy proposed in this embodiment effectively destroys the adversarial perturbation effect from multiple scales. Combined with a deep neural network image reconstruction model, adversarial examples can be restored to a clean data distribution. By combining traditional methods with deep learning methods, the robustness of adversarial defense is significantly improved. Furthermore, the strategy proposed in steps S1 and S2 for fine-tuning the classifier using the clean dataset reconstructed from the latent noise image significantly improves clean accuracy while further enhancing robust accuracy, effectively solving the trade-off between adversarial robustness and clean accuracy. Therefore, the image classification adversarial example defense based on latent noise image reconstruction proposed in this embodiment enables the protected classifier to achieve superior adversarial robustness while maintaining excellent clean accuracy.

[0067] The above method will be explained in detail below with reference to the accompanying drawings and specific embodiments.

[0068] Figure 1 This paper presents a flowchart of an image classification adversarial example defense method based on latent noise image reconstruction. First, we design a multi-scale latent noise image construction strategy based on tile-level PCA to construct latent noise images at multiple scales, thus disrupting the adversarial perturbation from a multi-scale perspective. Next, we propose a reconstruction model with cross-scale feature interactions to reconstruct the latent noise image back to a clean data distribution. Finally, we fine-tune the classifier using the clean dataset reconstructed from the latent noise image, effectively addressing the trade-off between adversarial robustness and clean accuracy faced by previous adversarial elimination-based methods.

[0069] (a) Construction of multi-scale hidden noise images

[0070] To more effectively counteract the effects of adversarial perturbations, we designed a multi-scale latent noise image construction strategy based on tile-level PCA, the principle of which is as follows: Figure 3 As shown.

[0071] Given an image to be input into the classifier, it is first transformed to three different scales through a resizing operation: large scale, medium scale, and small scale. The large-scale, medium-scale, and small-scale images are then divided into image patches of the same size and flattened into D-dimensional column vectors, denoted as follows: Since image patches of the same size segmented from images at different scales have very different receptive fields, we calculated three PCA bases using these image patches segmented from images at different scales, denoted as . Where d is the dimension of the latent space.

[0072] Next, we use patch-level PCA to map image patches to the latent space, introduce Gaussian noise into the latent space, and then use inverse PCA to map these noisy latent space features back to the pixel space, thereby constructing a multi-scale latent noise image. The construction process can be represented by the following formula:

[0073]

[0074] Where σ represents the noise scale, used to control the degradation level of the image with hidden noise. For example... Figure 3 As shown on the right, large-scale image patches have smaller receptive fields, preserving more details; while small-scale image patches have larger receptive fields, capturing more global features. In other words, multi-scale latent noise images contain image information with different granularities and degrees of degradation, which may be beneficial for reconstruction models to learn the intrinsic patterns of clean images. For large-scale images (such as ImageNet), we used multi-scale image resolutions of 512, 256, and 128, with a patch size of 16; for small-scale images (such as CIFAR-10), we used resolutions of 64, 32, and 16, with a patch size of 8. The latent space size was set to 16 for all cases.

[0075] (b) Reconstruction model with cross-scale feature interactions

[0076] After adversarial examples are constructed into latent noise images, the adversarial perturbation is largely destroyed, but the image itself is also degraded. Therefore, we propose a reconstruction model with powerful modeling capabilities based on the Swin Transformer to reconstruct clean images from multi-scale latent noise images, thereby restoring adversarial examples to cleanly distributed defensive targets. The reconstruction model is essentially a denoising autoencoder (DAE), consisting of a cross-scale feature fusion encoder and a simple decoder.

[0077] 1) Cross-scale feature fusion encoder

[0078] Cross-scale feature fusion encoder structure as follows Figure 4 As shown. To enhance the modeling ability, we designed a cross-scale feature interaction strategy. For large-scale, medium-scale, and small-scale inputs, we first use embedding layers to embed the multi-scale inputs into a suitable dimension, using downsampling of 8×, 4×, and 2× respectively. The cross-scale feature interaction can be expressed by the following formula:

[0079]

[0080] Where k∈{0,1,2,3}, and Layer represents large-scale, medium-scale, and small-scale embeddings, respectively. k This represents the k-th Swing Transformer block in the encoder. The final output of the encoder is the fused high-level feature. It is fed into the decoder for reconstruction.

[0081] 2) Simple decoder

[0082] In contrast to most traditional decoders that employ complex structures, we have used a very simple decoder. For example... Figure 5 As shown, the decoder consists of only a 1×1 convolutional layer and a pixel shuffle layer. Thanks to the powerful feature extraction capabilities of the cross-scale feature fusion encoder, such an extremely simple decoder is sufficient to reconstruct high-quality, clean images.

[0083] 3) Random layer

[0084] To further improve the model's robustness against attacks, a random layer is added after model reconstruction during inference. This random layer first randomly adjusts the reconstructed image size to [299, 331), then pads it with zeros to a size of 331×331. The processing of the random layer is as follows: Figure 6 As shown.

[0085] (c) A simple and effective fine-tuning strategy

[0086] Images reconstructed from noisy images exhibit the characteristic of retaining essential structural information (robust features) while losing some non-robust features sensitive to perturbations (including adversarial perturbations). Since the original classifier may also utilize these discarded non-robust features in its decision-making, using the original classifier to predict these reconstructed images could lead to a performance degradation. Therefore, we propose a simple yet effective fine-tuning strategy. Specifically, we introduce latent noise into the clean training set. After reconstructing the noisy images, we construct a fine-tuning dataset. This fine-tuning dataset, entirely derived from clean data, is used to fine-tune the classifier, enabling it to better adapt to the reconstructed data distribution. This improves both clean accuracy and robust accuracy, effectively addressing the trade-off between adversarial robustness and clean accuracy faced by existing adversarial elimination-based methods. The principle of the fine-tuning strategy is as follows: Figure 7 As shown.

[0087] Example 2

[0088] This embodiment provides an image classification adversarial example defense device, including:

[0089] The image noise-adding module is used to acquire the input image and construct a multi-scale latent noise image based on the input image;

[0090] The image reconstruction module is used to build and train an image reconstruction model. It inputs multi-scale noisy images into the trained image reconstruction model to obtain clean images.

[0091] The classifier fine-tuning module is used to construct a fine-tuning dataset based on clean images, fine-tune the classifier based on the fine-tuning dataset, and use the fine-tuned classifier to perform classification.

[0092] Since this device is an image classification adversarial sample defense device according to an embodiment of the present invention, and the principle of the device in solving the problem is similar to that of the method, the implementation of this device can refer to the implementation process of the above method embodiment, and the repeated parts will not be described again.

[0093] Example 3

[0094] This invention also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to achieve the following: Figure 1 This paper presents an image classification adversarial sample defense method based on latent noise image reconstruction.

[0095] It is understood that the memory may include random access memory (RAM) or read-only memory. Optionally, the memory may include non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the stored data area may store data created according to the use of the server, etc.

[0096] A processor may include one or more processing cores. The processor connects to various parts of the server via various interfaces and lines, executing instructions, programs, code sets, or instruction sets stored in memory, and accessing data stored in memory to perform various server functions and process data. Optionally, the processor may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor may integrate one or more of the following: Central Processing Unit (CPU) and Modem. The CPU primarily handles the operating system and applications; the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor.

[0097] Since this electronic device is the electronic device corresponding to the image classification adversarial sample defense method based on latent noise image reconstruction in this embodiment of the invention, and the principle of solving the problem by this electronic device is similar to that of this method, the implementation of this electronic device can refer to the implementation process of the above method embodiment, and the repeated parts will not be described again.

[0098] Example 4

[0099] This invention also provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to achieve the following: Figure 1This paper presents an image classification adversarial sample defense method based on latent noise image reconstruction.

[0100] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0101] Since this storage medium is the storage medium corresponding to the image classification adversarial sample defense method based on latent noise image reconstruction in the embodiments of the present invention, and the principle of the storage medium in solving the problem is similar to that of the method, the implementation of this storage medium can refer to the implementation process of the above method embodiments, and the repeated parts will not be described again.

[0102] Example 5

[0103] In some possible implementations, various aspects of the methods of the embodiments of the present invention can also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of an image classification adversarial example defense method based on latent noise image reconstruction according to various exemplary embodiments of this application as described above. The executable computer program code or "code" for performing the various embodiments can be written in high-level programming languages ​​such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0104] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0105] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0106] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for defending against adversarial examples in image classification, characterized in that, Includes the following steps: Acquire the input image and construct a multi-scale latent noise image based on the input image; A model for image reconstruction is constructed and trained. Multi-scale images with hidden noise are input into the trained model to obtain clean images. A fine-tuned dataset is constructed based on clean images. The classifier is then fine-tuned based on the fine-tuned dataset, and the fine-tuned classifier is used to perform classification. The process of acquiring the input image and constructing a multi-scale latent noise image based on the input image includes: The input image is transformed to three different scales through resizing operations to obtain large-scale, medium-scale, and small-scale images. Large-scale, medium-scale, and small-scale images are divided into image patches of the same size and flattened. D A column vector of dimension, denoted as , ; Based on image patches segmented from large-scale, medium-scale, and small-scale images, three PCA bases are calculated, denoted as follows: , ; Patch-level PCA is used to map image patches to the latent space, Gaussian noise is introduced into the latent space, and these noisy latent space features are mapped back to the pixel space through inverse PCA to construct a multi-scale latent noise image. The image reconstruction model is a reconstruction model with cross-scale feature interaction, and the image reconstruction model includes a cross-scale feature fusion encoder and a decoder; The cross-scale feature fusion encoder is used to extract features from images at different scales and fuse the extracted features to obtain fused features. The decoder is used to reconstruct a clean image based on the fusion features; The clean image retains the essential structural information of the image while discarding some non-robust features that are sensitive to disturbances. The expression for the cross-scale feature interaction is: In the formula, , ,and These represent large-scale, medium-scale, and small-scale embeddings, respectively. In the cross-scale feature fusion encoder, the first... A Swing Transformer block.

2. The image classification adversarial example defense method according to claim 1, characterized in that, The expression for the multi-scale latent noise image is as follows: In the formula, Represents the noise level. It is Gaussian noise. This indicates transpose.

3. The image classification adversarial example defense method according to claim 1, characterized in that, The decoder consists of a 1×1 convolutional layer and a pixel shuffle layer.

4. The image classification adversarial example defense method according to claim 1, characterized in that, The image reconstruction model also includes a random layer, which is set after the decoder. The random layer is used to resize the clean image reconstructed by the decoder and obtain an image of a preset size by padding.

5. An image classification adversarial example defense device, characterized in that, include: The image noise-adding module is used to acquire the input image and construct a multi-scale latent noise image based on the input image; The image reconstruction module is used to build and train an image reconstruction model. It inputs multi-scale noisy images into the trained image reconstruction model to obtain clean images. The classifier fine-tuning module is used to construct a fine-tuning dataset based on clean images, fine-tune the classifier based on the fine-tuning dataset, and use the fine-tuned classifier to perform classification. The process of acquiring the input image and constructing a multi-scale latent noise image based on the input image includes: The input image is transformed to three different scales through resizing operations to obtain large-scale, medium-scale, and small-scale images. Large-scale, medium-scale, and small-scale images are divided into image patches of the same size and flattened. D A column vector of dimension, denoted as , ; Based on image patches segmented from large-scale, medium-scale, and small-scale images, three PCA bases are calculated, denoted as follows: , ; Patch-level PCA is used to map image patches to the latent space, Gaussian noise is introduced into the latent space, and these noisy latent space features are mapped back to the pixel space through inverse PCA to construct a multi-scale latent noise image. The image reconstruction model is a reconstruction model with cross-scale feature interaction, and the image reconstruction model includes a cross-scale feature fusion encoder and a decoder; The cross-scale feature fusion encoder is used to extract features from images at different scales and fuse the extracted features to obtain fused features. The decoder is used to reconstruct a clean image based on the fusion features; The clean image retains the essential structural information of the image while discarding some non-robust features that are sensitive to disturbances. The expression for the cross-scale feature interaction is: In the formula, , ,and These represent large-scale, medium-scale, and small-scale embeddings, respectively. In the cross-scale feature fusion encoder, the first... A Swing Transformer block.

6. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method as described in any one of claims 1 to 4.

Images