Image classification confrontation sample defense method and device, electronic equipment and medium

Abstract

The invention discloses an image classification confrontation sample defense method and device, electronic equipment and a medium, and the method comprises the following steps: obtaining an input image, and constructing and obtaining a multi-scale hidden noise image according to the input image; constructing and training an image reconstruction model, and inputting the multi-scale implicit noise image into the trained image reconstruction model to obtain a clean image; and constructing a fine adjustment data set according to the clean image, performing fine adjustment on the classifier according to the fine adjustment data set, and realizing classification by adopting the classifier after fine adjustment. The invention provides a multi-scale implicit noise image construction strategy, which can effectively destroy the effect of adversarial disturbance from a multi-scale perspective by using a traditional method before adversarial samples are input into a reconstruction model, and solves the problem that the adversarial disturbance cannot be effectively destroyed by independently relying on a deep neural network model. The method can be widely applied to the technical fields of automatic driving, face recognition, medical image analysis and the like.

Description

Technical Field

[0001] The present invention relates to adversarial sample defense technology, and in particular to an image classification adversarial sample defense method, device, electronic equipment and medium. Background Art

[0002] Despite their success in various pattern recognition tasks, deep neural networks are vulnerable to adversarial examples. Adversarial examples generated by attackers introducing carefully crafted, imperceptible adversarial perturbations into clean images can easily mislead trained deep neural network models into making incorrect decisions. The existence of adversarial examples poses a significant threat to the deployment of deep neural networks in safety-critical areas such as autonomous driving, face recognition, and medical image analysis. Therefore, a high-performance adversarial defense method is urgently needed to ensure the security and robustness of neural network models.

[0003] Existing adversarial sample defense methods can be roughly divided into three categories: adversarial robustness, adversarial detection, and adversarial elimination. Adversarial robustness retrains the model to improve its adversarial robustness through strategies such as adversarial training, adversarial distillation, model structure and parameter optimization. However, this type of method is not effective in defending against large-scale data (such as ImageNet) and faces high deployment costs. Adversarial detection methods attempt to identify adversarial samples by training a detector. However, this type of method is often difficult to generalize to unknown attacks, and in some scenarios (such as autonomous driving), it is not enough to simply detect adversarial samples. It is also necessary to eliminate adversarial perturbations so that the model can make correct decisions.

[0004] Compared with the first two methods, adversarial elimination eliminates perturbations through some transformations or deep neural network models. It does not require retraining or optimization of the model and has better scalability and application potential. However, there are two problems with existing adversarial elimination methods: 1) The trade-off between adversarial robustness and clean accuracy. Existing methods face a fundamental dilemma: if adversarial perturbations can be effectively eliminated, the performance on clean images will often be affected; if the quality of clean images can be maintained, more adversarial perturbations will often be retained. 2) Poor defense robustness. Existing methods will experience a 40%-100% performance drop when facing defense-aware attacks (i.e., having all the knowledge of the target model and its defense mechanisms at the same time). Therefore, how to achieve superior adversarial robustness while maintaining excellent clean accuracy remains an unsolved problem in the field of adversarial defense. Summary of the invention

[0005] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide an image classification adversarial sample defense method, device, electronic device and medium based on hidden noise image reconstruction.

[0006] The first technical solution adopted by the present invention is:

[0007] A method for defending against adversarial samples in image classification, comprising the following steps:

[0008] Obtain an input image, and construct a multi-scale hidden noise image according to the input image;

[0009] Build and train an image reconstruction model, input the multi-scale hidden noise image into the trained image reconstruction model to obtain a clean image;

[0010] A fine-tuning dataset is constructed based on the clean images, the classifier is fine-tuned based on the fine-tuning dataset, and the fine-tuned classifier is used to implement classification.

[0011] Furthermore, the step of acquiring an input image and constructing a multi-scale hidden noise image according to the input image includes:

[0012] The input image is transformed into three different scales through the resizing operation to obtain a large-scale image, a medium-scale image, and a small-scale image;

[0013] The large-scale image, medium-scale image and small-scale image are divided into image blocks of the same size and flattened into a D-dimensional column vector, denoted as x p , p=l,m,s;

[0014] According to the image blocks divided from the large-scale image, medium-scale image and small-scale image, three PCA bases are calculated respectively, denoted as V p , p=l,m,s;

[0015] The image blocks are mapped to the latent space using block-level PCA, Gaussian noise is introduced into the latent space, and these noisy latent space features are mapped back to the pixel space through inverse PCA to construct a multi-scale latent noise image.

[0016] Furthermore, the expression of the multi-scale hidden noise image is as follows:

[0017]

[0018] Where σ represents the noise scale, ∈ is Gaussian noise, and T represents transposition.

[0019] Furthermore, the image reconstruction model is a reconstruction model with cross-scale feature interaction, and the image reconstruction model includes a cross-scale feature fusion encoder and decoder;

[0020] The cross-scale feature fusion encoder is used to extract features of images of different scales, and fuse the extracted features to obtain fused features;

[0021] The decoder is used to reconstruct a high-quality clean image based on the fused features;

[0022] The clean image retains the essential structural information (robust features) of the image, while discarding some non-robust features that are sensitive to disturbances (including adversarial disturbances).

[0023] Furthermore, the expression of the cross-scale feature interaction is:

[0024]

[0025]

[0026] Where k∈{0,1,2,3}, and Respectively represent large-scale, medium-scale and small-scale embedding, Layer k Represents the kth Swin Transformer block in the cross-scale feature fusion encoder.

[0027] Furthermore, the decoder is composed of a 1×1 convolution layer and a Pixel Shuffle layer.

[0028] Furthermore, the image reconstruction model also includes a random layer, which is arranged behind the decoder. The random layer is used to resize the clean image reconstructed by the decoder and obtain an image of a preset size by filling.

[0029] The second technical solution adopted by the present invention is:

[0030] An image classification adversarial sample defense device, comprising:

[0031] An image denoising module is used to obtain an input image and construct a multi-scale latent noise image based on the input image;

[0032] The image reconstruction module is used to build and train the image reconstruction model, input the multi-scale hidden noise image into the trained image reconstruction model to obtain a clean image;

[0033] The classifier fine-tuning module is used to construct a fine-tuning dataset based on clean images, fine-tune the classifier based on the fine-tuning dataset, and use the fine-tuned classifier to achieve classification.

[0034] The third technical solution adopted by the present invention is:

[0035] An electronic device comprises a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement an image classification adversarial sample defense method as described above.

[0036] The fourth technical solution adopted by the present invention is:

[0037] A computer-readable storage medium, wherein at least one instruction, at least one program, code set or instruction set is stored in the storage medium, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement an image classification adversarial sample defense method as described above.

[0038] The fifth technical solution adopted by the present invention is:

[0039] A computer program product or a computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above method.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] (1) The present invention proposes a multi-scale hidden noise image construction strategy, which can effectively destroy the effect of adversarial perturbations from a multi-scale perspective using traditional methods before the adversarial samples are input into the reconstruction model, thereby solving the problem that deep neural network models alone cannot effectively destroy adversarial perturbations.

[0042] (2) The present invention proposes a deep neural network reconstruction model with cross-scale feature interaction, which makes the reconstruction model have stronger modeling ability and can better reconstruct clean images from multi-scale latent noise images. Combined with the multi-scale latent noise image construction strategy, it can significantly improve the robustness of adversarial defense.

[0043] (3) We propose to fine-tune the classifier using the clean dataset reconstructed from latent noise images, so that the classifier can better adapt to the reconstructed data distribution. This improves adversarial robustness and clean accuracy, and solves the trade-off problem faced by previous adversarial elimination-based methods in terms of adversarial robustness and clean accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 is a flow chart of an image classification adversarial sample defense method based on latent noise image reconstruction proposed in an embodiment of the present invention;

[0046] Figure 2 are clean samples and adversarial samples and their corresponding latent noise images in an embodiment of the present invention;

[0047] Figure 3 is a schematic diagram of a multi-scale hidden noise image construction strategy in an embodiment of the present invention;

[0048] Figure 4 is a structural diagram of a cross-scale feature fusion encoder in a reconstruction model in an embodiment of the present invention;

[0049] Figure 5 is a structural diagram of a decoder in a reconstruction model in an embodiment of the present invention;

[0050] Figure 6 is a schematic diagram of random layer processing in an embodiment of the present invention;

[0051] Figure 7 It is a schematic diagram of the fine-tuning strategy in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0053] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0054] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0055] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0056] In response to the existing technical problems, the present invention proposes an image classification adversarial sample defense scheme based on latent noise image reconstruction, which combines the traditional principal component analysis method and deep learning method to eliminate the effects of adversarial attacks from a multi-scale perspective, restore the adversarial samples to the clean data distribution, and solve the trade-off between adversarial robustness and clean accuracy through a simple and effective fine-tuning strategy.

[0057] Example 1

[0058] like Figure 1 As shown, this embodiment provides an image classification adversarial sample defense method based on latent noise image reconstruction, which is essentially an adversarial sample defense method for image classification that combines traditional methods and deep learning methods. The specific steps are as follows:

[0059] S1. Obtain an input image (clean image / adversarial image) and construct the input image into a multi-scale latent noise image.

[0060] In some embodiments, step S1 designs a multi-scale latent noise image construction strategy, and uses a tile-level principal component analysis (PCA) method to construct an input image (clean image / adversarial image) into a multi-scale latent noise image.

[0061] In particular, the latent noise images constructed from the clean samples and adversarial samples in step S1 have similar representations, such as Figure 2 As shown, Figure 2 (a) is a clean sample and its corresponding hidden noise image. Figure 2 (b) shows the adversarial sample and its corresponding latent noise image; this allows us to use only clean samples to construct latent noise images and use self-supervision to train the reconstruction model in step S2 (the reconstruction model is essentially a denoising autoencoder).

[0062] S2. Build and train an image reconstruction model, input the multi-scale hidden noise image into the trained image reconstruction model to obtain a clean image.

[0063] In some embodiments, step S2 designs a deep neural network image reconstruction model with cross-scale feature interaction, and uses the trained image reconstruction model to reconstruct the multi-scale latent noise image obtained in step S1 to restore the image to the distribution of clean data.

[0064] S3. Construct a fine-tuning dataset based on the clean image, fine-tune the classifier based on the fine-tuning dataset, and use the fine-tuned classifier to implement classification.

[0065] In some embodiments, step S3 proposes a simple and effective fine-tuning strategy based on steps S1 and S2, and uses steps S1 and S2 to perform latent noise image reconstruction on the clean training set to obtain a fine-tuning data set to fine-tune the classifier.

[0066] The multi-scale hidden noise image construction strategy proposed in the embodiment of the present invention can effectively destroy the effect of adversarial disturbances from a multi-scale perspective. Combined with the deep neural network image reconstruction model, the adversarial sample can be restored to the clean data distribution, and the traditional method and deep learning method are combined to significantly improve the robustness of adversarial defense. In addition, the strategy of fine-tuning the classifier using the clean data set reconstructed by the hidden noise image proposed in steps S1 and S2 in this embodiment can significantly improve the clean accuracy while further improving the robust accuracy, effectively solving the trade-off problem between adversarial robustness and clean accuracy. Therefore, the image classification adversarial sample defense based on hidden noise image reconstruction proposed in this embodiment can enable the protected classifier to achieve superior adversarial robustness while maintaining excellent clean accuracy.

[0067] The above method is explained in detail below with reference to the accompanying drawings and specific embodiments.

[0068] Figure 1 This paper presents a flowchart of an image classification adversarial sample defense method based on latent noise image reconstruction. First, we design a multi-scale latent noise image construction strategy based on patch-level PCA to construct multi-scale latent noise images and destroy the effect of adversarial perturbations from a multi-scale perspective; then, we propose a reconstruction model with cross-scale feature interactions to reconstruct latent noise images back to clean data distribution. Finally, we fine-tune the classifier using a clean dataset reconstructed from latent noise images, effectively solving the trade-off problem between adversarial robustness and clean accuracy faced by previous adversarial elimination-based methods.

[0069] (a) Multi-scale hidden noise image construction

[0070] In order to more effectively destroy the effect of adversarial perturbations, we designed a multi-scale latent noise image construction strategy based on patch-level PCA. The principle is as follows Figure 3 shown.

[0071] Given an image to be input to the classifier, it is first transformed into three different scales, namely large scale, medium scale and small scale, through a resizing operation; the large scale, medium scale and small scale images are divided into image blocks of the same size and flattened into D-dimensional column vectors, which are recorded as Since the image patches of the same size separated from images of different scales have very different receptive fields, we use these image patches separated from images of different scales to calculate three PCA bases, denoted as Where d is the dimension of the latent space.

[0072] Next, we use the block-level PCA to map the image block to the latent space, introduce Gaussian noise in the latent space, and map these latent space features with added noise back to the pixel space through inverse PCA, thereby constructing a multi-scale latent noise image. The construction process can be expressed by the following formula:

[0073]

[0074] Where σ represents the noise scale, which is used to control the degradation level of the latent noise image. Figure 3 As shown on the right, large-scale image patches have smaller receptive fields and can retain more details, while small-scale image patches have larger receptive fields and can capture more global features. In other words, multi-scale latent noise images have image information with different granularity and degradation, which may help the reconstruction model learn the intrinsic patterns of clean images. For large-scale images (such as ImageNet), we use multi-scale image resolutions of 512, 256, and 128, and the image patch size is 16; for small-scale images (such as CIFAR-10), we use three resolutions of 64, 32, and 16, and the image patch size is 8. The latent space size is set to 16.

[0075] (b) Reconstruction model with cross-scale feature interaction

[0076] After the adversarial sample is constructed into a latent noise image, the adversarial perturbation is largely destroyed, but the image itself is also degraded. Therefore, we propose a reconstruction model with powerful modeling capabilities based on Swin Transformer to reconstruct clean images from multi-scale latent noise images, thereby achieving the defense goal of restoring the adversarial sample to a clean distribution. The reconstruction model is essentially a denoising autoencoder (DAE), which consists of a cross-scale feature fusion encoder and a simple decoder.

[0077] 1) Cross-scale feature fusion encoder

[0078] The cross-scale feature fusion encoder structure is as follows Figure 4 As shown. In order to enhance the modeling ability of the model, we designed a cross-scale feature interaction strategy. For large-scale, medium-scale, and small-scale inputs, the multi-scale input is first embedded into a suitable dimension using an embedding layer, using 8×, 4×, and 2× downsampling respectively. The cross-scale feature interaction can be expressed by the following formula:

[0079]

[0080] where k∈{0,1,2,3}, and Respectively represent large-scale, medium-scale and small-scale embedding, Layer k Represents the kth Swin Transformer block in the encoder. The fused high-level features finally output by the encoder is sent to the decoder for reconstruction.

[0081] 2) Simple decoder

[0082] Most decoders in the past have adopted complex structures. On the contrary, we only use a decoder with a very simple structure. Figure 5 As shown in Figure 1, the decoder consists of only one 1×1 convolutional layer and one Pixel Shuffle layer. Thanks to the powerful feature extraction capability of the cross-scale feature fusion encoder, such an extremely simple decoder is sufficient to reconstruct a high-quality clean image.

[0083] 3) Random Layer

[0084] In order to further improve the model's ability to defend against robust attacks, during inference, we add a random layer after the reconstruction model. The random layer first randomly adjusts the size of the reconstructed image to [299,331), and then fills the image to 331×331 by 0 padding. The random layer is processed as follows: Figure 6 shown.

[0085] (c) A simple yet effective fine-tuning strategy

[0086] The image obtained after latent noise image reconstruction exhibits the following characteristics: the essential structural information of the image (robust features) is retained, while some non-robust features that are sensitive to disturbances (including adversarial disturbances) are discarded. Since the original classifier may also use these discarded non-robust features when making decisions, if the original classifier is used to predict these reconstructed images, it may lead to a certain degree of performance degradation. Therefore, we proposed a simple and effective fine-tuning strategy. Specifically, we introduced latent noise into the clean training set, and after latent noise image reconstruction, we constructed a fine-tuning dataset. This fine-tuning dataset constructed entirely from clean data is used to fine-tune the classifier, so that the classifier can better adapt to the reconstructed data distribution. While improving the clean accuracy, it also further improves the robust accuracy, effectively solving the trade-off problem faced by existing adversarial elimination-based methods in terms of adversarial robustness and clean accuracy. The principle of the fine-tuning strategy is as follows: Figure 7 shown.

[0087] Example 2

[0088] This embodiment provides an image classification adversarial sample defense device, including:

[0089] An image denoising module is used to obtain an input image and construct a multi-scale latent noise image based on the input image;

[0090] The image reconstruction module is used to build and train the image reconstruction model, input the multi-scale hidden noise image into the trained image reconstruction model to obtain a clean image;

[0091] The classifier fine-tuning module is used to construct a fine-tuning dataset based on clean images, fine-tune the classifier based on the fine-tuning dataset, and use the fine-tuned classifier to achieve classification.

[0092] Since the device is an image classification adversarial sample defense device of an embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0093] Example 3

[0094] An embodiment of the present invention further provides an electronic device, the electronic device comprising a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or the instruction set being loaded and executed by the processor to implement the following Figure 1 A method for defending against adversarial samples in image classification based on latent noise image reconstruction is shown.

[0095] It is understood that the memory may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data created according to the use of the server, etc.

[0096] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect the various parts of the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor can integrate one or a combination of a central processing unit (CPU) and a modem. Among them, the CPU mainly processes the operating system and application programs; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor, but implemented separately through a chip.

[0097] Since the electronic device is an electronic device corresponding to an image classification adversarial sample defense method based on latent noise image reconstruction in an embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0098] Example 4

[0099] The embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the following Figure 1A method for defending against adversarial samples in image classification based on latent noise image reconstruction is shown.

[0100] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0101] Since the storage medium is a storage medium corresponding to an image classification adversarial sample defense method based on latent noise image reconstruction in an embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0102] Example 5

[0103] In some possible implementations, various aspects of the method of the embodiments of the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the image classification adversarial sample defense method based on hidden noise image reconstruction according to various exemplary embodiments of the present application described above in this specification. Among them, the executable computer program code or "code" used to execute various embodiments can be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, structured query language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0104] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0105] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0106] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for defending against adversarial samples in image classification, characterized in that: The following steps are involved: Obtain an input image, and construct a multi-scale hidden noise image according to the input image; Build and train an image reconstruction model, input the multi-scale hidden noise image into the trained image reconstruction model to obtain a clean image; A fine-tuning dataset is constructed based on the clean images, the classifier is fine-tuned based on the fine-tuning dataset, and the fine-tuned classifier is used to implement classification.

2. The image classification adversarial sample defense method according to claim 1, characterized in that: The step of obtaining an input image and constructing a multi-scale hidden noise image according to the input image includes: The input image is transformed into three different scales through the resizing operation to obtain a large-scale image, a medium-scale image, and a small-scale image; The large-scale image, medium-scale image and small-scale image are divided into image blocks of the same size and flattened into a D-dimensional column vector, denoted as x p , p=l,m,s; According to the image blocks divided from the large-scale image, medium-scale image and small-scale image, three PCA bases are calculated respectively, denoted as V p , p=l,m,s; The image blocks are mapped to the latent space using block-level PCA, Gaussian noise is introduced into the latent space, and these noisy latent space features are mapped back to the pixel space through inverse PCA to construct a multi-scale latent noise image.

3. The image classification adversarial sample defense method according to claim 2, characterized in that: The expression of the multi-scale hidden noise image is as follows: Where σ represents the noise scale, ∈ is Gaussian noise, and T represents transposition.

4. The method for defending against adversarial samples in image classification according to claim 1, characterized in that: The image reconstruction model is a reconstruction model with cross-scale feature interaction, and the image reconstruction model includes a cross-scale feature fusion encoder and a decoder; The cross-scale feature fusion encoder is used to extract features of images of different scales, and fuse the extracted features to obtain fused features; The decoder is used to reconstruct a clean image based on the fused features; The clean image retains the essential structural information of the image while discarding some non-robust features that are sensitive to disturbances.

5. The method for defending against adversarial samples in image classification according to claim 4, characterized in that: The expression of the cross-scale feature interaction is: Where k∈{0,1,2,3}, and Respectively represent large-scale, medium-scale and small-scale embedding, Layer k Represents the kth Swin Transformer block in the cross-scale feature fusion encoder.

6. The method for defending against adversarial samples in image classification according to claim 4, characterized in that: The decoder consists of a 1×1 convolution layer and a Pixel Shuffle layer.

7. The image classification adversarial sample defense method according to claim 4, characterized in that: The image reconstruction model also includes a random layer, which is arranged behind the decoder. The random layer is used to resize the clean image reconstructed by the decoder and obtain an image of a preset size by filling.

8. An image classification adversarial sample defense device, characterized in that: include: An image denoising module is used to obtain an input image and construct a multi-scale latent noise image based on the input image; The image reconstruction module is used to build and train the image reconstruction model, input the multi-scale hidden noise image into the trained image reconstruction model to obtain a clean image; The classifier fine-tuning module is used to construct a fine-tuning dataset based on clean images, fine-tune the classifier based on the fine-tuning dataset, and use the fine-tuned classifier to achieve classification.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 7.

Images