An object-level adversarial attack method for hyperspectral image classification network

Abstract

The application discloses a kind of object-level counter-attack methods for hyperspectral image classification network, comprising the following: step one, original hyperspectral image is carried out image blocking pixel by pixel, obtain multiple clean samples;Step two, input multiple clean samples, according to the size of input multiple clean samples, randomly generate initial disturbance consistent with the size of clean sample, add with multiple clean samples, obtain initial hyperspectral image's counter-attack sample;Step three, iteratively update disturbance;Step 4, iteratively update the disturbance constrained by 2D over-segmentation space template;Step 5, the final disturbance is added to clean sample, and the final hyperspectral image counter-attack sample is obtained.Adopt the counter-attack method, improve the migration and robustness of counter-attack sample, reach better attack effect.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an object-level adversarial attack method for hyperspectral image classification networks. Background Technology

[0002] Remote sensing image classification is a crucial component of understanding land cover. Hyperspectral images not only describe the two-dimensional spatial features of land cover but also contain detailed spectral information; therefore, they can identify more precise land cover objects. Deep learning techniques have become the dominant research method in hyperspectral image classification. Deep neural networks (DNNs) explore rich and complementary spectral spatial features to further improve classification performance.

[0003] However, the vulnerability of DNNs to adversarial examples has attracted widespread attention. Adding imperceptible perturbations to clean samples can cause DNNs to output incorrect predictions. This problem has spurred research into adversarial attacks and defenses, with researchers assessing the risks faced by DNNs and further improving their robustness. Most adversarial attack methods have achieved excellent results on natural images, but further applying these methods to generate adversarial examples for hyperspectral images in high-dimensional spaces still faces the following two challenges:

[0004] 1. Poor Transferability of Hyperspectral Image Data: Adversarial attack methods can be divided into two categories: white-box attacks and black-box attacks. In real-world scenarios, black-box attacks are more challenging. Transfer-based black-box attacks are currently a hot topic, aiming to leverage the transferability of adversarial examples generated by white-box attack methods. Existing research mainly focuses on adversarial attacks on single-band or three-band images, with less research on adversarial attack methods for high-dimensional image data. Compared to natural images, hyperspectral data contains additional spectral dimensions and has spectral redundancy. However, existing attack methods often ignore the relationships between spectral bands, resulting in unsatisfactory transferability.

[0005] 2. Poor robustness to image processing-based defenses: Most adversarial attack methods are implemented at the pixel level, i.e., perturbations are added independently to each pixel. However, pixel-level perturbations are not robust to image processing-based defenses. In hyperspectral image classification tasks, hyperspectral images are often affected by image preprocessing techniques, and pixel-level perturbations can be easily removed. Summary of the Invention

[0006] The purpose of this invention is to provide an object-level adversarial attack method for hyperspectral image classification networks, which improves the transferability and robustness of adversarial samples and achieves better attack results.

[0007] This invention employs the following technical solution: an object-level adversarial attack method for hyperspectral image classification networks, comprising the following steps:

[0008] Step 1: Divide the original hyperspectral image into pixels to obtain multiple clean samples;

[0009] Step 2: Input multiple clean samples. Based on the size of the input clean samples, randomly generate an initial perturbation that matches the size of the clean samples. Add the perturbation to the multiple clean samples to obtain the adversarial sample of the initial hyperspectral image.

[0010] Step 3: Iteratively update the perturbation:

[0011] Step 3a: Perform spectral dimensionality reduction on the original hyperspectral image using principal component analysis, and generate a 2D oversegmented spatial template from the dimensionality-reduced image using the SLIC superpixel segmentation method.

[0012] Step 3b: Input the initial perturbation from Step 2, and according to the 2D oversegmentation space template, replace each pixel value of the initial perturbation with the average perturbation value within its corresponding superpixel to obtain a spatially smooth perturbation.

[0013] Step 3c: Update and trim the spatially local smoothing perturbation in step 3b to obtain the updated spatially local smoothing perturbation;

[0014] Step 3d: Using the updated spatial local smoothing perturbation from step 3c as input, repeat steps 3b and 3c, iteratively update, and obtain the perturbation constrained by the 2D oversegmented spatial template.

[0015] Step 4: Iterative update of perturbations caused by 2D oversegmented space template constraints:

[0016] Step 4a: Generate a 1D oversegmented spectral template from the original hyperspectral image using the K-means clustering method;

[0017] Step 4b: Input the perturbation generated in step 3d that is constrained by the 2D oversegmented spatial template, and obtain the perturbation with local smoothness in the spectral dimension based on the 1D oversegmented spectral template.

[0018] Step 4c: Update and trim the locally smoothed perturbation in the spectral dimension to obtain the updated locally smoothed perturbation in the spectral dimension;

[0019] Step 4d: Using the updated spectral dimension local smoothing perturbation generated in step 4c as input, repeat steps 4b to 4c, iteratively update, and obtain the final perturbation constrained by the 2D oversegmented spatial template and the 1D oversegmented spectral template.

[0020] Step 5: Add the final perturbation to the clean sample to obtain the final hyperspectral image adversarial sample.

[0021] Furthermore, in step 3b, the spatially localized smoothing perturbation is obtained using formula (A), as shown below:

[0022] (A);

[0023] in: This indicates that the nth perturbation within the mth superpixel is replaced by the average perturbation value within that superpixel;

[0024] This represents the perturbation in the i-th iteration;

[0025] This represents the classification result of the nth spatial pixel within the mth superpixel of the 2D oversegmented spatial template generated based on SLIC superpixel segmentation;

[0026] i represents the i-th iteration.

[0027] Furthermore, in step 4b, the updated spatially smoothed perturbation is obtained using formula (B), as shown below:

[0028] (B);

[0029] in: This represents the perturbation in the (i+1)th iteration;

[0030] This indicates that the disturbance will be limited to , Inside, This represents the attack strength, with a value ranging from 0.01 to 0.1.

[0031] The parameter represents the constraint ratio between the gradient and the gradient norm, and its value is 1.0;

[0032] Represents the loss function;

[0033] Indicates multiple clean samples;

[0034] Indicates a label;

[0035] Indicates multiple clean samples Plus disturbance The classification results of the post-model.

[0036] Furthermore, in step 3c, the spatially localized perturbations are updated and pruned based on the loss gradient of the classification model.

[0037] Furthermore, in step 4c, the perturbations that are locally smoothed in the spectral dimension are updated and pruned according to the loss gradient of the classification model.

[0038] The beneficial effects of this invention are: 1. Based on hyperspectral image classification, oversegmented spatial and spectral templates are generated using the SLIC superpixel segmentation method and K-means clustering method, respectively. Under the constraints of the spatial and spectral templates, locally smooth spatial and spectral perturbations are obtained. Adversarial examples with smooth perturbations can avoid overfitting, thus adversarial examples generated by the white-box model are more easily transferred to the black-box model, achieving better transferability and attack performance. 2. An iterative update method is adopted to continuously force the spatial and spectral perturbations to be smoother in an iterative manner, thereby generating adversarial examples robust to image processing-based defenses. Attached Figure Description

[0039] Figure 1 This is the dataset used in the embodiments of the present invention.

[0040] Where: 1a is the Pavia University dataset, and 1b is the Houston 2018 dataset;

[0041] Figure 2 Adversarial examples generated for different attack methods on the PaviaU dataset.

[0042] Wherein: 2a is the MI-FGSM method, and 2b is the M-SSI-FGSM method. Detailed Implementation

[0043] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0044] Step 1: Divide the original hyperspectral image into pixels to obtain multiple clean samples.

[0045] Step 2: Input multiple clean samples. Based on the sizes of the input clean samples, randomly generate an initial perturbation with the same size as the clean samples. Add this perturbation to the multiple clean samples to obtain the initial adversarial example of the hyperspectral image. The size of the initial perturbation is limited to... Inside, The attack strength is given (with a range of values).

[0046] Step 3: Iteratively update the perturbation:

[0047] Step 3a: Principal component analysis is used to perform spectral dimensionality reduction on the original hyperspectral image. The SLIC superpixel segmentation method is then used to generate a 2D oversegmented spatial template from the dimensionality-reduced image. The spatial superpixel size is set to 4.

[0048] Step 3b: Input the initial perturbation from Step 2. Based on the 2D oversegmentation space template, replace each pixel value of the initial perturbation with the average perturbation value within its corresponding superpixel to obtain a spatially smooth perturbation, as shown in Formula (A):

[0049] (A);

[0050] in: This indicates that the nth perturbation within the mth superpixel is replaced by the average perturbation value within that superpixel;

[0051] This represents the perturbation in the i-th iteration;

[0052] This represents the SLIC superpixel segmentation result corresponding to the nth spatial pixel within the mth superpixel in the 2D oversegmented spatial template generated by SLIC superpixel segmentation.

[0053] i represents the i-th iteration and is an integer greater than 0.

[0054] Step 3c: Update the spatial local smoothing perturbation based on the loss gradient of the classification model and prune it to obtain the updated spatial local smoothing perturbation. The spatial local smoothing perturbation is restricted to... Within the attack strength range, as shown in formula (B):

[0055] (B);

[0056] in: This represents the perturbation in the (i+1)th iteration;

[0057] This indicates that the disturbance will be limited to , Inside, This represents the attack strength, with a value ranging from 0.01 to 0.1.

[0058] The parameter represents the constraint ratio between the gradient and the gradient norm, and its value is 1.0;

[0059] Represents the loss function;

[0060] Indicates multiple clean samples;

[0061] Indicates a label;

[0062] Indicates multiple clean samples Plus disturbance The classification results of the post-model.

[0063] Step 3d: Using the updated spatial local smoothing perturbation from step 3c as input, repeat steps 3b and 3c, iteratively updating to obtain the perturbation constrained by the 2D oversegmented spatial template. The number of iterations is set to 10.

[0064] Step 4: Iterative update of perturbations caused by 2D oversegmented space template constraints:

[0065] Step 4a: Use the K-means clustering method to generate a 1D oversegmented spectral template from the original hyperspectral image.

[0066] The spectral cluster size is set to 20.

[0067] Step 4b: Input the perturbation generated in step 3d that is constrained by the 2D oversegmented spatial template. Based on the 1D oversegmented spectral template, replace each pixel value of the perturbation constrained by the 2D oversegmented spatial template with the average perturbation value within its corresponding spectral subclass to obtain a locally smoothed perturbation in the spectral dimension, as shown in formula (C):

[0068] (C);

[0069] in: This means that the perturbation of the q-th spectrum in the p-th cluster is replaced by the average perturbation in that sub-cluster;

[0070] This represents the perturbation in the i-th iteration;

[0071] This represents the K-means clustering result corresponding to the nth spectrum within the p-th cluster of the 1D oversegmented spectral template generated using the K-means clustering method.

[0072] i represents the i-th iteration, and its value is an integer greater than 0.

[0073] Step 4c: Update and trim the locally smoothed perturbation in the spectral dimension based on the loss gradient of the classification model, obtaining the updated locally smoothed perturbation in the spectral dimension. Restrict the locally smoothed perturbation in the spectral dimension to... , Inside, The attack strength is shown in Formula (B).

[0074] Step 4d: Using the updated spectral dimension local smoothing perturbation generated in step 4c as input, repeat steps 4b-4c to iteratively update the perturbation, obtaining the final perturbation constrained by the 2D oversegmented spatial template and the 1D oversegmented spectral template. The number of iterations is set to 10.

[0075] Step 5: Add the final perturbation to the clean sample to obtain the final hyperspectral image adversarial sample.

[0076] The pixel value size of the adversarial example is limited to [0,1], as shown in Equations D and E:

[0077] (D);

[0078] (E);

[0079] Where: i represents the i-th iteration;

[0080] It is the adversarial example updated in the i-th iteration;

[0081] This means restricting the pixel values ​​of adversarial samples to within [0,1].

[0082] This is the final adversarial example output.

[0083] To further illustrate the object-level adversarial attack method for hyperspectral image classification networks in this invention, the following simulation experiment was conducted:

[0084] Simulation conditions:

[0085] The hardware requirements for the simulation are: Windows XP, SPI, CPU Pentium(R)4, with a base frequency of 2.4GHz; the software platform is: Matlab R2022a, PyTorch.

[0086] The images used in the simulation were from the Pavia University high-resolution remote sensing image dataset and the Houston 2018 image dataset. The Pavia University image dataset contains nine types of land features, such as... Figure 2 As shown in (2a), the Houston 2018 dataset contains 20 types of land features, such as... Figure 2 As shown in (2b).

[0087] Simulation content and results:

[0088] Simulation 1: Comparing the object-level adversarial attack method for hyperspectral image classification networks in this invention with the existing FGSM, MI-FGSM, and VMI-FGSM techniques. In the object-level adversarial attack method for hyperspectral image classification networks in this invention, FGSM, MI-FGSM, and VMI-FGSM techniques are used in steps 3c and 4c, respectively. The application of FGSM, MI-FGSM, and VMI-FGSM techniques in steps 3c and 4c of this invention is defined as SS-FGSM, M-SSI-FGSM, and VMI-SS-FGSM.

[0089] right Figure 1 Adversarial examples were generated for the two datasets shown, and the classification accuracy was tested. The results are as follows: Figure 2 As shown, where:

[0090] Figure 2 In the examples 2a and 2b, respectively, adversarial samples were generated on the Pavia University image dataset using two methods: MI-FGSM and M-SSI-FGSM, which are object-level adversarial attack methods for hyperspectral image classification networks according to the present invention. Bands 102, 56, and 31 were selected for display.

[0091] from Figure 2 As can be seen from the adversarial example diagram, the adversarial examples generated using existing technologies for MI-FGSM are as follows: Figure 2 As shown in 2a, there is a lot of noise. The adversarial sample generated by the M-SSI-FGSM of this invention is as follows: Figure 2 As shown in 2b, the local smoothness is better.

[0092] Table 1 shows a numerical comparison of the detection accuracy of adversarial examples generated by existing technologies FGSM, MI-FGSM, VMI-FGSM and SS-FGSM, M-SSI-FGSM, VMI-SS-FGSM on the Pavia University dataset.

[0093] Table 2 is a numerical comparison of the detection accuracy of adversarial examples generated by existing technologies FGSM, MI-FGSM, VMI-FGSM and SS-FGSM, M-SSI-FGSM, VMI-SS-FGSM on the Houston 2018 dataset.

[0094] Tables 1 and 2 show the classification accuracy of adversarial samples generated by the above methods based on ResNet-18 on five networks: ResNet-18, VGG-11, VGG-19, Inc-v3, and IncRes-v2, with attack strengths set to 0.01 and 0.03, respectively. On the Pavia University dataset, at an intensity of 0.03, adversarial samples generated by ResNet-18, when transferred to VGG11, showed an FGSM detection accuracy of 0.5607, while the SS-FGSM method reduced this to 0.3468, significantly enhancing the attack performance. On the HoustonU 2018 dataset, at an intensity of 0.01, adversarial samples generated by ResNet-18, when transferred to IncRes-v2, showed a VMI-FGSM detection accuracy of 0.8258, while the VMI-SS-FGSM method reduced this to 0.6835, demonstrating better transferability. Therefore, it can be concluded that the method in this invention has lower classification accuracy, better attack effect, and stronger transferability.

[0095] Table 1 shows a numerical comparison of detection accuracy on the Pavia University dataset.

[0096] ;

[0097] Table 2 shows a numerical comparison of detection accuracy on the HoustonU 2018 dataset.

[0098]

[0099] This invention presents an object-level adversarial attack method for hyperspectral image classification networks. It employs hyperspectral image classification combined with superpixel segmentation and band clustering techniques to generate smoother perturbations, thereby improving the attack effectiveness and transferability of adversarial examples. By enhancing the local smoothness of the high-dimensional space and spectral perturbations, the transferability and robustness of adversarial examples are improved. Smooth perturbations can reduce the sensitivity of different classification models to pattern discrimination and are more robust to image processing-based defenses.

Claims

1. A method for object-level adversarial attacks against hyperspectral image classification networks, characterized in that, Includes the following steps: Step 1: Divide the original hyperspectral image into pixels to obtain multiple clean samples; Step 2: Input multiple clean samples. Based on the size of the input clean samples, randomly generate an initial perturbation that matches the size of the clean samples. Add the perturbation to the multiple clean samples to obtain the adversarial sample of the initial hyperspectral image. Step 3: Iteratively update the perturbation: Step 3a: Perform spectral dimensionality reduction on the original hyperspectral image using principal component analysis, and generate a 2D oversegmented spatial template from the dimensionality-reduced image using the SLIC superpixel segmentation method. Step 3b: Input the initial perturbation from Step 2, and according to the 2D oversegmentation space template, replace each pixel value of the initial perturbation with the average perturbation value within its corresponding superpixel to obtain a spatially smooth perturbation. Step 3c: Update and trim the spatially local smoothing perturbation in step 3b to obtain the updated spatially local smoothing perturbation; Step 3d: Using the updated spatial local smoothing perturbation from step 3c as input, repeat steps 3b and 3c, iteratively update, and obtain the perturbation constrained by the 2D oversegmented spatial template. Step 4: Iterative update of perturbations caused by 2D oversegmented space template constraints: Step 4a: Generate a 1D oversegmented spectral template from the original hyperspectral image using the K-means clustering method; Step 4b: Input the perturbation generated in step 3d that is constrained by the 2D oversegmented spatial template, and obtain the perturbation with local smoothness in the spectral dimension based on the 1D oversegmented spectral template. Step 4c: Update and trim the locally smoothed perturbation in the spectral dimension to obtain the updated locally smoothed perturbation in the spectral dimension; Step 4d: Using the updated spectral dimension local smoothing perturbation generated in step 4c as input, repeat steps 4b to 4c, iteratively update, and obtain the final perturbation constrained by the 2D oversegmented spatial template and the 1D oversegmented spectral template. Step 5: Add the final perturbation to the clean sample to obtain the final hyperspectral image adversarial sample; In step 4b, the updated spatially smoothed perturbation is obtained using formula (B), as shown below: Where: δ i+1 This represents the perturbation in the (i+1)th iteration; clip ε This indicates that the disturbance is limited to [-ε, ε], where ε is the attack strength, with a value of 0.01 to 0.1; α represents a parameter that limits the ratio of the gradient to the gradient norm, and its value is 1.0; J represents the loss function; x represents multiple clean samples; y represents the label; f(x+δ i ) represents multiple clean samples x plus perturbation δ i The classification results of the post-model.

2. The object-level adversarial attack method for hyperspectral image classification networks as described in claim 1, characterized in that, In step 3b, spatially localized smoothing perturbations are obtained using formula (A), as shown below: in: This indicates that the nth perturbation within the mth superpixel is replaced by the average perturbation value within that superpixel; δ i This represents the perturbation in the i-th iteration; This represents the classification result of the nth spatial pixel within the mth superpixel of the 2D oversegmented spatial template generated based on SLIC superpixel segmentation; i represents the i-th iteration.

3. The object-level adversarial attack method for hyperspectral image classification networks as described in claim 2, characterized in that, In step 3c, the spatially localized perturbations are updated and pruned based on the loss gradient of the classification model.

4. The object-level adversarial attack method for hyperspectral image classification networks as described in claim 3, characterized in that, In step 4c, the perturbations that are locally smoothed in the spectral dimension are updated and pruned according to the loss gradient of the classification model.

Images