A transferable adversarial attack method for hyperspectral image classification

Abstract

The application discloses a kind of transferable adversarial attack methods for hyperspectral image classification, to generate adversarial samples with transferability using white-box model, to realize effective attack to target black-box model, the method includes: selecting white-box hyperspectral image classification model as source model, the gradient of each band is calculated by back propagation algorithm, and the importance score of band is determined;Based on the importance score, the pixel masking probability of the band is calculated, and the image is randomly pixel masked;The gradient information under multiple random masking is aggregated to obtain the aggregated gradient value;Loss function is designed in combination with aggregated gradient and feature matrix, and it is used to guide adversarial sample generation;Finally, the generated adversarial sample is applied to black-box model, and transfer attack is implemented.The generated adversarial sample has strong transferability, universality and adaptability, and provides a new idea for the security research in the field of hyperspectral image classification.

Description

Technical Field

[0001] This invention belongs to the field of adversarial attacks in image classification, and specifically relates to a transferable adversarial attack method for hyperspectral image classification. Background Technology

[0002] Hyperspectral images are a form of remote sensing data containing rich spectral information. Each pixel possesses spectral information across multiple bands and is commonly used in fields such as land cover classification, environmental monitoring, and precision agriculture. In hyperspectral image classification tasks, models identify the category to which a pixel belongs by analyzing the spectral features of pixels across different bands. In recent years, deep learning techniques have been widely applied to hyperspectral image classification tasks. These models can capture the complex relationships between spectral and spatial features, achieving significant improvements in classification accuracy. However, the high dimensionality of hyperspectral data and the redundancy of spectral bands make these models highly sensitive to noise and adversarial perturbations.

[0003] Adversarial attacks are techniques that add small perturbations to input data to mislead a model. In hyperspectral image classification, adversarial attacks can not only reduce the model's classification accuracy but also reveal its vulnerabilities, thus providing a basis for improving the model's robustness. However, existing adversarial attack research mainly focuses on generating adversarial samples in the RGB domain. These studies typically utilize the spatial characteristics of RGB images to mislead the classification model by adding small perturbations. However, RGB images and hyperspectral images differ significantly in their data characteristics. Hyperspectral images have higher spectral resolution and multi-band characteristics, and their classification models are more dependent on inter-band correlations and spectral features. Therefore, adversarial attack methods for RGB images are difficult to directly transfer to hyperspectral images.

[0004] Transfer attacks refer to generating adversarial examples on one model and applying them to other models to cause cross-model adversarial interference. The generated adversarial examples have cross-model attack effects, strong versatility and adaptability, and are applicable to a variety of models. However, current transferable adversarial attack methods for hyperspectral image classification have many shortcomings and challenges. (1) The complexity of high-dimensional data: Hyperspectral images usually contain spectral information of hundreds of bands, which makes their data dimension much higher than that of conventional RGB images. Most traditional adversarial example generation methods are designed for low-dimensional images, and often face problems such as high computational resource consumption and high optimization difficulty when processing high-dimensional data. The high-dimensional feature space increases the diversity and complexity of perturbations, making the generation of adversarial examples more difficult. (2) Transferability problem: Due to the structural differences of different models, data bias in the training process, and different feature learning methods, the transferability of adversarial examples is poor. Therefore, how to design an adversarial example generation strategy that can be effectively transferred between different hyperspectral image classification models is still a major challenge. (3) Efficiency problem of adversarial example generation: Generating adversarial examples usually requires a lot of iterative optimization. For hyperspectral images, due to their high dimensionality and complex spectral characteristics, the process of generating adversarial examples usually requires a large amount of computational resources and time. In addition, hyperspectral image classification models usually contain multiple levels and complex feature maps, which means that adversarial perturbations are not just simple pixel-level modifications, but require fine-tuning at a higher level, making the adversarial example generation process more time-consuming and computationally intensive. (4) Generality and adaptability issues: There are significant differences between different hyperspectral image classification tasks and datasets. For example, different object categories, sensor types, and data acquisition conditions will affect the performance of the model. This means that some adversarial examples designed for specific datasets or tasks may not be directly applicable to other tasks or datasets. Therefore, how to design a general and adaptable adversarial attack method that can adapt to a variety of different hyperspectral image classification tasks has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a transferable adversarial attack method for hyperspectral image classification. This method disrupts specific noise in the model by randomly masking different bands of the hyperspectral image, and obtains an aggregated gradient by aggregating the gradients after random pixel masking. The aggregated gradient is then used to guide the generation of adversarial examples, thereby improving the transferability of the adversarial examples.

[0006] The technical solution adopted by this invention to solve the above-mentioned technical problems is: a transferable adversarial attack method for hyperspectral image classification, characterized by including the following steps:

[0007] Step 1: For a hyperspectral image x in a given dataset, select a white-box hyperspectral image classification model as the source model, use the backpropagation algorithm to calculate the gradient values ​​of each band of the hyperspectral image, and use the average gradient of each band as the importance score I of the band. b , is used to measure the impact of each band on the classification decision, where b represents the b-th band;

[0008] Step 2: Based on feature importance score I b Set corresponding masking probabilities P for different frequency bands. b This allows for targeted perturbation of high-importance bands, based on P. b Generate k random pixel masking matrices M b Using M b Random pixel masking is applied to the input hyperspectral images to obtain images with random pixel masking. Where i∈{1,2,...,k};

[0009] Step 3: Using the backpropagation algorithm, apply the loss function of the hyperspectral image classification model to all images that have undergone random pixel masking. Perform gradient calculation to obtain the corresponding gradient value G. (i) The obtained k gradient values ​​are then aggregated to obtain the aggregated gradient value G. agg :

[0010]

[0011] Step 4: Aggregate gradient G agg The loss function is calculated by the element-wise dot product of the hyperspectral image feature matrix X, and this loss function is used to guide the generation of adversarial examples.

[0012] Step 5: Repeat steps 3 and 4 to continuously optimize the adversarial examples until the predetermined attack objective is met or the maximum number of iterations is reached. Use the generated adversarial examples to attack the black-box model.

[0013] Furthermore, the specific process of step 1 is as follows:

[0014] Step 1.1: Randomly select an image from the hyperspectral image dataset and input it into the model. Calculate the gradient G of the loss function with respect to the features of each node through backpropagation:

[0015]

[0016] Where L is the loss function and X is the feature matrix of the hyperspectral image;

[0017] Step 1.2: Take the absolute value of the gradient for each band and then average it. Use this average as the importance score I for that band.b .

[0018] Furthermore, the specific process of step 2 is as follows:

[0019] Step 2.1: Assign feature importance scores I b Normalized to the range [0,1], based on the normalized feature importance score I b Set the blocking probability P for different frequency bands b :

[0020]

[0021] Step 2.2: Set up k three-dimensional matrices with the same dimensions as the hyperspectral image feature matrix X, and initialize the values ​​in the three-dimensional matrices with random numbers sampled from a uniform distribution (0,1); then, correlate the values ​​in the three-dimensional matrices with the occlusion probability P. b Compare; when the value in the three-dimensional matrix is ​​greater than or equal to P b Set the values ​​in the three-dimensional matrix to 1; when the value in the three-dimensional matrix is ​​less than P b Set the values ​​in the 3D matrix to 0; generate k random pixel occlusion matrices M. b By M b The element-wise dot product with X is used to generate the masked image.

[0022] Furthermore, in step 4, based on the aggregation gradient G... agg We can design the dot product loss function of the gradient and features using the feature matrix X:

[0023]

[0024] Construct the final loss function by combining the cross-entropy loss function:

[0025] L=α·L(X)+β·L ce

[0026] Where α and β are weight hyperparameters, L ce It is the cross-entropy loss function; adversarial examples x are generated using the loss function L. adv :

[0027]

[0028] Where η is the learning rate. The gradient of the total loss with respect to X is given by t, where t is the number of iterations, x is the input image, ∈ is the perturbation magnitude, and Clip is the gradient of the total loss with respect to X. X,∈ Ensure the disturbance strength satisfies ||x adv -x‖ ∞ ≤∈.

[0029] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0030] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the above-described method.

[0031] A computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method.

[0032] Compared with the prior art, the significant advantages of the present invention are:

[0033] (1) This invention calculates the gradient of each band using the backpropagation algorithm and uses its average value as the importance score of the band, thereby quantifying the impact of each band on the classification decision. This mechanism ensures that when generating adversarial examples, the focus is on perturbing the bands that have a greater impact on the classification decision, thus enabling a more precise attack strategy.

[0034] (2) The present invention employs random pixel masking technology when generating adversarial samples, and highlights important features of the image by setting the masking probability of each band; by focusing on perturbing important bands and performing element-wise dot product between the masking matrix and the image features, it can reduce unnecessary computational overhead while ensuring the attack effect.

[0035] (3) This invention utilizes the loss function of the hyperspectral image classification model and combines the aggregated gradient values ​​under different occlusion configurations to optimize adversarial samples. This strategy destroys the specific noise features of the model by weighted aggregation of gradients obtained under different random pixel occlusion configurations, making the generated adversarial samples more transferable. Compared with the local optimization methods in the prior art, the aggregated gradient calculation method of this invention can better control the perturbation in the global scope and ensure the maximization of the adversarial attack effect. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention.

[0037] Figure 2 This is a schematic diagram of the polymerization gradient principle of the present invention. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0039] like Figure 1As shown, this invention provides a transferable adversarial attack method for hyperspectral image classification. By evaluating band importance and using specific noise features from a random pixel masking suppression model, the inherent characteristics of hyperspectral images are preserved. Furthermore, the method utilizes aggregated gradients to guide adversarial example generation, enhancing the transferability of adversarial examples. The specific steps are as follows:

[0040] Step 1: For the hyperspectral images in the given dataset, select a white-box hyperspectral image classification model as the source model, randomly select an image x and input it into the model, and calculate the gradient G of the loss function with respect to the features of each node through backpropagation:

[0041]

[0042] Where L is the loss function and X is the feature matrix of the hyperspectral image;

[0043] Then, the absolute values ​​of the gradients for each band of the input image x are taken and averaged to obtain the importance score I for that band. b :

[0044]

[0045] Where H and W are the height and width of the image, respectively; G b (i,j) represents the value of the gradient matrix at band b and pixel position (i,j).

[0046] By calculating the band importance score of hyperspectral images, the model can identify the bands that have the greatest impact on the classification task, thereby focusing on perturbing these key bands in adversarial attacks and improving the efficiency and effectiveness of the attack.

[0047] Step 2: Based on feature importance score I b Set corresponding masking probabilities P for different frequency bands. b First, the feature importance score I... b Normalized to the range [0,1], based on the normalized feature importance score I b Calculate the occlusion probability P b :

[0048]

[0049] This allows for targeted perturbation of highly important bands. Then, k three-dimensional matrices with the same dimensions as the hyperspectral image feature matrix X are set, and the values ​​in these matrices are initialized with random numbers sampled from a uniform distribution (0,1). The values ​​in the three-dimensional matrices are then compared with the occlusion probability P. b Compare; when the value in the three-dimensional matrix is ​​greater than or equal to P b Set the values ​​in the three-dimensional matrix to 1; when the value in the three-dimensional matrix is ​​less than Pb Set the values ​​in the 3D matrix to 0; generate k random pixel occlusion matrices M through the above steps. b By M b The element-wise dot product with X yields the image after random pixel masking.

[0050]

[0051] Where i∈{1,2,...,k}, is the masking matrix of the i-th random pixel, X represents the feature matrix of the input image, and ⊙ represents the element-wise dot product.

[0052] Random pixel masking introduces uncertainty through randomness, breaking the model's inherent noise characteristics and making it less reliant on specific pixel or band information. This effectively prevents adversarial examples from overfitting to specific features of the source model, thereby improving the transferability of adversarial examples.

[0053] Step 3: Using the backpropagation algorithm, apply the loss function of the hyperspectral image classification model to all images that have undergone random pixel masking. Perform gradient calculation to obtain the corresponding gradient value G. (i) The obtained k gradient values ​​are then aggregated to obtain the aggregated gradient value G. agg :

[0054]

[0055] The process of aggregating gradients is as follows Figure 2 As shown, the process includes random pixel masking of the input image, calculating the masking probability based on the image band importance score, and then generating a random pixel masking matrix. The k random pixel masking matrices are then element-wise multiplied with the input image to obtain the image after random pixel masking. The gradient G is calculated by backpropagation of the image after random pixel masking. (i) Then, these gradients are aggregated to obtain the aggregated gradient, where i∈{1,2,...,k}.

[0056] Step 4: Aggregate gradient G agg The loss function is calculated by the element-wise dot product of the feature matrix X of the hyperspectral image, and this loss function is used to guide the generation of adversarial examples. First, based on the aggregated gradient G... agg We can design the dot product loss function of the gradient and features using the feature matrix X:

[0057]

[0058] Then, the final loss function is constructed by combining the cross-entropy loss function:

[0059] L=α·L(X)+β·L ce

[0060] Where α and β are weight hyperparameters, L ce It is the cross-entropy loss function; adversarial examples x are generated using the loss function L. adv :

[0061]

[0062] Where η is the learning rate. The gradient of the total loss with respect to X is given by t, where t is the number of iterations, x is the input image, ∈ is the perturbation magnitude, and clip is the gradient of the total loss with respect to X. X,∈ Ensure the disturbance strength meets ||x adv -x|| ∞ ≤∈.

[0063] This loss function design leverages the combination of aggregated gradients and classification objectives, enabling more effective generation of adversarial examples with high transferability.

[0064] Step 5: Repeat steps 3 and 4 to continuously optimize the adversarial examples until the predetermined attack objective is met or the maximum number of iterations is reached. Use the generated adversarial examples to attack the black-box model.

[0065] This invention calculates the band importance of hyperspectral images and applies a lower pixel masking probability to important bands, ensuring that these bands containing key object information are preserved, while less important bands can discard more pixels. This helps preserve key object features in the image. By calculating and aggregating gradients for images under different random pixel masking configurations, a comprehensive gradient representation can be obtained. This aggregated gradient better reflects key changes in the image, thus guiding the model to focus on more important features. This invention generates adversarial examples through random pixel masking and aggregated gradient optimization, significantly improving the transferability of adversarial examples and effectively revealing the vulnerability of hyperspectral image classification models.

Claims

1. A transferable adversarial attack method for hyperspectral image classification, characterized in that, Includes the following steps: Step 1: For a hyperspectral image x in a given dataset, select a white-box hyperspectral image classification model as the source model, use the backpropagation algorithm to calculate the gradient values ​​of each band of the hyperspectral image, and use the average gradient of each band as the importance score I of the band. b , is used to measure the impact of each band on the classification decision, where b represents the b-th band; Step 2: Based on feature importance score I b Set corresponding masking probabilities P for different frequency bands. b This allows for targeted perturbation of high-importance bands, based on P. b Generate k random pixel masking matrices M b Using M b Random pixel masking is applied to the input hyperspectral images to obtain images with random pixel masking. Where i∈{1,2,...,k}; Step 3: Using the backpropagation algorithm, apply the loss function of the hyperspectral image classification model to all images that have undergone random pixel masking. Perform gradient calculation to obtain the corresponding gradient value G. (i) The obtained k gradient values ​​are then aggregated to obtain the aggregated gradient value G. agg : Step 4: Aggregate gradient G agg The loss function is calculated by the element-wise dot product of the hyperspectral image feature matrix X, and this loss function is used to guide the generation of adversarial examples. Step 5: Repeat steps 3 and 4 to continuously optimize the adversarial examples until the predetermined attack objective is met or the maximum number of iterations is reached. Use the generated adversarial examples to attack the black-box model.

2. The transferable adversarial attack method for hyperspectral image classification according to claim 1, characterized in that, The specific process of step 1 is as follows: Step 1.1: Randomly select an image from the hyperspectral image dataset and input it into the model. Calculate the gradient G of the loss function with respect to the features of each node through backpropagation: Where L is the loss function and X is the feature matrix of the hyperspectral image; Step 1.2: Take the absolute value of the gradient for each band and then average it. Use this average as the importance score I for that band. b .

3. The transferable adversarial attack method for hyperspectral image classification according to claim 1, characterized in that, The specific process of step 2 is as follows: Step 2.1: Assign feature importance scores I b Normalized to the range [0,1], based on the normalized feature importance score I b Set the blocking probability P for different frequency bands b : Step 2.2: Set up k three-dimensional matrices with the same dimensions as the hyperspectral image feature matrix X, and initialize the values ​​in the three-dimensional matrices with random numbers sampled from a uniform distribution (0,1); then, correlate the values ​​in the three-dimensional matrices with the occlusion probability P. b Compare; when the value in the three-dimensional matrix is ​​greater than or equal to P b Set the values ​​in the three-dimensional matrix to 1; When the value in the three-dimensional matrix is ​​less than P b Set the values ​​in the 3D matrix to 0; generate k random pixel occlusion matrices M. b By M b The element-wise dot product with X generates the masked image.

4. The transferable adversarial attack method for hyperspectral image classification according to claim 1, characterized in that, In step 4, based on the aggregation gradient G agg We can design the dot product loss function of the gradient and features using the feature matrix X: L(X)=∑G agg ⊙X Construct the final loss function by combining the cross-entropy loss function: L=α·L(X)+β·L ce Where α and β are weight hyperparameters, L ce It is the cross-entropy loss function; adversarial examples x are generated using the loss function L. adv : Where η is the learning rate. The gradient of the total loss with respect to X is given by t, where t is the number of iterations, x is the input image, ∈ is the perturbation magnitude, and Clip is the gradient of the total loss with respect to X. X,∈ Ensure the disturbance strength meets ||x adv -x|| ∞ ≤∈.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-4.

7. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-4.

Images