Transferable adversarial attack method for hyperspectral image classification

Abstract

The invention discloses a migratable adversarial attack method for hyperspectral image classification, and aims to generate an adversarial sample with mobility by using a white-box model so as to realize effective attack on a target black-box model.The method comprises the steps that a white-box hyperspectral image classification model is selected as a source model; calculating the gradient of each wave band through a back propagation algorithm, and determining the importance score of the wave band; calculating the pixel shielding probability of the wave band based on the importance score, and carrying out random pixel shielding on the image; aggregating the gradient information under multiple times of random shielding to obtain an aggregated gradient value; combining the aggregation gradient and the feature matrix to design a loss function, and guiding generation of an adversarial sample; and finally, applying the generated adversarial sample to a black box model, and implementing a migration attack. The adversarial sample generated by the method has high mobility, universality and adaptability, and a new thought is provided for safety research in the field of hyperspectral image classification.

Description

Technical Field

[0001] The present invention belongs to the field of adversarial attacks for image classification, and particularly relates to a transferable adversarial attack method for hyperspectral image classification. Background Art

[0002] Hyperspectral images are a form of remote sensing data containing rich spectral information. Each pixel of a hyperspectral image has spectral information in multiple bands and is commonly used in fields such as land cover classification, environmental monitoring, and precision agriculture. In the hyperspectral image classification task, a model identifies the class to which a pixel belongs by analyzing the spectral features of the pixel in different bands. In recent years, deep learning techniques have been widely applied to the hyperspectral image classification task. These models can capture the complex relationships between spectral and spatial features and have achieved significant improvements in classification accuracy. However, the high dimensionality of hyperspectral data and the redundancy of spectral bands make these models very sensitive to noise and adversarial perturbations.

[0003] Adversarial attack is a technique that misleads a model by adding small perturbations to the input data. In hyperspectral image classification, adversarial attacks can not only reduce the classification accuracy of the model for the input but also reveal the vulnerability of the model, thus providing a basis for improving the robustness of the model. However, existing research on adversarial attacks mainly focuses on the generation of adversarial samples in the RGB domain. These studies usually utilize the spatial characteristics of RGB images and mislead the classification model by adding small perturbations. However, there are significant differences in data characteristics between RGB images and hyperspectral images. Hyperspectral images have higher spectral resolution and multi-band characteristics, and their classification models are more dependent on the inter-band correlation and spectral features. Therefore, adversarial attack methods for RGB images are difficult to directly transfer to hyperspectral images.

[0004] A transfer attack refers to generating adversarial examples on one model and applying them to other models, causing cross-model adversarial interference. The generated adversarial examples have cross-model attack effects, with strong generality and adaptability, and are applicable to multiple models. However, current transferable adversarial attack methods for hyperspectral image classification have many deficiencies and challenges. (1) The complexity problem of high-dimensional data: Hyperspectral images usually contain spectral information of hundreds of bands, making their data dimensions much higher than those of conventional RGB images. Most traditional adversarial example generation methods are designed for low-dimensional images and often face problems such as high computational resource consumption and high optimization difficulty when dealing with high-dimensional data. The high-dimensional feature space increases the diversity and complexity of perturbations, making the generation of adversarial examples more difficult. (2) The transferability problem: Due to differences in the structures of different models, data biases during the training process, and different feature learning methods, the transferability of adversarial examples is poor. Therefore, how to design an adversarial example generation strategy that can be effectively transferred between different hyperspectral image classification models remains a major challenge. (3) The efficiency problem of adversarial example generation: Generating adversarial examples usually requires a large number of iterative optimizations. For hyperspectral images, due to their high dimensions and complex spectral characteristics, the process of generating adversarial examples usually requires a large amount of computational resources and time. In addition, the models for hyperspectral image classification usually contain multiple levels and complex feature mappings, resulting in adversarial perturbations not being simply pixel-level modifications but requiring fine-tuning at a higher level, which makes the process of generating adversarial examples more time-consuming and computationally intensive. (4) The generality and adaptability problem: There are significant differences between different hyperspectral image classification tasks and datasets. For example, different object categories, sensor types, and data acquisition conditions, etc., will all affect the performance of the model. This makes some adversarial examples designed for specific datasets or tasks may not be directly applicable to other tasks or datasets. Therefore, how to design general and highly adaptable adversarial attack methods that can adapt to a variety of different hyperspectral image classification tasks has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of the present invention is to provide a transferable adversarial attack method for hyperspectral image classification. By randomly pixel masking different bands of the hyperspectral image to destroy the model-specific noise, and aggregating the gradients after random pixel masking to obtain the aggregated gradient, and using the aggregated gradient to guide the generation of adversarial examples to improve the transferability of the adversarial examples.

[0006] The technical solution adopted by the present invention to solve the above technical problems is as follows: A transferable adversarial attack method for hyperspectral image classification, characterized by comprising the following steps:

[0007] Step 1: For the hyperspectral image x in the given dataset, select a white-box hyperspectral image classification model as the source model, calculate the gradient values of each band of the hyperspectral image using the backpropagation algorithm, and take the average gradient value of each band as the importance score I of the band b , which is used to measure the influence of each band on the classification decision, where b represents the b-th band;

[0008] Step 2: According to the feature importance score I b , set the corresponding masking probability P b for different bands, so as to focus on perturbing the high-importance bands. According to P b , generate k random pixel masking matrices M b , and use M b to randomly mask the input hyperspectral image respectively to obtain the randomly pixel-masked image where i ∈ {1, 2,..., k};

[0009] Step 3: Use the backpropagation algorithm to calculate the gradients of all the images after random pixel masking through the loss function of the hyperspectral image classification model to obtain the corresponding gradient values G (i) , and aggregate the k obtained gradient values to obtain the aggregated gradient value G agg :

[0010]

[0011] Step 4: Calculate the loss function through the element-wise dot product of the aggregated gradient G agg and the hyperspectral image feature matrix X, and use this loss function to guide the generation of adversarial samples;

[0012] Step 5: Repeat Step 3 and Step 4 to continuously optimize the adversarial samples until the established attack goal is met or the maximum number of iterations is reached, and use the generated adversarial samples to attack the black-box model.

[0013] Furthermore, the specific process of Step 1 is as follows:

[0014] Step 1.1: Randomly select an image from the hyperspectral image dataset and input it into the model, and calculate the gradient G of each node feature through the loss function by backpropagation:

[0015]

[0016] where L is the loss function and X is the feature matrix of the hyperspectral image;

[0017] Step 1.2: Take the absolute value of the gradient of each band and then average it to use as the importance score I of the bandb 。

[0018] Further, the specific process of step 2 is as follows:

[0019] Step 2.1: Normalize the feature importance score I b to the range [0, 1], and set the masking probability P for different bands according to the normalized feature importance score I b : b :

[0020]

[0021] Step 2.2: Set k three-dimensional matrices with the same dimension as the hyperspectral image feature matrix X, and initialize the values in the three-dimensional matrices using random numbers sampled from the uniform distribution (0, 1); compare the values in the three-dimensional matrices with the masking probability P b ; when the value in the three-dimensional matrix is greater than or equal to P b , set the value in the three-dimensional matrix to 1; when the value in the three-dimensional matrix is less than P b , set the value in the three-dimensional matrix to 0; generate k random pixel masking matrices M b ; generate the masked image by taking the element-wise dot product of M b and X.

[0022] Further, in step 4, design a dot product loss function of gradient and feature based on the aggregated gradient G agg and the feature matrix X:

[0023]

[0024] Construct the final loss function by combining with the cross-entropy loss function:

[0025] L = α·L(X) + β·L ce

[0026] where α and β are weight hyperparameters, and L ce is the cross-entropy loss function; generate the adversarial sample x adv using the loss function L:

[0027]

[0028] where η is the learning rate, is the gradient of the total loss with respect to X, t is the number of iterations, x is the input image, ∈ is the perturbation size, and Clip X,∈ ensures that the perturbation intensity satisfies ‖x adv - x‖ ∞ ≤ ∈.

[0029] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the above method are implemented.

[0030] A computer-readable storage medium stores a computer program thereon. When the program is executed by a processor, the steps of the above method are implemented.

[0031] A computer program product includes a computer program. When the computer program is executed by a processor, the steps of the above method are implemented.

[0032] Compared with the prior art, the significant advantages of the present invention are as follows:

[0033] (1) The present invention calculates the gradients of each band through the backpropagation algorithm and uses their average value as the importance score of the band, thereby quantifying the influence of each band on the classification decision; this mechanism ensures that when generating adversarial samples, perturbations are mainly targeted at the bands that have a greater impact on the classification decision, enabling a more precise attack strategy.

[0034] (2) The present invention adopts a random pixel masking technique when generating adversarial samples and highlights the important features of the image by setting the masking probability of each band; by focusing on perturbing the important bands and performing an element-wise dot product of the masking matrix and the image features, unnecessary computational overhead can be reduced while ensuring the attack effect.

[0035] (3) The present invention utilizes the loss function of the hyperspectral image classification model and optimizes the adversarial samples by combining the aggregated values of the gradients under different masking configurations; this strategy aggregates the gradients obtained under different random pixel masking configurations through weighting, destroying the specific noise features of the model, making the generated adversarial samples have strong transferability; compared with the local optimization methods in the prior art, the aggregated gradient calculation method of the present invention can better control the perturbations globally, ensuring the maximization of the adversarial attack effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flowchart for implementing the method of the present invention.

[0037] Figure 2 is a schematic diagram of the aggregated gradient of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0038] The following further describes the present invention in conjunction with the drawings and specific embodiments.

[0039] As Figure 1As shown in the figure, a transferable adversarial attack method for hyperspectral image classification according to the present invention suppresses the specific noise characteristics of the model by evaluating the importance of bands and randomly masking pixels, and retains the inherent characteristics of the hyperspectral image; and uses aggregated gradients to guide the generation of adversarial samples, enhancing the transferability of the adversarial samples. The specific steps are as follows:

[0040] Step 1: For the hyperspectral images in a given dataset, select a white-box hyperspectral image classification model as the source model. Randomly select an image x and input it into the model. Calculate the gradient G of each node feature of the loss function through backpropagation:

[0041]

[0042] where L is the loss function and X is the feature matrix of the hyperspectral image;

[0043] Then, take the absolute value of the gradient of each band of the input image x and average it to obtain the importance score I of this band b :

[0044]

[0045] where H and W are the height and width of the image respectively; G b (i,j) represents the value of the gradient matrix at band b and pixel position (i,j).

[0046] By calculating the importance scores of the bands of the hyperspectral image, it helps the model identify the bands that have the greatest impact on the classification task, so as to concentrate on perturbing these key bands in the adversarial attack and improve the efficiency and effect of the attack.

[0047] Step 2: According to the feature importance score I b , set the corresponding masking probability P b for different bands. First, normalize the feature importance score I b to the range [0,1], and calculate the masking probability P b according to the normalized feature importance score I b :

[0048]

[0049] Thus, key perturbations are made to the high-importance bands. Then, set k three-dimensional matrices with the same dimension as the hyperspectral image feature matrix X, and initialize the values in the three-dimensional matrices with random numbers sampled from the uniform distribution (0,1); compare the values in the three-dimensional matrices with the masking probability P b ; when the value in the three-dimensional matrix is greater than or equal to P b , set the value in the three-dimensional matrix to 1; when the value in the three-dimensional matrix is less than Pb , set the values in the three-dimensional matrix to 0; generate k random pixel masking matrices M through the above steps b ; by multiplying M b element-wise with X, obtain the randomly pixel-masked image

[0050]

[0051] where i ∈ {1, 2,..., k}, is the i-th random pixel masking matrix, X represents the feature matrix of the input image, and ⊙ represents element-wise multiplication.

[0052] Random pixel masking introduces uncertainty through randomness, breaks the noise characteristics unique to the model, and makes it difficult for the model to rely on certain specific pixel or band information. This can effectively avoid the overfitting of adversarial samples to the specific characteristics of the source model, thereby improving the transferability of adversarial samples.

[0053] Step 3: Using the backpropagation algorithm, calculate the gradients of all the images processed by random pixel masking through the loss function of the hyperspectral image classification model to obtain the corresponding gradient values G (i) , and aggregate the k gradient values obtained to get the aggregated gradient value G agg :

[0054]

[0055] The process of aggregating gradients is as Figure 2 shown, including the process of performing random pixel masking on the input image, calculating the masking probability according to the image band importance scores, and then generating the random pixel masking matrix multiplying these k random pixel masking matrices element-wise with the input image to obtain the randomly pixel-masked image performing backpropagation on the randomly pixel-masked image to calculate the gradient G (i) , and then aggregating these gradients to obtain the aggregated gradient, where i ∈ {1, 2,..., k}.

[0056] Step 4: Calculate the loss function through the element-wise multiplication of the aggregated gradient G agg and the hyperspectral image feature matrix X, and use this loss function to guide the generation of adversarial samples. First, design the dot product loss function of the gradient and the feature according to the aggregated gradient G agg and the feature matrix X:

[0057]

[0058] Then construct the final loss function by combining the cross-entropy loss function:

[0059] L = α·L(X) + β·L ce

[0060] where α and β are weight hyperparameters, and L ce is the cross - entropy loss function; an adversarial sample x is generated using the loss function L adv :

[0061]

[0062] where η is the learning rate, is the gradient of the total loss with respect to X, t is the number of iterations, x is the input image, ∈ is the perturbation magnitude, and clip X,∈ ensures that the perturbation intensity satisfies ||x adv - x|| ∞ ≤ ∈.

[0063] The loss function designed in this way utilizes the combination of aggregated gradients and classification objectives, and can more effectively generate adversarial samples with high transferability.

[0064] Step 5: Repeat Step 3 and Step 4, continuously optimize the adversarial sample until the established attack objective is met or the maximum number of iterations is reached, and use the generated adversarial sample to attack the black - box model.

[0065] The present invention calculates the band importance of hyperspectral images, and adopts a lower pixel masking probability for important bands, which can ensure that these bands containing key object information are retained, while more pixels can be discarded for less important bands. This helps to retain the key object features in the image; by calculating the gradients of the images under different random pixel masking configurations and aggregating the gradients, a comprehensive gradient representation can be obtained, and this aggregated gradient can better reflect the key changes in the image, thereby guiding the model to focus on more important features; the present invention effectively improves the transferability of adversarial samples and reveals the vulnerability of hyperspectral image classification models by generating adversarial samples through random pixel masking and aggregated gradient optimization.

Claims

1. A transferable adversarial attack method for hyperspectral image classification, characterized in that: The following steps are involved: Step 1: For a given hyperspectral image x in a data set, select a white box hyperspectral image classification model as the source model, use the back propagation algorithm to calculate the gradient value of each band of the hyperspectral image, and take the average gradient of each band as the importance score I of the band b , used to measure the impact of each band on the classification decision, where b represents the bth band; Step 2: According to the feature importance score I b , set the corresponding masking probability P for different bands b , so as to focus on disturbing the high-importance bands, according to P b Generate k random pixel masking matrices M b , using M b Perform random pixel masking on the input hyperspectral image to obtain the image after random pixel masking where i∈{1,2,...,k}; Step 3: Use the back propagation algorithm to classify all images processed with random pixel masking through the loss function of the hyperspectral image classification model. Perform gradient calculation and get the corresponding gradient value G (i) , and aggregate the obtained k gradient values ​​to obtain the aggregated gradient value G agg : Step 4: By aggregating the gradient G agg The loss function is calculated by the element-by-element dot product of the hyperspectral image feature matrix X, and the loss function is used to guide the generation of adversarial samples; Step 5: Repeat steps 3 and 4 to continuously optimize the adversarial sample until the established attack target is met or the maximum number of iterations is reached, and use the generated adversarial sample to attack the black box model.

2. The transferable adversarial attack method for hyperspectral image classification according to claim 1, characterized in that: The specific process of step 1 is as follows: Step 1.1: Randomly select an image from the hyperspectral image dataset and input it into the model. Calculate the gradient G of the loss function for each node feature through back propagation: Where L is the loss function, X is the feature matrix of the hyperspectral image; Step 1.2: Take the absolute value of the gradient of each band and average it as the importance score I of the band b .

3. The transferable adversarial attack method for hyperspectral image classification according to claim 1, characterized in that: The specific process of step 2 is: Step 2.1: Set the feature importance score I b Normalized to the range of [0,1], according to the normalized feature importance score I b Set the masking probability P of different bands b : Step 2.2: Set k three-dimensional matrices with the same dimension as the hyperspectral image feature matrix X, and use random numbers sampled from a uniform distribution (0, 1) to initialize the values ​​in the three-dimensional matrix; compare the values ​​in the three-dimensional matrix with the masking probability P b Compare; when the value in the three-dimensional matrix is ​​greater than or equal to P b , set the values ​​in the three-dimensional matrix to 1; When the value in the three-dimensional matrix is ​​less than P b , set the values ​​in the three-dimensional matrix to 0; generate k random pixel mask matrices M b ; By b Element-wise dot product with X to generate the masked image.

4. The transferable adversarial attack method for hyperspectral image classification according to claim 1, characterized in that: In step 4, according to the polymerization gradient G agg And feature matrix X to design the dot product loss function of gradient and features: L(X)=∑G agg ⊙X Combine the cross entropy loss function to construct the final loss function: L=α·L(X)+β·L ce Where α and β are weight hyperparameters, L ce is the cross entropy loss function; using the loss function L to generate adversarial samples x adv : Where η is the learning rate, is the gradient of the total loss with respect to X, t is the number of iterations, x is the input image, ∈ is the perturbation size, Clip X,∈ Ensure that the perturbation intensity satisfies ||x adv -x|| ∞ ≤∈.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images