Uncertainty-driven pixel-level anti-noise detection method in remote sensing image

Abstract

The invention discloses a pixel-level anti-noise detection method driven by uncertainty in a remote sensing image. For an input remote sensing image to be detected, image information is extracted through a Monte Carlo batch normalization feature extraction network, and batch normalization training is equivalent to variational inference in a Bayesian model, so that the uncertainty of image pixels can be estimated and modeled under the condition that a network architecture and a training process are not modified. And calculating a mean square error by using an output image calculated by the feature extraction network, and judging whether the input image belongs to an adversarial sample or not according to the mean square error. And if the sample is an adversarial sample, pixel-level adversarial noise detection is carried out according to an uncertainty modeling result obtained by Monte Carlo sample calculation. The method can be suitable for various types of anti-noise, and can accurately locate the distribution position of the noise in the image.

Description

Technical Field

[0001] The present invention relates to the technical field of pattern recognition and machine learning, and in particular to a pixel-level anti-noise detection method driven by uncertainty in remote sensing images. Background Art

[0002] With the continuous development of remote sensing technology, remote sensing images are increasingly widely used in various fields. However, in the process of remote sensing image acquisition and transmission, due to the limitations of sensor performance, interference from other signals and the influence of the external environment, noise is often introduced, which reduces the quality of the image and limits its effectiveness and reliability in practical applications. In particular, the emergence of adversarial noise has brought serious security risks to the application of remote sensing images. The so-called adversarial noise refers to the imposition of small perturbations on the original samples. Although these perturbations will not affect human visual recognition, they may have a significant impact on machine learning models, causing the model to make wrong judgments. Adversarial sample detection, as a corresponding defense strategy, can determine whether the input image belongs to an adversarial sample, thereby refusing to input or output abnormal samples, which is of great significance to the improvement of the reliability of remote sensing data.

[0003] Remote sensing images usually have a large data size due to their wide coverage and high resolution, and the images often contain rich information, including various types of features such as terrain, vegetation, water bodies, and buildings. Therefore, adversarial noise is not evenly distributed throughout the image, but may appear in certain key areas, such as important target objects or specific areas of interest. Therefore, it is particularly important to design pixel-level detection algorithms that are adaptable to various types of adversarial noise for remote sensing images. On the one hand, the pixel-level detection algorithm can accurately locate the distribution of noise in the image, avoid excessive processing of noise-free areas, and thus reduce unnecessary information loss. On the other hand, clarifying the spatial distribution of noise also improves the transparency of adversarial sample identification and provides a clearer basis for subsequent analysis and decision-making.

[0004] Existing adversarial sample detection methods can be divided into three categories: detection based on feature learning, detection based on distribution statistics, and detection based on intermediate output:

[0005] Adversarial sample detection methods based on feature learning use the feature differences between adversarial samples and normal samples for detection. This type of method regards the subtle differences between normal samples and adversarial samples as noise, first denoises the input sample, and then uses the samples before and after denoising as model input. If the model output results vary greatly between the two times, it means that the input sample may be an adversarial sample. However, this type of feature learning-based method focuses on the data itself, which limits the improvement of the model's generalization.

[0006] The core idea of ​​adversarial sample detection methods based on distribution statistics is to use the different distribution characteristics of adversarial samples and normal samples to calculate the distribution of model inputs and compare them with clean samples to determine whether the original input is an adversarial sample. This type of method has a large amount of calculation and is not suitable for large data sets.

[0007] Since there are significant differences in the output states of the intermediate layers of deep neural networks when processing normal samples and adversarial samples, the detection method based on intermediate output uses the network intermediate output features as the input of the detector to achieve the recognition and detection of adversarial samples. Although the existing adversarial sample detection methods based on intermediate outputs show excellent performance in specific scenarios, the detection algorithm may fail and have poor generalization performance when dealing with multiple attacks. In addition, the current detection method can only determine whether the input image is an adversarial sample, but cannot accurately locate the specific area of ​​adversarial noise in the image. Remote sensing images are large in size, and their noisy areas may only exist on certain key targets in the image.

[0008] Therefore, how to provide a pixel-level anti-noise detection method that is applicable to a variety of anti-noise types and can accurately locate the distribution position of noise in an image is a technical problem that needs to be urgently solved by those skilled in the art. Summary of the invention

[0009] In view of the problems existing in the prior art, the present invention combines the anti-noise characteristics and the remote sensing image uncertainty modeling method to propose a pixel-level anti-noise detection method applicable to various anti-noise types, which effectively realizes the regional positioning of anti-noise in the image.

[0010] The present invention provides a method for detecting pixel-level noise in remote sensing images driven by uncertainty, comprising the following steps:

[0011] Network training steps:

[0012] Construct a training sample set, including remote sensing image input samples and corresponding remote sensing image denoising output samples;

[0013] Constructing a feature extraction network for training, including a feature extraction layer and a batch normalization layer, wherein feature vectors extracted by the feature extraction layer from the current batch of remote sensing image input samples are input into the batch normalization layer;

[0014] estimating the mean and variance of the moving average of the feature vectors of the current batch during iterative training as a set of network random parameters, determining the loss function of the feature extraction network based on the network random parameters, and performing batch normalization training;

[0015] Network detection steps:

[0016] Adding noise perturbations to the set of network random parameters;

[0017] Based on the variational inference method of the Bayesian probability model, the remote sensing image to be detected is input into the trained feature extraction network, and random parameters with noise perturbations are added to generate Monte Carlo samples and their corresponding output prediction images, and the mean of the outputs of multiple batches of Monte Carlo samples is used as the posterior estimate of the feature extraction network;

[0018] Calculate the mean of the predicted images corresponding to multiple batches of Monte Carlo samples as the estimated result of the network output image, and compare the mean square error of the predicted images corresponding to multiple batches of Monte Carlo samples with the preset mean threshold to determine the output adversarial sample;

[0019] For the adversarial sample, the variance of the corresponding predicted image is calculated as the network uncertainty map estimation result, and the distribution area of ​​the output adversarial noise is determined by comparing it with a preset variance threshold.

[0020] Preferably, it also includes:

[0021] In the network training phase, a network random parameter set is constructed based on network random parameters from different batches. In the network detection phase, the network random parameter set is randomly sampled and attached to the remote sensing image to be detected to generate Monte Carlo samples.

[0022] Preferably, it also includes:

[0023] Based on the variational inference method of the Bayesian probability model, the normalization training process of the feature extraction network is equivalent to the variational inference process of the Bayesian probability model.

[0024] Preferably, the loss function of the feature extraction network is:

[0025]

[0026] Among them, the training data set D = {(x i ,y i )} i＝1:N , where x i is the remote sensing image input sample, y i is the output sample of the corresponding remote sensing image after denoising; θ is the learnable parameter of the feature extraction network; ω i is the sample x in the network training phase i The mean and variance of the mini-batch data, i.e., the random parameters; is the decision function of the feature extraction network; Ω(θ) is the random parameter ω i The regularization term of .

[0027] Preferably, the uncertainty map segmentation threshold that maximizes the inter-class variance of the predicted image is obtained based on the maximum inter-class variance method as the variance threshold.

[0028] Preferably, the feature extraction network adopts VGG19 as the basic network architecture, including 19 groups of convolution Conv and activation function ReLu, and a batch normalization layer BN is added to each group.

[0029] Compared with the prior art, it has the following beneficial effects:

[0030] This paper proposes an uncertainty-driven pixel-level adversarial noise detection method in remote sensing images. By introducing uncertainty modeling methods, the algorithm can more accurately detect adversarial noise areas in images. In particular, the algorithm can effectively detect and locate subtle adversarial noise that is difficult to identify with traditional methods.

[0031] Due to the use of the uncertainty modeling method based on Monte Carlo batch normalization, the algorithm can estimate and model the uncertainty of image pixels without changing the existing network structure, which improves the robustness of the algorithm. Even in the face of different types of adversarial attacks, the algorithm can maintain good detection performance.

[0032] This algorithm is applicable to various types of remote sensing images, and has strong generalization performance for adversarial noise generated by various attack methods. The algorithm is flexible in design and can adapt to different application scenarios and needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work.

[0034] Figure 1 A flow chart of a method for detecting pixel-level anti-noise in remote sensing images driven by uncertainty according to an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of a method for detecting pixel-level noise in remote sensing images driven by uncertainty according to an embodiment of the present invention;

[0036] Figure 3 Different types of adversarial samples and their uncertainty graphs provided by embodiments of the present invention;

[0037] The corresponding figure (a) is an adversarial sample and its uncertainty map in the remote sensing image of a baseball field provided by an embodiment of the present invention;

[0038] The corresponding figure (b) is an adversarial sample and its uncertainty map in the remote sensing image of the tank area provided by an embodiment of the present invention;

[0039] Figure 4 Different categories of noise-resistant pixel-level detection results provided by the embodiments of the present invention;

[0040] The corresponding figure (a) is the pixel-level detection result of anti-noise in the remote sensing image of the baseball field provided by the embodiment of the present invention;

[0041] The corresponding figure (b) is the noise-resistant pixel-level detection result in the remote sensing image of the tank area provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Based on the analysis and discovery of various types of adversarial noise characteristics in remote sensing images, the present invention proposes an uncertainty-driven pixel-level adversarial noise detection algorithm to address the problems existing in the current adversarial sample detection algorithm. The algorithm adopts an uncertainty modeling method based on Monte Carlo batch normalization, and uses the characteristic that adversarial noise areas usually have higher uncertainty than normal areas to achieve pixel-level detection.

[0044] For the input remote sensing image to be detected, the image information is first extracted through the Monte Carlo batch normalization feature extraction network. The feature extraction network is used to extract features from low-quality remote sensing images to obtain high-quality remote sensing images. Batch normalization training is equivalent to variational inference in the Bayesian model, so the uncertainty of image pixels can be estimated and modeled without modifying the network architecture and training process. Then, the mean square error is calculated using the output image calculated by the feature extraction network, and the mean square error is used to determine whether the input image is an adversarial sample. If it is an adversarial sample, pixel-level adversarial noise detection is performed based on the uncertainty modeling results calculated by the Monte Carlo sample.

[0045] like Figure 1 As shown, the uncertainty-driven pixel-level anti-noise detection method in remote sensing images disclosed in the embodiment of the present invention is specifically implemented in the following steps:

[0046] Network training steps:

[0047] Construct a training sample set, including remote sensing image input samples and corresponding remote sensing image denoising output samples; construct a feature extraction network for training, including a feature extraction layer and a batch normalization layer, and input the feature vectors of the current batch of remote sensing image input samples extracted by the feature extraction layer into the batch normalization layer; estimate the mean and variance of the moving average of the current batch of feature vectors during iterative training as a set of network random parameters, determine the loss function of the feature extraction network based on the network random parameters, and perform batch normalization training;

[0048] Network detection steps:

[0049] Noise disturbance is added to a set of random parameters of the network; based on the variational inference method of the Bayesian probability model, the remote sensing image to be detected is input into the trained feature extraction network, and random parameters with noise disturbance are attached to generate Monte Carlo samples and their corresponding output prediction images, and the mean of the outputs of multiple batches of Monte Carlo samples is used as the posterior estimate of the feature extraction network; the mean of the prediction images corresponding to multiple batches of Monte Carlo samples is calculated as the estimation result of the network output image, and the mean square error of the prediction images corresponding to multiple batches of Monte Carlo samples is compared with a preset mean threshold to determine the output adversarial sample; for the adversarial sample, the variance of the corresponding prediction image is calculated as the estimation result of the network uncertainty map, and the distribution area of ​​the output adversarial noise is determined by comparing it with a preset variance threshold.

[0050] like Figure 2 As shown in the figure, the pixel-level adversarial noise detection algorithm based on uncertainty modeling consists of three parts: feature extraction network, adversarial sample discrimination and adversarial noise pixel-level detection. The input image is first subjected to feature extraction, and then it is judged whether the image belongs to an adversarial sample based on the extracted features. If it does, the image uncertainty map is used to further perform pixel-level adversarial noise detection.

[0051] Usually, higher uncertainty is observed in high-frequency areas such as the edges of an image, but the adversarial noise in adversarial samples creates false texture details in the image, resulting in higher uncertainty, and the uncertainty in the adversarial noise area increases with the increase in noise intensity. Therefore, by selecting a suitable threshold to segment the uncertainty map, we can obtain the specific areas where the adversarial noise may exist, thereby achieving pixel-level detection of adversarial noise.

[0052] In one embodiment, the feature extraction network uses VGG19 as the basic network architecture, including 19 groups of convolution Conv and activation function ReLu, and a batch normalization layer BN is added to each group.

[0053] The embodiment of the present invention is based on the idea of ​​the variational inference process in the equivalent Bayesian model of neural network batch normalization training, which can be proved by network model derivation:

[0054] For neural network based feature extraction network:

[0055] Known training data set D = {(x i ,y i )} i＝1:N , where x i is the input image, y i For the corresponding label, the neural network can learn the decision function f ω (x, y), where ω is the parameter of the neural network. For the deterministic model, the label of the test sample is directly obtained by the decision function, as shown in the following formula:

[0056]

[0057] For the Bayesian probability estimation model:

[0058] Write the decision function as the conditional probability distribution f ω (x,y)=p(y|x,ω), and the network parameters are regarded as probability distribution p(ω|D). The probability prediction of the obtained test sample is shown in formula (2):

[0059] p(y|x,D)=∫f ω (x,y)p(ω|D)dω=E p(ω|D) [p(y|x,ω)] (2)

[0060] In the formula, E p(ω|D) [.] indicates the expected operation on p(y|x,ω).

[0061] According to formula (2), the predicted label The Monte Carlo method can be used to approximate the prediction probability of the Bayesian neural network, that is, sampling multiple parameter sets from the posterior distribution p(ω|D) and averaging all the prediction results to get the predicted label

[0062] Since the distribution of parameters p(ω|D) is difficult to solve directly, the Bayesian model adopts the variational inference method (Variational Inference) and uses an approximate distribution q θ (ω) to estimate p(ω|D), and optimize by minimizing the Kullback-Leibler (KL) divergence between the two to approximate the distribution q θ (ω) is usually chosen to be a simple and easy-to-compute distribution that makes the optimization process feasible, such as independent Gaussian distribution, factorized Gaussian distribution, or Bernoulli distribution. The optimization objective is equivalent to minimizing the following loss function:

[0063]

[0064] According to the Monte Carlo estimation method, for each remote sensing image input sample, the corresponding network parameter ω is selected. The optimization objective on a small batch of samples of size M is equivalent to the following loss function:

[0065]

[0066] It should be noted that the Monte Carlo method is a statistical simulation method based on probability theory. The basic idea is to use random numbers to estimate problems that are difficult to solve directly, including a large number of random sampling and gradual approximation processes. Monte Carlo methods can be divided into two categories: one is that the problem to be solved can be converted into a characteristic number about a certain random distribution, such as the expected value of a random variable, the probability of a random event, etc. Through a large number of random sampling, the frequency of occurrence of random events is used to estimate the probability of their occurrence, and the digital features of the samples are used to estimate the digital features of random variables, thereby approaching the true solution to the problem. The other is that there is randomness in the problem to be solved itself, and the computing power of the computer is used to simulate various random processes in the problem to obtain a solution to the problem. The uncertainty modeling process in this embodiment belongs to the former, using noise to perturb the random parameters, attaching them to the remote sensing image to be detected to obtain Monte Carlo samples, and using the mean of these sample outputs as the posterior estimate of the neural network.

[0067] In one embodiment, it further includes:

[0068] In the network training phase, a network random parameter set is constructed based on network random parameters from different batches. In the network detection phase, the network random parameter set is randomly sampled and attached to the remote sensing image to be detected to generate Monte Carlo samples.

[0069] In one embodiment, the batch normalization step includes:

[0070] For a deep neural network with L layers, the decision function can be expressed as formula (5):

[0071] f ω (x) = W L a(W L-1 …a(W2a(W1x)) (5)

[0072] In the formula, function a(*) is a nonlinear activation function, W L is the weight vector of the Lth layer of the network. Each layer of the neural network consists of several neurons, and the batch normalization operation is performed on a single neuron. For a set of input small batch data B = {x1, x2, ..., x m}, calculate the mean μ of the data B and variance The data is then normalized, as shown in formula (6) and formula (7):

[0073]

[0074] In the formula, ε is a small positive number used to avoid the denominator being 0, and γ and β as restoration parameters retain the distribution of the original data to a certain extent and participate in the training of the entire network. Batch normalization can not only speed up the convergence of the model, but also alleviate the gradient diffusion problem in deep neural networks to a certain extent, making the network training more stable.

[0075] The loss function for training a deep neural network using batch normalization can be expressed as formula (8):

[0076]

[0077] In the formula, the first term is the loss on the training data set, and the second term is the regularization term of the network parameter ω. For the cross entropy loss of the classification problem or the mean square error loss of the regression problem, the first term is equivalent to the negative log-likelihood, so the loss function is equivalent to formula (9):

[0078]

[0079] In a neural network with a batch normalization layer, the model parameters include the weights W L , the scaling parameter γ L , the offset parameter β L , the mean μ of the moving average generated by the iterative training process of each batch of remote sensing image input samples B and variance L represents the Lth layer in the network, where the learnable parameters of the neural network include θ = {W L ,γ L ,β L}, mean and variance of batch normalized data in each layer is a non-trained random parameter used for normalization during inference. The loss function can be rewritten as formula (10):

[0080]

[0081] In the formula, ω i is the sample x during training i After introducing the batch normalization layer, the neural network parameters can be divided into learnable parameters θ and non-learnable parameters ω. The decision function is represented as Therefore, the uncertainty of predictions can be estimated from the trained neural network by leveraging the inherent stochasticity of the batch normalization layer and the non-learnable parameters.

[0082] In one embodiment, batch normalization training is equivalent to variational inference in the Bayesian model in that:

[0083] The equivalence of batch normalization and variational inference during neural network training requires that the loss functions (4) and (10) be equivalent, as shown in equation (11):

[0084]

[0085] In the formula, K is the proportionality coefficient. In the variational inference process, formula (4) shows that KL(q θ The contribution of (ω)||p(ω)) to the optimization objective is inversely proportional to the size of the training dataset N. For the batch normalization network, formula (10) shows that the corresponding regularization term Ω(θ) is smaller when N is larger. Therefore, when N is large or tends to infinity, the corresponding regularization term can be ignored, and the optimization objectives of variational inference and batch normalization are equivalent.

[0086] Use batch normalization for uncertainty estimation, using random batches of the same size B to estimate random parameters during training Create multiple sets of random parameters for different batches. Specifically, during inference, noise perturbations are added to the random parameters to obtain Where S is the noise of Gaussian distribution. In the test process, these random parameters are added to the remote sensing image to be detected to generate Monte Carlo (MC) samples, and the remote sensing image to be detected h is converted into MC samples. And a sample is generated for a set of random parameters. Therefore, one input sample can generate multiple MC samples, and the mean of these MC samples is used as the output result of the network, and the variance is used as the uncertainty estimation result.

[0087] In one embodiment, a Monte Carlo sample corresponding to the test image is obtained through a feature extraction network, and the mean of its output is used as an estimate of the network output image. The mean square error value is calculated based on the network output result image and the input test image. Compared with the clean sample, the mean square error value of the adversarial sample is higher, so the mean square error can be used as the classification basis of the adversarial sample. A suitable threshold is selected based on the training set data. During the test, the mean square error of the sample is compared with the threshold. If it is greater than the threshold, it is judged as an adversarial sample, otherwise it is judged as a clean sample.

[0088] In one embodiment, the uncertainty map segmentation threshold that maximizes the inter-class variance of the predicted image is obtained based on the maximum inter-class variance method as the variance threshold. In the specific implementation, the adversarial sample is selected, and the corresponding uncertainty estimation result is calculated using the feature extraction network. Since the pixel value on the uncertainty map corresponding to the area where the adversarial noise exists is large, a suitable threshold is selected to segment the uncertainty map to obtain the specific area where the adversarial noise may exist, thereby achieving pixel-level detection of the adversarial noise. The threshold segmentation and morphological processing process of the uncertainty map is as follows.

[0089] The maximum inter-class variance method is a threshold segmentation algorithm widely used in image binarization, also known as the Otsu method. Its basic idea is to set a threshold so that the inter-class variance between the foreground and background after segmentation is maximized. Assuming that there is a threshold θ that divides all pixels in the image into two categories, and the probabilities of the pixels being divided into two categories are p1 and p2 respectively, and the means of the two categories of pixels are m1 and m2 respectively, then the mean of all pixels in the image is m=p1m1+p2m2. The variance between the two categories of pixels is shown in formula (12):

[0090] σ 2 =p1(m1-m) 2 +p2(m2-m) 2 (12)

[0091] Simplifying the above formula, we can get formula (13):

[0092] σ 2 =p1p2(m1-m2) 2 (13)

[0093] According to the above formula, find the variance σ between classes. 2 The maximum threshold θ is the segmentation threshold of the maximum inter-class variance method.

[0094] In this embodiment, in order to further improve the accuracy of the detection results, morphological processing is applied to the results after the uncertainty map threshold segmentation, including erosion and dilation operations. First, an appropriate structural element (similar to a convolution kernel) is selected and used to traverse each pixel in the image. In the erosion operation, the structural element and the image of the area it covers are bitwise ANDed. If the result is all 1, the pixel is set to 1, otherwise it is set to 0; in the dilation operation, the structural element and the image part are bitwise ORed. If the result is all 0, the pixel is set to 0, otherwise it is set to 1. Through the erosion and dilation operations, the point noise in the non-adversarial noise area of ​​the segmentation result image can be eliminated, and the holes inside the adversarial noise area can be filled, thereby improving the accuracy of the detection results.

[0095] The technical effects of the present invention are described below in conjunction with specific experimental examples:

[0096] The present invention uses the publicly available optical remote sensing image dataset DIOR, which contains 23,463 images with a resolution of 800×800 and 190,288 instances. The spatial resolution ranges from 0.5m to 30m, covering 20 target categories. Based on DIOR, four typical adversarial attack algorithms, FGSM, BIM, DeepFool and AdvGAN, are used to add adversarial noise to key areas with targets in the image to create a remote sensing image dataset with adversarial noise, providing a data basis for the experiment. The experiment uses accuracy, precision P (Precision), recall R (Recall) and F1 score as evaluation indicators of the experiment.

[0097] In order to demonstrate the superiority of the method of the present invention, the method of the present invention was experimented on remote sensing images generated by four adversarial attack algorithms: FGSM, BIM, DeepFool and AdvGAN. The experimental results of adversarial sample discrimination are shown in Table 1. As can be seen from the table, the method of the present invention achieved good detection results in the data generated by these four adversarial attack methods, and the detection accuracy was all above 0.87, which shows that the method of the present invention has strong generalization performance for different adversarial attack methods. It is worth noting that although the adversarial samples processed by the DeepFool method are more difficult to detect visually, the evaluation indicators of the method of the present invention did not show a significant decrease when detecting such adversarial samples, which further proves the generalization performance of the method of the present invention on a variety of adversarial attack methods, as well as its strong application prospects for other adversarial attack methods that may appear in the future.

[0098] Table 1 The adversarial sample discrimination results of the present invention for different adversarial sample generation methods

[0099]

[0100] To further verify the effectiveness of the method of the present invention, a comparative experiment was conducted on the DeepFool method, which has a better attack effect and is more difficult to detect visually. The method of the present invention was compared with the current adversarial sample detection algorithms MAD and PACA. The experimental results are shown in Table 2, and the optimal indicators are shown in bold. From the comparative experimental results, it can be seen that the method of the present invention has achieved the highest accuracy, precision and F1 score. Although the recall rate is slightly lower than the other two methods, it still shows excellent detection performance compared with the other two methods.

[0101] Table 2 The adversarial sample discrimination results of the present invention for different adversarial sample generation methods

[0102]

[0103] Existing adversarial sample identification methods cannot detect areas where noise exists, and can only classify the input image into two categories to determine whether it is an adversarial sample, and are usually only applicable to a specific adversarial attack method. The adversarial noise detection method based on uncertainty modeling designed by the present invention can detect pixel-level noise in adversarial samples on the basis of identifying adversarial samples, and has strong generalization performance for adversarial noise generated under various attack methods. Therefore, pixel-level adversarial noise detection is further performed on the screened adversarial samples to obtain uncertainty maps and pixel-level detection results as shown in the figure. Figure 3 and Figure 4 As shown. Figure 3 As shown in the figure, the baseball field and storage tank areas in the image contain adversarial noise, and these areas will show higher uncertainty than the background. Accordingly, the pixel values ​​corresponding to these areas on the uncertainty map are significantly higher than other parts. This shows that the feature extraction part of the method of the present invention is not robust enough in the prediction of adversarial noise areas and will be disturbed by adversarial noise, resulting in increased uncertainty in the output results. By capturing these pixel areas with high uncertainty, the algorithm can effectively locate the existence of adversarial noise, providing strong support for subsequent noise processing and model defense.

[0104] like Figure 4 As shown, the detection result classifies the area with a pixel value of 0 in the image as a clean pixel, and the area with a pixel value of 255 as a noisy pixel. This binary representation clearly divides the distribution area of ​​the adversarial noise, which helps to intuitively display the spatial position of the noise. As can be seen from the figure, the method of the present invention can be applied to various types of adversarial noise, and shows a high accuracy in the detection of adversarial noise in baseball fields and storage tank areas, and can effectively detect most of the noise areas in the image. However, due to the presence of some high-frequency areas with complex textures in the image background, the uncertainty of these areas is also high.

[0105] In addition, the present invention quantitatively evaluates the pixel-level detection results of the adversarial noise on the test set. As shown in Table 3, it can also be seen from the table that the method of the present invention can be applied to various types of adversarial noise, and has a high detection accuracy, and has a certain generalization performance on different types of adversarial noise.

[0106] Table 3 Quantitative evaluation results of different categories of anti-noise pixel-level detection

[0107]

[0108] The above is a detailed introduction to the uncertainty-driven pixel-level anti-noise detection method in remote sensing images provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0109] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. An uncertainty-driven pixel-level adversarial noise detection method in remote sensing images, characterized in that: The steps include: Network training steps: Construct a training sample set, including remote sensing image input samples and corresponding remote sensing image denoising output samples; Constructing a feature extraction network for training, including multiple feature extraction layers and a batch normalization layer, wherein feature vectors extracted by the feature extraction layer of the current batch of remote sensing image input samples are input into the batch normalization layer; estimating the mean and variance of the moving average of the feature vectors of the current batch during iterative training as a set of network random parameters, determining the loss function of the feature extraction network based on the network random parameters, and performing batch normalization training; Network detection steps: Adding noise perturbations to the set of network random parameters; Based on the variational inference method of the Bayesian probability model, the remote sensing image to be detected is input into the trained feature extraction network, and random parameters with noise perturbations are added to generate Monte Carlo samples and their corresponding output prediction images, and the mean of the outputs of multiple batches of Monte Carlo samples is used as the posterior estimate of the feature extraction network; Calculate the mean of the predicted images corresponding to multiple batches of Monte Carlo samples as the estimated result of the network output image, and compare the mean square error of the predicted images corresponding to multiple batches of Monte Carlo samples with the preset mean threshold to determine the output adversarial sample; For the adversarial sample, the variance of the corresponding predicted image is calculated as the network uncertainty map estimation result, and the distribution area of ​​the output adversarial noise is determined by comparing it with a preset variance threshold.

2. The uncertainty-driven pixel-level anti-noise detection method in remote sensing images according to claim 1, characterized in that: Also includes: In the network training phase, a network random parameter set is constructed based on network random parameters from different batches. In the network detection phase, the network random parameter set is randomly sampled and attached to the remote sensing image to be detected to generate Monte Carlo samples.

3. The uncertainty-driven pixel-level anti-noise detection method in remote sensing images according to claim 1, characterized in that: Also includes: Based on the variational inference method of the Bayesian probability model, the normalization training process of the feature extraction network is equivalent to the variational inference process of the Bayesian probability model.

4. The uncertainty-driven pixel-level anti-noise detection method in remote sensing images according to claim 1, characterized in that: The loss function of the feature extraction network is: Among them, the training data set D = {(x i ,y i )} i＝1:N , where x i is the remote sensing image input sample, y i is the output sample of the corresponding remote sensing image after denoising; θ is the learnable parameter of the feature extraction network; ω i is the sample x in the network training phase i The mean and variance of the mini-batch data, i.e., the random parameters; f {θ,ωi} (x i ,y i ) is the decision function of the feature extraction network; Ω(θ) is the random parameter ω i The regularization term of .

5. The uncertainty-driven pixel-level anti-noise detection method in remote sensing images according to claim 1, characterized in that: Based on the maximum inter-class variance method, an uncertainty map segmentation threshold that maximizes the inter-class variance of the predicted image is obtained as the variance threshold.

6. The uncertainty-driven pixel-level anti-noise detection method in remote sensing images according to claim 1, characterized in that: The feature extraction network adopts VGG19 as the basic network architecture, including 19 groups of convolution Conv and activation function ReLu, and a batch normalization layer BN is added to each group.

Images