A method for detecting environmental anomalies based on spectral diffusion

Abstract

The application discloses an environment anomaly detection method based on spectral diffusion, and aims at the field of hyperspectral image target detection, and proposes an anomaly spectrum generation method based on principal component analysis and linear spectral mixing model, solves the problem of insufficient paired label data existing and not existing anomalies, guarantees the effectiveness of learned features, and improves the reality of a traditional anomaly target generation method; a double-window spectral diffusion model is designed for hyperspectral data, diffusion processes of each pixel are regarded as spectral mixing of a target and surrounding background, the accuracy of spatial-spectral dimension estimation in a reconstruction process is improved, and the background estimation has high spatial and spectral accuracy; a double-window spectral diffusion background reconstruction method is introduced, spectrum is iteratively accumulated in the double window excluding the target, the influence of an anomaly expansion area on background estimation is reduced, and the target spectrum is effectively inhibited.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of hyperspectral image processing, and relates to a hyperspectral anomaly detection method, in particular to an environmental anomaly detection method based on spectral diffusion. BACKGROUND

[0002] Hyperspectral remote sensing captures tens to hundreds of continuous wavebands of spectral information and combines them with spatial data. This unique "image-spectrum integration" capability makes hyperspectral imaging widely used in fields such as agriculture, forestry, mineral exploration, environmental monitoring, and land cover investigation. Anomaly detection is an important branch of machine learning and has important applications in computer vision. In the context of hyperspectral imaging, the focus of anomaly detection is to identify observations that are inconsistent with the surrounding environment, and it is an active research field in hyperspectral image processing. Due to the lack of prior spectral and spatial information of the target, hyperspectral anomaly detection is a challenging task.

[0003] Currently, hyperspectral anomaly detection methods can be roughly divided into three methods: statistical modeling-based methods, representation learning-based methods, and deep learning-based methods. In recent years, deep learning-based methods have become increasingly prominent due to their ability to learn complex patterns from data. Among them, reconstruction-based methods have been proven feasible through extensive validation. However, these methods often emphasize the difficulty of reconstructing abnormal targets, while ignoring the spectral differences between abnormal targets and the surrounding background, which is a key aspect of anomaly detection.

[0004] Diffusion-based models have recently gained attention in the field of anomaly detection, mainly because they can generate high-quality, realistic images, more effectively capturing the complex distribution of the background, thereby improving the accuracy of anomaly detection. Although diffusion models have been applied to various hyperspectral image processing tasks, their application in hyperspectral anomaly detection is relatively limited. Existing diffusion models usually add spatial random noise in each spectral band of the original image without fully considering spectral dimension information, leading to potential inaccuracies in spectral reconstruction.

[0005] Considering the data-driven nature of deep learning and the high cost of acquiring hyperspectral images in real-world scenarios, especially the lack of image pairs containing / without abnormal targets in the same environment, this problem is particularly prominent. To address this issue, simulated abnormal target data close to real-world scenarios can be generated to enhance the model's generalization ability. However, current spectral of abnormal targets are usually randomly generated or extracted from existing spectral databases, which ignores the influence of factors such as lighting conditions and atmospheric absorption on spectral morphology during actual imaging. This approach, which does not consider actual physical conditions, often leads to poor performance of trained networks in real-world scenarios, making it difficult to accurately detect abnormal targets. SUMMARY

[0006] In view of the problems that there is lack of labeled data when capturing abnormal targets and normal scenes in the same environment in hyperspectral images, and the diffusion model does not fully consider the spectral dimension information, the abnormal target spectrum is generated by using principal component analysis and a linear spectral mixing model, a spectral diffusion model is established by introducing a double-window spectral diffusion background reconstruction method, the diffusion model is combined with the double-window strategy, and an environmental anomaly detection method based on spectral diffusion is designed.

[0007] The purpose of the present application is achieved by the following technical solutions:

[0008] An environmental anomaly detection method based on spectral diffusion comprises the following steps:

[0009] Step one: load the hyperspectral image, generate abnormal targets in the hyperspectral image containing only background information by using Berlin noise, and obtain a hyperspectral image containing abnormal targets;

[0010] Step two: load the hyperspectral image containing abnormal targets, adopt a double-window spectral diffusion strategy, gradually mix the target pixels with the spectral information around them, and obtain the diffused image data;

[0011] Step three: estimate the abundance matrix by using an abundance estimation network based on a U-Net framework, and use the estimated abundance matrix to reconstruct the background image in reverse;

[0012] Step four: input the reconstructed background image and the hyperspectral image containing abnormal targets into a target detection network based on a U-Net framework to detect abnormal targets;

[0013] Step five: diffuse the test image data through a double window, input the test image data into the trained abundance estimation network to reconstruct the background, and obtain the background image of the test image; input the obtained test data background image and test data into the trained target detection network to detect abnormal targets.

[0014] Compared with the prior art, the present application has the following advantages:

[0015] (1) The present application proposes an abnormal spectrum generation method based on principal component analysis and a linear spectral mixing model in the field of hyperspectral image target detection, solves the problem of lack of paired labeled data in the presence and absence of abnormalities, ensures the effectiveness of the learned features, and improves the realism of the traditional abnormal target generation method.

[0016] (2) The present application designs a double-window spectral diffusion model for hyperspectral data, regards the diffusion process of each pixel as the spectral mixing of the target and the surrounding background, improves the accuracy of the spatial-spectral dimension estimation in the reconstruction process, and makes the background estimation have high spatial and spectral accuracy.

[0017] (3) The present application introduces a double-window spectral diffusion background reconstruction method, which reduces the influence of abnormal expansion area on background estimation by iteratively accumulating the spectrum within the double window excluding the target, effectively suppressing the target spectrum.

[0018] (4) The AVIRIS dataset is used as training data, and the Airport-Beach-Urban hyperspectral imaging dataset is used as test data to evaluate the detection ability of the algorithm, which proves that the method can effectively detect abnormal information in hyperspectral images. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Flowchart of the spectral diffusion-based environmental anomaly detection method;

[0020] Figure 2 Flowchart of the abnormal target synthesis strategy;

[0021] Figure 3 Schematic diagram of the double-window diffusion model;

[0022] Figure 4 Comparison chart of the detection results of the present application and other hyperspectral anomaly detection algorithms. DETAILED DESCRIPTION

[0023] The technical solutions of the present application will be further described below in conjunction with the drawings, but are not limited thereto. Any modification or equivalent replacement of the technical solutions of the present application without departing from the spirit and scope of the present application shall be covered in the protection scope of the present application.

[0024] The present application provides a spectral diffusion-based environmental anomaly detection method, which generates abnormal spectra using principal component analysis and linear spectral mixing model, establishes a spectral diffusion model by introducing a double-window spectral diffusion and reverse background reconstruction method, generates a background image in combination with a double-window strategy, and finally inputs the background image and the original hyperspectral image containing abnormal targets into a U-Net detection network to detect abnormal targets in the hyperspectral image. Figure 1 As shown in the figure, the method specifically includes the following steps:

[0025] Step one: load the hyperspectral image, generate abnormal targets in the hyperspectral image containing only background information using Berlin noise, and obtain the hyperspectral image containing abnormal targets. The specific steps are as follows:

[0026] Step one: load a hyperspectral image of m x n x L, where m is the height of the hyperspectral image, n is the width of the hyperspectral image, and L is the number of spectral bands of the hyperspectral image.

[0027] Step one two: Apply principal component analysis to hyperspectral imaging for dimension reduction, get principal component matrix V and average spectrum μ; then, use the transpose of principal component matrix V T Multiply with the randomly generated weight vector w, divide by the number of principal components n components , reconstruct the new target spectrum s by principal component:

[0028]

[0029] Step one three: In the generated target area, use Gaussian distribution model for spectral mixing. Assuming that the generated target area mask is M, the target spectrum is s, and the background spectrum (where M = 1) is X background , calculate the abundance α of each abnormal target according to the distance d from the center to the edge of the target area:

[0030]

[0031] In the formula, σ is the standard deviation of the Gaussian kernel.

[0032] Step one four: The abundance is highest at the center of the target area and gradually decreases towards the boundary. The final spectrum X new of the target area is represented as:

[0033] X new = αX background +(1-α)s+n background

[0034] In the formula, n background is the random noise added to increase the variability of the data.

[0035] Step two: Load the hyperspectral image containing abnormal targets, use the double window spectral diffusion strategy to gradually mix the target pixels with their surrounding spectral information, get the diffused image data. As shown in Figure 2 and Figure 3 , the specific steps are as follows:

[0036] Step two one: For the hyperspectral image X tar containing abnormal targets, use the double window diffusion algorithm to create a mask matrix combined with the definition of outer window and inner window, calculate the double window matrix The specific steps are as follows:

[0037] Step two one: In a hyperspectral image containing abnormal targets, for the (i,j) pixel , define a window centered at x i,j , the outer window size is R, and the inner window size is r, where R and r are odd numbers, a hollow window is obtained by subtracting the inner window from the outer window, the outer window and the inner window The definition is as follows:

[0038]

[0039] Step 212: Create a mask matrix The specific definitions are as follows:

[0040]

[0041] Where M u,v is the element in the uth row and vth column of the matrix;

[0042] Step 2-3: Obtain the double window matrix by element-by-element multiplication of the outer window and the mask:

[0043]

[0044] Where ⊙ represents element-wise multiplication.

[0045] Step 2: Randomly generate a normal distribution abundance matrix, determine the distribution of each pixel after double-window diffusion, and combine the abundance matrix and double-window matrix to obtain the final expression of the diffused pixel. The specific steps are as follows:

[0046] Step 221: Randomly generate a normal distribution abundance matrix Each element in the matrix satisfies the distribution At the (i,j)th pixel x i,j The corresponding dual-window abundance matrix is

[0047]

[0048] Step 222: Let the pixel at position (i, j) be x, the corresponding double window be W, and the double window matrix be The tth element in is represented by w t , t=1,2,...,R 2 -r 2 . Double window abundance matrix The tth element in is represented by Assume that the original pixel in the image before diffusion is x0, and after continuously adding to the tth element, the pixel value x t for:

[0049] x t =x t-1 +a t w t

[0050] =x0+a1w1+a2w2+...+a t w t

[0051] Step two two three: because So x t Can be expressed as:

[0052]

[0053] Step two two four: assuming Then x t Is:

[0054]

[0055] Step two two five: for the pixel x i In the image size m x n, i = 1, 2, …, m x n, repeat step two one to step two four, get the diffusion image data.

[0056] Step three: inverse background reconstruction. Estimate the abundance matrix by using the abundance estimation network based on the U-Net framework, and use the estimated abundance matrix to reconstruct the background image. The specific steps are as follows:

[0057] Step three one: derive the true posterior distribution by using Bayes theorem, from x0 and x t To x t-1 The denoising process is expressed as:

[0058]

[0059] In the formula, q(x t-1 |x t ,x0) represents the conditional probability distribution of the previous state x t Under the current state x t-1 And the initial state x0;

[0060] Step three two: estimate the mean of the distribution x t-1 , x t-1 Can be expressed as:

[0061]

[0062] Step three three: after estimating the abundance matrix, use the element value in the abundance matrix to find the value of x t-1 :

[0063]

[0064] Recursively find x t-2 , x t-3 ……x0;

[0065] Step three four: repeat step three one to step three three for each pixel in the diffused image to reconstruct the background image Xrec ;

[0066] Step three five: compare the reconstructed background image with the original hyperspectral image, and take the reconstruction loss as the loss function, defined as follows:

[0067]

[0068] In the formula, X represents the hyperspectral image X tar The background reconstruction result of the i-th pixel.

[0069] Step four: anomaly target detection. The reconstructed background image and the hyperspectral image containing the anomaly target are input into the target detection network based on the U-Net framework for anomaly target detection. The specific steps are as follows:

[0070] Step four one: input the reconstructed background image X rec and the hyperspectral image containing the anomaly target X tar into the target detection network, and the target detection network adopts the U-Net architecture:

[0071]

[0072] In the formula, is the detection result, is the target detection network.

[0073] Step four two: the detection loss is defined as follows:

[0074]

[0075] In the formula, δ is the weight for balancing the two loss functions, The loss function is defined as follows:

[0076]

[0077] In the formula, M r is the true value mask;

[0078] The loss function is defined as follows:

[0079]

[0080] In the formula, α is a weight coefficient for balancing the influence of positive and negative samples, γ is a focus parameter for adjusting the focus, usually taking a value greater than 0, p t is the prediction probability of the model to the real category, if the real label M r = 1, then If the real label M r = 0, then

[0081] Step four three: training the abundance estimation network by minimizing the total loss function and the target detection network The total loss function is defined as follows:

[0082]

[0083] where k rec and k mask are the weights that adjust the importance of the reconstruction loss and the detection loss, respectively.

[0084] Step five: inference using the abundance estimation network and the target detection network. The test image data is passed through the double window diffusion, input into the trained abundance estimation network to reconstruct the background, and the background image of the test image is obtained. The obtained test data background image and test data are input into the trained target detection network for testing to detect the performance of the hyperspectral anomaly detection method based on the spectral diffusion model. The specific steps are as follows:

[0085] Step five one: for the hyperspectral test image Repeat step two to obtain the diffused image data.

[0086] Step five two: input the diffused image into the trained abundance estimation network, and repeat step three to reconstruct the background to obtain the reconstructed background image X rec .

[0087] Step five three: input the reconstructed background image X rec and the hyperspectral test image X into the target detection network to calculate the detection result:

[0088]

[0089] The present application uses online accessible hyperspectral datasets, including the Airport-Beach-Urban (ABU) dataset and the AVIRIS dataset, to evaluate the proposed hyperspectral anomaly detection capability based on the spectral diffusion model.

[0090] In all experiments, all methods used use their official code implementation with default algorithm parameters. Data preprocessing is carried out according to the official guidelines. Most methods use max-min normalization during data preprocessing, adjusting each test hyperspectral image to fit the [0, 1] range. However, some methods use different normalization strategies. For example, 2S-GLRT and PTA perform max-min normalization on each hyperspectral image band, respectively, while FrFE normalizes the entire test hyperspectral image by scaling all values relative to the maximum value.

[0091] Figure 4 Detection maps of 18 different hyperspectral anomaly detection methods on Airport-3, Urban-1 and Beach-1 datasets are given in Fig. 1. Figure 4 In (a), (b), (c), the first and second images represent the hyperspectral image in false color and corresponding ground truth, respectively. These images cover detection results in complex and simple backgrounds, including single-target and multi-target scenes.

[0092] As can be seen from the figures, different methods show different characteristics in anomaly detection. Methods such as GRX, LRX, 2S-GLRT, FrFE, RPCA-RX, GoDec-RX, Auto-AD, etc. tend to focus on background suppression, resulting in a cleaner background area in the anomaly score of the detection map. On the other hand, other comparison methods emphasize improving the anomaly score of the target.

[0093] All comparison methods improve the target-background contrast to some extent, but are also prone to misclassifying background areas as targets. In contrast, the DWSDiff method proposed in the present application effectively suppresses the background while significantly enhancing the anomaly contrast. This advantage is particularly evident in complex scenes such as Airport-3 and Urban-1.

Claims

1. A method for detecting environmental anomalies based on spectral diffusion, characterized in that The method comprises the following steps: Step 1: Load the hyperspectral image and use Perlin noise to generate abnormal targets in the hyperspectral image containing only background information to obtain a hyperspectral image containing abnormal targets; Step 2: Load the hyperspectral image containing the abnormal target and use the double-window spectral diffusion strategy to gradually mix the spectral information of the target pixel and its surroundings to obtain the diffused image data. The specific steps are as follows: Step 21: For hyperspectral images containing abnormal targets , using the double window diffusion algorithm, combined with the definition of the outer window and inner window to create a mask matrix, calculate the double window matrix , the specific steps are as follows: Step 2: In a hyperspectral image containing anomaly targets In the Pixels , define a i,j The outer window size is R, the inner window size is r, where R and r are odd numbers. A hollow window is obtained by subtracting the inner window from the outer window. and inner windows The definition is as follows: Step 212: Create a mask matrix , specifically defined as follows: Where, The matrix Rank Column elements; Step 2-3: Obtain the double window matrix by element-by-element multiplication of the outer window and the mask: Where, Expressed as element-wise multiplication; Step 22: Randomly generate a normal distribution abundance matrix. By determining the distribution of each pixel after double-window diffusion, the final expression of the diffused pixel is obtained by combining the abundance matrix and the double-window matrix. The specific steps are as follows: Step 221: Randomly generate a normal distribution abundance matrix , each element in the matrix satisfies the distribution ; in the Pixel x i,j The corresponding dual-window abundance matrix is : Step 222: Set the location The pixel at is denoted as x, the corresponding double window is denoted as W, and the double window matrix The tth element in is represented by , ;Dual window abundance matrix The tth element in is represented by , , assuming that the original pixel in the image before diffusion is , after adding to the tth element, the pixel value for: Step 223: Because ,so It can be expressed as: Step 224: When , ,but for: Step 225: For image size Pixels in , , repeat steps 221 to 224 to obtain diffused image data; Step 3: Use the abundance estimation network based on the U-Net framework to estimate the abundance matrix, and use the estimated abundance matrix to reversely reconstruct the background image; Step 4: Input the reconstructed background image and the hyperspectral image containing abnormal targets into the target detection network based on the U-Net framework for abnormal target detection; Step 5: The test image data is diffused through a double window and input into the trained abundance estimation network to reconstruct the background to obtain the background image of the test image; the obtained test data background image and the test data are input into the trained target detection network for abnormal target detection.

2. The method for detecting an environmental anomaly based on spectral diffusion according to claim 1 is characterized in that The specific steps of step one are as follows: Step 1: Load a Where m is the height of the hyperspectral image, n is the width of the hyperspectral image, and L is the number of spectral segments of the hyperspectral image; Step 1 and 2: Apply principal component analysis to hyperspectral imaging for dimensionality reduction to obtain the principal component matrix V and the mean spectrum μ; Then, using the transpose of the principal component matrix V T Multiply by the randomly generated weight vector w and divide by the number of principal components , reconstruct the new target spectrum s through the principal components: Step 13: Use the Gaussian distribution model to perform spectral mixing in the generated target area. Assuming that the generated target area mask is M, the target spectrum is s and the background spectrum is X. background , calculate the abundance α of each abnormal target according to the distance d from the center to the edge of the target area: Where, is the standard deviation of the Gaussian kernel; Step 14: The abundance is highest in the center of the target area and gradually decreases towards the edge. The final spectrum of the target area Expressed as: Where, It is random noise added to increase the variability of the data.

3. The method for detecting an environmental anomaly based on spectral diffusion according to claim 2 is characterized in that The specific steps of step three are as follows: Step 31: Use Bayes’ theorem to derive the true posterior distribution from and arrive The denoising process is expressed as: Where, Indicates the current state and the initial state Next, previous state The conditional probability distribution of ; Step 32: Estimate the distribution The mean of Expressed as: Step 3: After estimating the abundance matrix, use the element values ​​in the abundance matrix to find Value: Continuously recursively find 、 … ; represents the abundance estimation network; Step 34: Repeat steps 31 to 33 for each pixel in the diffused image to reconstruct the background image ; Step 35: Compare the reconstructed background image with the original hyperspectral image and use the reconstruction loss as the loss function, which is defined as follows: Where, Represents hyperspectral image The background reconstruction result of the i-th pixel.

4. The method for detecting an environmental anomaly based on spectral diffusion according to claim 3 is characterized in that The specific steps of step 4 are as follows: Step 41: Reconstruct the background image and hyperspectral images containing abnormal targets Input to the target detection network, the target detection network uses the U-Net architecture: Where, For the test results, For target detection network; Step 42: Detection loss is defined as follows: Where, To balance the weights of the two loss functions, The loss function is defined as follows: Where, is the true value mask; The loss function is defined as follows: Where, is a weight coefficient used to balance the influence of positive and negative samples, γ is a focus parameter for adjusting the focus, and is usually greater than 0. Is the model's predicted probability for the true category. If the true label ,but ; If the true label ,but ; Step 43: Train the abundance estimation network by minimizing the total loss function and target detection network , the total loss function is defined as follows: Where, and are weights for adjusting the importance of reconstruction loss and detection loss respectively.

5. The method for detecting an environmental anomaly based on spectral diffusion according to claim 4 is characterized in that The specific steps of step five are as follows: Step 51: For the hyperspectral test image , repeat step 2 to obtain the diffused image data; Step 52: Input the diffused image into the trained abundance estimation network, repeat step 3 to reversely reconstruct the background and obtain the reconstructed background image. ; Step 53: Reconstruct the background image and hyperspectral test images Input into the target detection network and calculate the detection results: 。

Images