Environmental anomaly detection method based on spectral diffusion

Abstract

The invention discloses an environmental anomaly detection method based on spectral diffusion, provides an abnormal spectrum generation method based on principal component analysis and a linear spectrum mixed model for the field of hyperspectral image target detection, and solves the problem of insufficient pairing mark data with and without anomaly. The effectiveness of the learned features is ensured, and the sense of reality of a traditional abnormal target generation method is improved. A double-window spectrum diffusion model is designed for hyperspectral data, the diffusion process of each pixel is regarded as spectrum mixing of a target and a surrounding background, the precision of space-spectrum dimension estimation in the reconstruction process is improved, and background estimation has high space and spectrum precision; a double-window spectrum diffusion background reconstruction method is introduced, and the spectrum is iteratively accumulated in the double windows excluding the target, so that the influence of an abnormal expansion area on background estimation is reduced, and the target spectrum is effectively inhibited.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral image processing, and relates to a hyperspectral anomaly detection method, and in particular to an environmental anomaly detection method based on spectral diffusion. Background Art

[0002] Hyperspectral remote sensing captures spectral information of tens to hundreds of continuous bands and combines them with spatial data. This unique "map-spectrum-in-one" capability makes hyperspectral imaging widely used in agriculture, forestry, mineral exploration, environmental monitoring, and land cover surveys. Anomaly detection is an important branch of machine learning and has important applications in computer vision. In the context of hyperspectral imaging, anomaly detection focuses on identifying observations that are inconsistent with the surrounding environment and is an active research area in hyperspectral image processing. Hyperspectral anomaly detection is a challenging task due to the lack of prior spectral and spatial information of the target.

[0003] At present, hyperspectral anomaly detection methods can be roughly divided into three methods: statistical modeling-based methods, representation learning-based methods, and deep learning-based methods. In recent years, deep learning-based methods have become increasingly prominent due to their ability to learn complex patterns from data. Among them, reconstruction-based methods have proven their feasibility through extensive validation. However, these methods often emphasize the difficulty of reconstructing anomaly targets and ignore the spectral difference between anomaly targets and the surrounding background, which is a key aspect of anomaly detection.

[0004] Diffusion-based models have recently gained attention in the field of anomaly detection, mainly because they can generate high-quality, realistic images and more effectively capture the complex distribution of the background, thereby improving the accuracy of anomaly detection. Although diffusion models have been applied to various hyperspectral image processing tasks, their application in hyperspectral anomaly detection is relatively rare. Existing diffusion models usually add spatial random noise to each spectral segment of the original image without fully considering the spectral dimension information, resulting in possible inaccuracies in spectral reconstruction.

[0005] Considering the data-driven nature of deep learning and the high cost of acquiring hyperspectral images in real scenes, this problem is particularly prominent, especially the lack of image pairs with / without abnormal targets in the same environment. To solve this problem, abnormal target data close to the real scene can be generated through simulation, thereby enhancing the generalization ability of the model. However, the spectra of abnormal targets are usually randomly generated or extracted from existing spectral databases, which ignores the influence of factors such as lighting conditions and atmospheric absorption on the spectral morphology during the actual imaging process. This processing method that does not consider the actual physical conditions often leads to poor performance of the trained network in real scenes and inability to accurately detect abnormal targets. Summary of the invention

[0006] In order to solve the problems of lack of labeled data and insufficient consideration of spectral dimension information in the diffusion model when capturing abnormal targets and normal scenes in the same environment in hyperspectral images, the present invention uses principal component analysis and linear spectral mixture model to generate abnormal target spectra, introduces a double-window spectral diffusion background reconstruction method, establishes a spectral diffusion model, combines the diffusion model with the double-window strategy, and designs an environmental anomaly detection method based on spectral diffusion.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] A method for detecting an environmental anomaly based on spectral diffusion comprises the following steps:

[0009] Step 1: Load the hyperspectral image, use Perlin noise to generate abnormal targets in the hyperspectral image containing only background information, and obtain a hyperspectral image containing abnormal targets;

[0010] Step 2: Load the hyperspectral image containing abnormal targets, use the double-window spectral diffusion strategy to gradually mix the target pixel with its surrounding spectral information, and obtain the diffused image data;

[0011] Step 3: Use the abundance estimation network based on the U-Net framework to estimate the abundance matrix, and use the estimated abundance matrix to reversely reconstruct the background image;

[0012] Step 4: Input the reconstructed background image and the hyperspectral image containing abnormal targets into the target detection network based on the U-Net framework for abnormal target detection;

[0013] Step 5: Diffusion of the test image data through double windows is input into the trained abundance estimation network to reconstruct the background, thereby obtaining the background image of the test image; the obtained test data background image and the test data are input into the trained target detection network together for abnormal target detection.

[0014] Compared with the prior art, the present invention has the following advantages:

[0015] (1) Aiming at the field of hyperspectral image target detection, the present invention proposes an anomaly spectrum generation method based on principal component analysis and linear spectral mixture model, which solves the problem of insufficient paired labeled data with and without anomalies to ensure the validity of the learned features and improves the realism of traditional anomaly target generation methods.

[0016] (2) The present invention designs a dual-window spectral diffusion model for hyperspectral data, which regards the diffusion process of each pixel as a spectral mixture of the target and the surrounding background, thereby improving the accuracy of spatial-spectral dimension estimation during the reconstruction process and making the background estimation have higher spatial and spectral accuracy.

[0017] (3) The present invention introduces a double-window spectral diffusion background reconstruction method, which reduces the influence of the abnormal expansion area on the background estimation by iteratively accumulating the spectrum in the double window excluding the target, and effectively suppresses the target spectrum.

[0018] (4) The AVIRIS dataset was used as training data, and the Airport–Beach–Urban hyperspectral imaging dataset was used as test data to evaluate the algorithm's detection capability, proving that the method can effectively detect abnormal information in hyperspectral images. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of the environmental anomaly detection method based on spectral diffusion;

[0020] Figure 2 Synthesize a strategy flow chart for anomaly targets;

[0021] Figure 3 It is a schematic diagram of the double-window diffusion model;

[0022] Figure 4 This is a comparison chart of the detection results of the present invention and other hyperspectral anomaly detection algorithms. DETAILED DESCRIPTION

[0023] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0024] The present invention provides an environmental anomaly detection method based on spectral diffusion. The method generates anomaly spectra using principal component analysis and linear spectral mixture models, establishes a spectral diffusion model by introducing double-window spectral diffusion and reverse background reconstruction methods, and generates a background image by combining with a double-window strategy. Finally, the background image and the original hyperspectral image containing abnormal targets are input into a U-Net detection network to detect abnormal targets in the hyperspectral image. Figure 1 As shown, the specific steps include:

[0025] Step 1: Load the hyperspectral image and use Perlin noise to generate abnormal targets in the hyperspectral image containing only background information to obtain a hyperspectral image containing abnormal targets. The specific steps are as follows:

[0026] Step 1: Load a hyperspectral image of size m×n×L, where m is the height of the hyperspectral image, n is the width of the hyperspectral image, and L is the number of spectral bands of the hyperspectral image.

[0027] Step 1 and 2: Apply principal component analysis to hyperspectral imaging for dimensionality reduction, and obtain the principal component matrix V and the average spectrum μ; then, use the transpose of the principal component matrix V T Multiply by the randomly generated weight vector w and divide by the number of principal components n components , reconstruct the new target spectrum s through the principal components:

[0028]

[0029] Step 13: Use the Gaussian distribution model to perform spectral mixing in the generated target area. Assuming the generated target area mask is M, the target spectrum is s and the background spectrum (where M = 1) is X. background , calculate the abundance α of each abnormal target according to the distance d from the center to the edge of the target area:

[0030]

[0031] Where σ is the standard deviation of the Gaussian kernel.

[0032] Step 14: The abundance is highest in the center of the target area and gradually decreases toward the edge. The final spectrum of the target area is X new It is expressed as:

[0033] X new =αX background +(1-α)s+n background

[0034] Where n background It is random noise added to increase the variability of the data.

[0035] Step 2: Load the hyperspectral image containing abnormal targets, use the double-window spectral diffusion strategy, gradually mix the target pixel with its surrounding spectral information, and obtain the diffused image data. Figure 2 and Figure 3 As shown, the specific steps are as follows:

[0036] Step 21: For the hyperspectral image X containing abnormal targets tar , using the double window diffusion algorithm, combined with the definition of the outer window and the inner window to create a mask matrix, calculate the double window matrix The specific steps are as follows:

[0037] Step 2.1: In a hyperspectral image containing anomalies In the example, for the pixel (i,j) Define a i,j The size of the outer window is R, and the size of the inner window is r, where R and r are odd numbers. A hollow window is obtained by subtracting the inner window from the outer window. and inner window The definition is as follows:

[0038]

[0039] Step 212: Create a mask matrix The specific definitions are as follows:

[0040]

[0041] Where M u,v is the element in the uth row and vth column of the matrix;

[0042] Step 213: Obtain the double window matrix by element-by-element multiplication of the outer window and the mask:

[0043]

[0044] Where ⊙ represents element-by-element multiplication.

[0045] Step 22: Randomly generate a normal distribution abundance matrix, determine the distribution of each pixel after double window diffusion, and combine the abundance matrix and double window matrix to obtain the final expression of the diffused pixel. The specific steps are as follows:

[0046] Step 221: Randomly generate a normal distribution abundance matrix Each element in the matrix satisfies the distribution At the (i,j)th pixel x i,j The corresponding dual-window abundance matrix is

[0047]

[0048] Step 222: Let the pixel at position (i, j) be x, the corresponding double window be W, and the double window matrix be The tth element in is denoted as w t , t=1,2,...,R 2 -r 2 . Dual-window abundance matrix The tth element in is represented by Assume that the original pixel in the image before diffusion is x0. After continuously adding to the tth element, the pixel value x t for:

[0049] x t =x t-1 +a t w t

[0050] =x0+a1w1+a2w2+...+a t w t

[0051] Step 223: Because So x t It can be expressed as:

[0052]

[0053] Step 224: Assumptions Then x t for:

[0054]

[0055] Step 225: For pixel x in the image size m×n i , i=1,2,...,m×n, repeat steps 221 to 224 to obtain the diffused image data.

[0056] Step 3: Reverse background reconstruction. Use the abundance estimation network based on the U-Net framework to estimate the abundance matrix, and use the estimated abundance matrix to reversely reconstruct the background image. The specific steps are as follows:

[0057] Step 31: Use Bayes’ theorem to derive the true posterior distribution from x0 and x t to x t-1 The denoising process is expressed as:

[0058]

[0059] In the formula, q(x t-1 |x t ,x0) means in the current state x t and the initial state x0, the previous state x t-1 The conditional probability distribution of ;

[0060] Step 32: Estimate the distribution x t-1 The mean value of x t-1 It can be expressed as:

[0061]

[0062] Step 3: After estimating the abundance matrix, use the element values ​​in the abundance matrix to find x t-1 Values:

[0063]

[0064] Keep recursively finding x t-2 、x t-3 ...x0;

[0065] Step 34: Repeat steps 31 to 33 for each pixel in the diffused image to reconstruct the background image Xrec ;

[0066] Step 35: Compare the reconstructed background image with the original hyperspectral image, and use the reconstruction loss as the loss function, which is defined as follows:

[0067]

[0068] In the formula, Represents the hyperspectral image X tar Background reconstruction result of the i-th pixel.

[0069] Step 4: Abnormal target detection. The reconstructed background image and the hyperspectral image containing abnormal targets are input into the target detection network based on the U-Net framework for abnormal target detection. The specific steps are as follows:

[0070] Step 41: Reconstruct the background image X rec and the hyperspectral image X containing abnormal targets tar Input to the target detection network, the target detection network uses the U-Net architecture:

[0071]

[0072] In the formula, For the test results, For the target detection network.

[0073] Step 42: Detection loss is defined as follows:

[0074]

[0075] In the formula, δ is the weight to balance the two loss functions. The loss function is defined as follows:

[0076]

[0077] Where M r is the truth mask;

[0078] The loss function is defined as follows:

[0079]

[0080] In the formula, α is a weight coefficient used to balance the influence of positive and negative samples, γ is a focus parameter for adjusting the focus, which is usually greater than 0, and p t is the model’s predicted probability for the true category. If the true label M r =1, then If the true label M r =0, then

[0081] Step 43: Train the abundance estimation network by minimizing the total loss function and target detection network The total loss function is defined as follows:

[0082]

[0083] In the formula, k rec and k mask They are the weights for adjusting the importance of reconstruction loss and detection loss respectively.

[0084] Step 5: Use the abundance estimation network and the target detection network for inference. Diffusion of the test image data through the double window is input into the trained abundance estimation network to reconstruct the background and obtain the background image of the test image. The obtained test data background image and test data are input into the trained target detection network for testing to detect the performance of the hyperspectral anomaly detection method based on the spectral diffusion model. The specific steps are as follows:

[0085] Step 51: For the hyperspectral test image Repeat step 2 to obtain the diffused image data.

[0086] Step 52: Input the diffused image into the trained abundance estimation network, repeat step 3 to reversely reconstruct the background, and obtain the reconstructed background image X rec .

[0087] Step 53: X the reconstructed background image rec And the hyperspectral test image X is input into the target detection network and the detection result is calculated:

[0088]

[0089] The present invention uses online accessible hyperspectral datasets, including the Airport–Beach–Urban (ABU) dataset and the AVIRIS dataset, to evaluate the proposed hyperspectral anomaly detection capability based on the spectral diffusion model.

[0090] In all experiments, all methods used were implemented using their official codes with default algorithm parameters. Data preprocessing was performed following the official guidelines. Most methods employed max-min normalization during data preprocessing, adjusting each test hyperspectral imaging to fit into the [0,1] range. However, some methods used different normalization strategies. For example, 2S-GLRT and PTA performed max-min normalization for each hyperspectral imaging band separately, while FrFE normalized the entire test hyperspectral imaging by scaling all values ​​relative to the maximum value.

[0091] Figure 4 The detection graphs of 18 different hyperspectral anomaly detection methods on Airport-3, Urban-1 and Beach-1 datasets are given. Figure 4 In (a), (b), and (c), the first and second images represent hyperspectral images in false color and the corresponding true value, respectively. These images cover the detection results under complex and simple backgrounds, including single-target and multi-target scenes.

[0092] As can be seen from the figure, different methods show different characteristics in anomaly detection. Methods such as GRX, LRX, 2S-GLRT, FrFE, RPCA-RX, GoDec-RX, and Auto-AD tend to focus on background suppression, resulting in cleaner background areas with lower anomaly scores in their detection images. On the other hand, other compared methods emphasize improving the anomaly score of the target.

[0093] All the compared methods improve the target-background contrast to some extent, but they are also prone to misclassifying the background area as the target. In contrast, the DWSDiff method proposed in this paper significantly enhances the anomaly contrast while effectively suppressing the background. This advantage is particularly evident in complex scenes such as Airport-3 and Urban-1.

Claims

1. A method for detecting environmental anomalies based on spectral diffusion, characterized in that The method comprises the following steps: Step 1: Load the hyperspectral image, use Perlin noise to generate abnormal targets in the hyperspectral image containing only background information, and obtain a hyperspectral image containing abnormal targets; Step 2: Load the hyperspectral image containing abnormal targets, use the double-window spectral diffusion strategy to gradually mix the target pixel with its surrounding spectral information, and obtain the diffused image data; Step 3: Use the abundance estimation network based on the U-Net framework to estimate the abundance matrix, and use the estimated abundance matrix to reversely reconstruct the background image; Step 4: Input the reconstructed background image and the hyperspectral image containing abnormal targets into the target detection network based on the U-Net framework for abnormal target detection; Step 5: Diffusion of the test image data through double windows is input into the trained abundance estimation network to reconstruct the background, thereby obtaining the background image of the test image; the obtained test data background image and the test data are input into the trained target detection network together for abnormal target detection.

2. The method for detecting anomalies in an environment based on spectral diffusion according to claim 1, characterized in that The specific steps of step one are as follows: Step 1: Load a hyperspectral image of size m×n×L, where m is the height of the hyperspectral image, n is the width of the hyperspectral image, and L is the number of spectral bands of the hyperspectral image; Step 1 and 2: Apply principal component analysis to hyperspectral imaging for dimensionality reduction, and obtain the principal component matrix V and the average spectrum μ; Then, using the transpose of the principal component matrix V T Multiply by the randomly generated weight vector w and divide by the number of principal components n components , reconstruct the new target spectrum s through the principal components: Step 13: Use the Gaussian distribution model to perform spectral mixing in the generated target area. Assuming the generated target area mask is M, the target spectrum is s and the background spectrum is X. background , calculate the abundance α of each abnormal target according to the distance d from the center to the edge of the target area: Where σ is the standard deviation of the Gaussian kernel; Step 14: The abundance is highest in the center of the target area and gradually decreases toward the boundary. The final spectrum of the target area is X new It is expressed as: X new =αX background +(1-α)s+n background Where n background It is random noise added to increase the variability of the data.

3. The method for detecting anomalies in an environment based on spectral diffusion according to claim 1, characterized in that The specific steps of step 2 are as follows: Step 21: For the hyperspectral image X containing abnormal targets tar , using the double window diffusion algorithm, combined with the definition of the outer window and the inner window to create a mask matrix, calculate the double window matrix Step 22: Randomly generate a normal distribution abundance matrix, determine the distribution of each pixel after double-window diffusion, and combine the abundance matrix and the double-window matrix to derive the final expression of the diffused pixel.

4. The method for detecting anomalies in an environment based on spectral diffusion according to claim 3 is characterized in that The specific steps of step 21 are as follows: Step 2.1: In a hyperspectral image containing anomalies In the example, for the pixel (i,j) Define a i,j The size of the outer window is R, and the size of the inner window is r, where R and r are odd numbers. A hollow window is obtained by subtracting the inner window from the outer window. and inner window The definition is as follows: Step 212: Create a mask matrix The specific definitions are as follows: Where M u,v is the element in the uth row and vth column of the matrix; Step 213: Obtain the double window matrix by element-by-element multiplication of the outer window and the mask: Where ⊙ represents element-by-element multiplication.

5. The method for detecting anomalies in an environment based on spectral diffusion according to claim 3 is characterized in that The specific steps of step 22 are as follows: Step 221: Randomly generate a normal distribution abundance matrix Each element in the matrix satisfies the distribution At the (i,j)th pixel x i,j The corresponding dual-window abundance matrix is Step 222: Let the pixel at position (i, j) be x, the corresponding double window be W, and the double window matrix be The tth element in is denoted as w t , t=1,2,...,R 2 -r 2 . Dual-window abundance matrix The tth element in is represented by Assume that the original pixel in the image before diffusion is x0. After continuously adding to the tth element, the pixel value x t for: x t =x t-1 +a t w t =x0+a1w1+a2w2+...+a t w t Step 223: Because So x t It can be expressed as: Step 224: Assumptions Then x t for: Step 225: For pixel x in the image size m×n i , i=1,2,...,m×n, repeat steps 221 to 224 to obtain the diffused image data.

6. The method for detecting anomalies in an environment based on spectral diffusion according to claim 1, characterized in that The specific steps of step three are as follows: Step 31: Use Bayes’ theorem to derive the true posterior distribution from x0 and x t to x t-1 The denoising process is expressed as: In the formula, q(x t-1 |x t ,x0) means in the current state x t and the initial state x0, the previous state x t-1 The conditional probability distribution of ; Step 32: Estimate the distribution x t-1 The mean value of x t-1 It is expressed as: Step 3: After estimating the abundance matrix, use the element values ​​in the abundance matrix to find x t-1 Values: Keep recursively finding x t-2 、x t-3 ...x0; Step 34: Repeat steps 31 to 33 for each pixel in the diffused image to reconstruct the background image X rec ; Step 35: Compare the reconstructed background image with the original hyperspectral image, and use the reconstruction loss as the loss function, which is defined as follows: In the formula, Represents the hyperspectral image X tar Background reconstruction result of the i-th pixel.

7. The method for detecting anomalies in an environment based on spectral diffusion according to claim 1, characterized in that The specific steps of step 4 are as follows: Step 41: Reconstruct the background image X rec and the hyperspectral image X containing abnormal targets tar Input to the target detection network, the target detection network uses the U-Net architecture: In the formula, For the test results, For the target detection network. Step 42: Detection loss is defined as follows: In the formula, δ is the weight to balance the two loss functions. The loss function is defined as follows: Where M r is the truth mask; The loss function is defined as follows: In the formula, α is a weight coefficient used to balance the influence of positive and negative samples, γ is a focus parameter for adjusting the focus, which is usually greater than 0, and p t is the model’s predicted probability for the true category. If the true label M r =1, then If the true label M r =0, then Step 43: Train the abundance estimation network by minimizing the total loss function and target detection network The total loss function is defined as follows: In the formula, k rec and k mask They are the weights for adjusting the importance of reconstruction loss and detection loss respectively.

8. The method for detecting anomalies in an environment based on spectral diffusion according to claim 1, characterized in that The specific steps of step five are as follows: Step 51: For the hyperspectral test image Repeat step 2 to obtain the diffused image data. Step 52: Input the diffused image into the trained abundance estimation network, repeat step 3 to reversely reconstruct the background, and obtain the reconstructed background image X rec . Step 53: X the reconstructed background image rec And the hyperspectral test image X is input into the target detection network and the detection result is calculated:

Images