Hyperspectral anomaly detection method based on spectral unmixing and feature fusion auto-encoder

Abstract

The invention discloses a hyperspectral anomaly detection method based on spectral unmixing and a feature fusion auto-encoder, and the method comprises the steps: firstly, providing a spectral unmixing method of a nonlinear mixed model, which is used for processing a complex spectral mixing phenomenon, is suitable for various complex application scenes, can reflect an actual spectral mixing process more truly, and improves the detection accuracy; the spectral features of the abnormal target can be effectively extracted, and the sensitivity and accuracy of anomaly detection are improved; secondly, a feature fusion automatic encoder is designed, prior information of anomaly and background is introduced into the automatic encoder, the flexibility of a hybrid model is not reduced, spectrum and space information of hyperspectral data is fully utilized to fuse feature information, and the accuracy of anomaly detection is effectively improved; compared with an advanced anomaly detection algorithm at present, a comparison experiment is carried out on real hyperspectral data, and the effectiveness of the method and the accuracy of abnormal target detection are verified.

Description

Technical Field

[0001] The present invention belongs to the field of hyperspectral remote sensing image processing, and particularly relates to a hyperspectral anomaly detection method based on spectral unmixing and feature fusion autoencoder. Background Art

[0002] Hyperspectral anomaly detection, as a branch of target detection (TD), aims to identify pixels of interest that are different from the surroundings spatially or spectrally without prior knowledge of the target. In hyperspectral remote sensing images, the spectral reflection characteristics of different surface-covered objects usually show obvious changes, and these changes often contain information related to some abnormal targets, such as geological mineralization area exploration, food quality pollution detection, plant diseases, etc. Therefore, hyperspectral anomaly detection (HAD) shows great potential in fields such as agricultural disease detection, rare mineral exploration, and military abnormal target discovery. Traditional hyperspectral anomaly detection methods mainly rely on statistical models, such as the RX algorithm based on the mean and covariance matrix. However, these methods usually assume that the data follows a specific distribution, such as a multivariate normal distribution, etc., which is often difficult to meet in practical applications.

[0003] In recent years, the great success of deep learning in computer vision has promoted its continuous development in the field of hyperspectral anomaly detection. Unsupervised deep learning methods do not require a large amount of labeled data for training and are suitable for scenarios of anomaly detection without prior information. The hyperspectral anomaly detection algorithm based on deep clustering jointly performs spectral unmixing and anomaly detection in an end-to-end manner. Specifically, a convolutional autoencoder is used for spectral unmixing to obtain the abundance image after unmixing, and then an anomaly detection process is realized by designing a clustering network. The clustering loss is regarded as a regularization term in unmixing, so that the spectral unmixing and anomaly detection processes are jointly optimized, forming a mutual feedback mechanism to obtain the detection result, which has good detection performance. Research and investigation have shown that the anomaly target is the area corresponding to the reference inclusion in the prominent scene, and the number of included pixels in each scene is very important for target detection. Therefore, Younis et al. proposed an ensemble anomaly detector that integrates the abundance map estimated by hyperspectral unmixing for the anomaly detection process. Zhang et al. proposed a dual-view hyperspectral anomaly point detection method. At the pixel level, the spectral angle distance is used to calculate the similarity between the central pixel and neighboring pixels, further mining the spatial consistency to perform anomaly detection. On the other hand, from the sub-pixel level analysis, the difference between the anomaly point and the background usually comes from dissimilar inclusions. In this case, unmixing processing will be fully implemented, and finally the detection results of the two views will be fused to obtain the anomaly point. A new detection algorithm based on spectral unmixing and dictionary-based low-rank decomposition obtains the abundance vector through spectral unmixing and uses these vectors for anomaly detection. Secondly, in order to better represent the highly correlated background and sparse anomaly points, a dictionary is constructed based on the mean shift clustering of the abundance vectors to improve the discriminability and representativeness of the algorithm. Finally, a low-rank matrix decomposition method based on the constructed dictionary is proposed, and the residual matrix is sparsified, and the anomaly is extracted by summing the columns of the residual matrix.

[0004] With the continuous development of spectral unmixing technology and anomaly detection algorithms, more powerful technical support is provided for various practical applications. However, since the spectra of most ground objects are formed by the mixture of multiple materials, a pixel may contain the mixed information of multiple ground objects, especially in complex terrains, making it difficult to distinguish the spectral characteristics of a single pixel. Therefore, how to handle the spectral mixing phenomenon in hyperspectral data is a challenging problem currently faced. In addition, how to extract effective spatial-spectral features from hyperspectral data and fuse them to improve the performance of anomaly detection is also worthy of further study. Summary of the Invention

[0005] The present invention is to solve the above-mentioned technical problems existing in the prior art, and provides a hyperspectral anomaly detection method based on spectral unmixing and feature fusion autoencoder.

[0006] The technical solution of the present invention is: a hyperspectral anomaly detection method based on spectral unmixing and feature fusion autoencoder, which is carried out according to the following steps:

[0007] Step 1. Input three-dimensional hyperspectral image where M×N is the size of the band image, and B is the number of bands in the band group;

[0008] Step 2. Transform the three-dimensional hyperspectral data H into two-dimensional hyperspectral data where L is the total number of pixels in a single-band image, which is calculated according to formula (1):

[0009] L = M×N (1)

[0010] Step 3. Input the two-dimensional hyperspectral data y l into the encoder to obtain the output g(·) for representing the abundance vector a l , and input the output data a l of the encoder and the endmember matrix Z into the decoder h(·) to reconstruct the two-dimensional image

[0011] The said g(·) is shown as formula (2):

[0012] g(y l ) = s(diag(α)Qy l + ω E (y l ; W E )) (2)

[0013] where: ω E is the non-linear function of the encoder, W E is the non-linear part of the encoder, s(·) is a mapping function, α is a learning parameter, and diag(·) is a diagonal matrix function;

[0014] y l is the pixel point of each band, y l ∈{Y1, Y2, …, Y l , …, Y L}, l = 1, 2, …, L, and y l is represented by using the non-linear fluctuation on linear mixing, as shown in formula (3);

[0015] y l = Za l + ψ(Z, a l ) + e l (3)

[0016] where, is the endmember matrix, a lis the abundance vector, e l is the additive noise term, ψ(·) represents the optimal function, which is a non-linear mapping based on the kernel function and is represented by the kernel Hilbert space and the Mercer kernel;

[0017] It is shown that the constraint Q is calculated according to formula (4);

[0018]

[0019] The represents the pseudo-inverse operator;

[0020] The decoder h(·) is a feature fusion auto-decoder, and h(·) is as shown in formula (5);

[0021] h(a l ) = r(Za l + ω D (a l , Z; W D )) (5)

[0022] where ω D is the non-linear function of the decoder, W D represents the fusion process, and r(·) is the function that maps the decoder result to the non-negative orthant;

[0023] The encoder and the decoder constitute a feature fusion auto-encoder. The network has a total of seven layers, and the number of neurons in each layer is B×L, BB×L, BB / 2×L, BB / 4×L, BB / 2×L, BB×L, and B×L in sequence, where BB = 100;

[0024] Step 4. Define the optimization variable Θ and the loss function C(Θ) of the model, and optimize the model;

[0025] Step 4.1 Define the optimization variable Θ, as shown in formula (6);

[0026] Θ = {Z, W, Q, α}, W = {W D , W E} (6)

[0027] Step 4.2 Define the loss function of the model as shown in formula (7);

[0028]

[0029] where represents the reconstruction error, is the regularization term of the neural network parameters, is the regularization term of the endmember matrix, is the pseudo-inverse constraint The relaxation approximation of, λ Q , and are hyperparameters for regularizing terms in the balance loss function;

[0030] Step 5. Transform the reconstructed two-dimensional image into three-dimensional reconstructed background information

[0031] Step 6. Determine the anomaly result map F through formula (8);

[0032]

[0033] where i ∈ {1, 2, …, M}, j ∈ {1, 2, …, N}, b ∈ {1, 2, …, B}, is the pixel of the reconstructed data at the b band, and f i,j is the pixel of the anomaly detection result image F at the (i, j) position.

[0034] Compared with the prior art, the present invention has two improvements: on the one hand, by using the non-linear fluctuations in linear mixing, a spectral unmixing method for a non-linear mixing model is proposed, which decomposes the mixed spectrum into multiple endmembers and the corresponding abundance matrix, making the spectral components of each pixel clear, applicable to diverse and complex application scenarios, capable of more realistically reflecting the actual spectral mixing process, and effectively extracting the spectral features of abnormal targets, improving the sensitivity and accuracy of anomaly detection. On the other hand, a feature fusion autoencoder is designed, and prior information of anomalies and backgrounds is introduced into the autoencoder without reducing the flexibility of the mixing model. By fully utilizing the spectral and spatial information of hyperspectral data to fuse the feature information, a reconstructed background target information image is obtained, effectively improving the accuracy of anomaly detection. The present invention is experimentally compared with current advanced anomaly detection algorithms on five groups of real hyperspectral data, verifying the effectiveness of the present invention and the accuracy of abnormal target detection.

[0035] Figure 1 are the sample image and ground truth map used in the embodiments of the present invention.

[0036] Figure 2 is the comparison chart of the detection results between the embodiments of the present invention and other methods. Detailed implementation manners

[0037] The hyperspectral anomaly detection method based on spectral unmixing and feature fusion autoencoder of the present invention is carried out according to the following steps:

[0038] Step 1. Input a three-dimensional hyperspectral image where M×N is the size of the band image, and B is the number of bands in the band group;

[0039] Step 2. Transform the three-dimensional hyperspectral data H into two-dimensional hyperspectral data where L is the total number of pixels in a single-band image, calculated according to formula (1):

[0040] L = M × N (1)

[0041] Step 3. Input the two-dimensional hyperspectral data y l into the encoder to obtain the output g(·) representing the abundance vector a l . Input the output data a l of the encoder and the endmember matrix Z into the decoder h(·) to reconstruct the two-dimensional image

[0042] The said g(·) is shown as formula (2):

[0043] g(y l ) = s(diag(α)Qy l + ω E (y l ; W E )) (2)

[0044] where: ω E is the non-linear function of the encoder, W E is the non-linear part of the encoder, s(·) is a mapping function, α is a learning parameter, and diag(·) is a diagonal matrix function;

[0045] y l is the pixel point of each band, y l ∈ {Y1, Y2, …, Y l , …, Y L}, l = 1, 2, …, L;

[0046] Use the non-linear fluctuation on linear mixing to represent y l , as shown in formula (3);

[0047] y l = Za l + ψ(Z, a l ) + e l (3)

[0048] where is the endmember matrix, a l is the abundance vector, e l is the additive noise term, ψ(·) is expressed as the optimal function, and the function is a non-linear mapping based on the kernel function, represented by the kernel Hilbert space and the Mercer kernel;

[0049] Show that the constraint Q is calculated according to formula (4);

[0050]

[0051] The represents the pseudo-inverse operator;

[0052] The decoder h(·) is a feature fusion-based auto-decoder, and h(·) is shown in formula (5);

[0053] h(a l ) = r(Za l + ω D (a l , Z; W D )) (5)

[0054] where ω D is the non-linear function of the decoder, W D represents the fusion process, and r(·) is a function that maps the decoder result to the non-negative orthant;

[0055] The encoder and the decoder constitute a feature fusion auto-encoder. The network has a total of seven layers, and the number of neurons in each layer is B×L, BB×L, BB / 2×L, BB / 4×L, BB / 2×L, BB×L, and B×L in sequence, where BB = 100;

[0056] Step 4. Define the optimization variable Θ and the loss function C(Θ) of the model, and optimize the model;

[0057] Step 4.1 Define the optimization variable Θ, as shown in formula (6);

[0058] Θ = {Z, W, Q, α}, W = {W D , W E} (6)

[0059] Step 4.2 Define the loss function of the model as shown in formula (7);

[0060]

[0061] where represents the reconstruction error, is the regularization term of the neural network parameters, is the regularization term of the endmember matrix, is the relaxation approximation of the pseudo-inverse constraint , λ Q , and are hyperparameters that balance the regularization terms in the loss function;

[0062] Step 5. Transform the reconstructed two-dimensional image into three-dimensional reconstructed background information

[0063] Step 6. Determine the abnormal result graph F through formula (8);

[0064]

[0065] where i ∈ {1, 2, …, M}, j ∈ {1, 2, …, N}, b ∈ {1, 2, …, B}, is the data after reconstruction the pixel at the pixel point in the b band, f i,j is the pixel of the abnormal detection result image F at the (i, j) position.

[0066] Compare the detection results of the embodiments of the present invention with other existing methods for Figure 1 five groups of data sets (a)-(e), and the results are as Figure 2 shown. Figure 2 A total of six hyperspectral anomaly detection algorithms are compared: (a) GRX; (b) RGAE; (c) GAED; (d) UFBSM; (e) FCAE; (g) SUF2A (embodiment of the present invention).

[0067] From Figure 2 it can be seen that in data set (a), the comparison algorithms are not obvious for detecting abnormal targets, and the relative background contour information is clear in the GRX, RGAE, GAED, and UFBSM algorithms. In data sets (b) and (c), the prominent targets of RX are not clear, the abnormal target information of RGAE, GAED, and UFBSM is not clearly shown but the background island information is prominent, and abnormal information can be seen in the SUF2A algorithm but the background information is also relatively prominent. In data set (d), the abnormal detection result of the SUF2A algorithm is most similar to the reference image, and the detection effect is the best among the comparison algorithms. In data set (e), the GRX and SUF2A algorithms detect abnormalities while the background information is also relatively obvious, and the result graph of the FCAE algorithm is blurred in each data set, and the boundary of obvious abnormal points cannot be seen clearly. Overall, the embodiments of the present invention have successfully detected abnormal targets on all data sets, separated the abnormal targets from the background, and the background suppression effect is ideal, that is, the embodiments of the present invention have achieved good detection results on each data set.

Claims

1. A hyperspectral anomaly detection method based on spectral unmixing feature fusion autoencoder, characterized in that Follow these steps: Step 1. Input 3D hyperspectral image Where M×N is the size of the band image, and B is the number of bands in the band group; Step 2. Transform the three-dimensional hyperspectral data H into two-dimensional hyperspectral data Where L is the total number of pixels of the single-band image, calculated according to formula (1): L=M×N (1) Step 3. Transform the two-dimensional hyperspectral data y l Input into the encoder to get the abundance vector a l The output g(·) of the encoder is converted into the output data a l and the end-element matrix Z are input into the decoder h(·) to reconstruct the two-dimensional image The g(·) is shown in formula (2): g(y l )=s(diag(α)Qy l +ω E (y l ;W E )) (2) Where: E is the nonlinear function of the encoder, W E is the nonlinear part of the encoder, s(·) is a mapping function, α is a learning parameter, and diag(·) is a diagonal matrix function; y l is the pixel point of each band, y l ∈{Y1,Y2,…,Y l ,…,Y L }, l = 1, 2, ..., L, using the nonlinear fluctuations on the linear mixture to y l It is expressed as shown in formula (3); y l =Za l +ψ(Z,a l )+e l (3) in, is the end member matrix, a l is the abundance vector, e l is an additive noise term, ψ(·) is represented as an optimal function, which is a nonlinear mapping based on a kernel function, using kernel Hilbert space and Mercer kernel correlation representation; The display constraint Q is calculated according to formula (4); Said T represents a pseudo-inverse operator; The decoder h(·) is an automatic decoder based on feature fusion, and h(·) is shown in formula (5); h(a l )=r(Za l +ω D (in l ,Z;W D )) (5) Among them, ω D is the nonlinear function of the decoder, W D represents the fusion process, r(·) is a function that maps the decoder result to a non-negative orthogonal one; The encoder and decoder constitute a feature fusion automatic encoder, the network has seven layers in total, the neuron sizes of each layer are B×L, BB×L, BB / 2×L, BB / 4×L, BB / 2×L, BB×L and B×L, and BB=100; Step 4. Define the optimization variable Θ and the model's loss function C(Θ) to optimize the model; Step 4.1: Define the optimization variable Θ, as shown in formula (6); Θ={Z,W,Q,α}, W={W D ,W E } (6) Step 4.2 defines the loss function of the model as shown in formula (7); in, represents the reconstruction error, is the regularization term of the neural network parameters, is the regularization term of the endmember matrix, is the pseudo-inverse constraint The relaxed approximation, λ Q , and is the hyperparameter of the regularization term in the balanced loss function; Step 5. Reconstruct the 2D image Transformed into 3D reconstruction background information Step 6. Determine the abnormal result graph F by formula (8); Among them, i∈{1,2,…,M}, j∈{1,2,…,N}, b∈{1,2,…,B}, After reconstruction, the data At the pixel point in the b band, f i,j is the pixel at position (i, j) of the anomaly detection result image F.

Images