A Hyperspectral Anomaly Detection Method Based on a Variational Background Inference Generative Adversarial Network

Abstract

The present invention relates to a hyperspectral anomaly detection method based on a variational background inference generative adversarial network, which includes the following steps: inputting a hyperspectral image into a trained hyperspectral anomaly detection model, performing image reconstruction and calculating the reconstruction error to obtain the hyperspectral anomaly detection result. The training process of the hyperspectral anomaly detection model is as follows: 1) constructing training samples; 2) establishing a variational background inference generative adversarial network; 3) obtaining sampled data latent variables; 4) inputting the latent variable z and a normal distribution sample into a latent variable discriminator for adversarial training; 5) inputting the latent variable and the normal distribution sample into a generator to generate two generated samples respectively; 6) jointly inputting the training samples and the two generated samples into a sample discriminator; 7) repeating steps 3) to 6) until the loss function of the variational background inference generative adversarial network converges, completing the training of the model. Compared with the prior art, the present invention has the advantages of high detection accuracy and the like.

Description

Technical Field

[0001] The present invention relates to the field of hyperspectral anomaly detection, and particularly to a hyperspectral anomaly detection method based on a variational background inference generative adversarial network. Background Art

[0002] With the development of hyperspectral imaging spectrometers, the spectral information of remote sensing images has rapidly evolved from panchromatic, multispectral to hyperspectral, propelling remote sensing technology into a brand-new stage. Hyperspectral images feature high spectral resolution, with each pixel in the obtained image having spectral information in hundreds of bands. Based on the different spectral characteristics among various ground objects, hyperspectral remote sensing has been widely applied in aspects such as ground object classification, quantitative inversion, target detection, and ecological environment monitoring. Hyperspectral image target detection can classify detection algorithms into two categories according to whether the target information is known. One category is the target detection algorithm with known target information, which mainly utilizes the spectral information of the known target and obtains the detected target by matching it with the target spectral curve. Usually, it is difficult to obtain the prior knowledge of the target and the background. Therefore, the other category is the anomaly detection algorithm with unknown target information.

[0003] In the existing anomaly detection methods based on deep learning, various variants of generative adversarial networks have problems such as unstable training, mode collapse, and low quality of generated samples, resulting in a high false alarm rate and low detection accuracy in anomaly detection. In addition, due to the lack of prior knowledge of the target and the background, the existing technology does not fully consider the statistical distribution characteristics of the background and anomalies. The training samples contain background and anomaly samples that are difficult to separate, and the anomaly samples will affect the training process of the network. Therefore, the detection accuracy of the reconstruction error will decrease. Summary of the Invention

[0004] The purpose of the present invention is to provide a hyperspectral anomaly detection method based on a variational background inference generative adversarial network with high detection accuracy to overcome the above-mentioned defects existing in the prior art.

[0005] A hyperspectral anomaly detection method based on a variational background inference generative adversarial network includes the following steps:

[0006] Input the hyperspectral image into the trained hyperspectral anomaly detection model, perform image reconstruction and calculate the reconstruction error, and obtain the hyperspectral anomaly detection result based on the reconstruction error and the set anomaly threshold.

[0007] Among them, the training process of the hyperspectral anomaly detection model is as follows:

[0008] 1) Obtain the original hyperspectral data, and construct training samples based on the preprocessed original hyperspectral data. The original hyperspectral data includes the spectra of all hyperspectral pixels.

[0009] 2) Establish a variational background inference generative adversarial network. The variational background inference generative adversarial network includes an encoder E, a generator G, a sample discriminator Ds, and a latent variable discriminator Dl. The loss function of the variational background inference generative adversarial network includes a variational inference loss, a spectral reconstruction loss, an adversarial loss, and a feature matching loss. The spectral reconstruction loss includes a mean square error and a spectral angle. The feature matching loss is constructed in the intermediate layer of the sample discriminator Ds and the latent variable discriminator Dl;

[0010] 3) Input the training samples into the encoder E to obtain the variance and mean of the variables. According to the variance and mean, perform reparameterization to obtain the sampled data latent variable z;

[0011] 4) Input the latent variable z and the normal distribution sample ξ into the latent variable discriminator Dl for adversarial training. The latent variable discriminator Dl guides the reparameterized data to match the prior distribution, and the prior distribution is the normal distribution sample ξ;

[0012] 5) Input the latent variable z and the normal distribution sample ξ into the generator G to generate two generated samples respectively, namely the spectral features x reconstructed by the generator G from the encoder Gz and the fake spectrum generated by the generator G from the normal distribution sample ξ;

[0013] 6) Input the training samples and the two generated samples generated by the generator G into the sample discriminator Ds together to add detailed features;

[0014] 7) Repeat steps 3) - 6) until the loss function of the variational background inference generative adversarial network converges to complete the training of the model.

[0015] Further, the expressions of each loss function are as follows:

[0016]

[0017]

[0018]

[0019]

[0020] Among them, is the total loss of the generator G, and L Ds are the generation loss and the adversarial loss of the adversarial training between the generator G and the sample discriminator Ds respectively. L Gl and L Dl are the generation loss and the adversarial loss of the adversarial training between the encoder E and the latent variable discriminator Dl respectively. L MSE is the mean square error, and L SAM is the spectral angle. is the total loss of the encoder E and the learned background distribution, L KL is the variational inference loss, is the feature matching loss between the data features extracted by the Ds network and the features of the latent variable z extracted by the Dl network, L fm_G is the feature matching loss between the features of two generated samples extracted by the Ds network and the features of the latent variable z extracted by the Dl network, L θDf and L θDb are the total losses of the sample discriminator Ds and the latent variable discriminator Dl respectively, and λ1, λ2, λ3 and λ4 are the adjustment parameters of each loss term.

[0021] Furthermore, the encoder E consists of three fully connected layers and is activated by the Leaky ReLU function.

[0022] Furthermore, the generator G consists of four fully connected layers. The output dimension of the generator G is the same as the number of bands of the hyperspectral image, and the last layer of the generator G is activated by the Sigmoid function.

[0023] Furthermore, both the sample discriminator Ds and the latent variable discriminator Dl consist of four fully connected layers, and the last layers of the sample discriminator Ds and the latent variable discriminator Dl are both activated by the Sigmoid function.

[0024] Furthermore, the training objective of the variational inference loss is to learn the distribution of the latent variable z to sample from the real data distribution.

[0025] Furthermore, the adversarial loss is the adversarial loss of the sample discriminator Ds and the latent variable discriminator Dl constructed by using the optimal transport distance loss based on gradient penalty.

[0026] Furthermore, the training objective of the feature matching loss is to tightly couple the samples that make up the variational background inference generative adversarial network with the latent variable generative adversarial network.

[0027] Furthermore, the normal distribution sample ξ is a standard normal distribution, and its expression is ξ = N(0,1).

[0028] Furthermore, the preprocessing is specifically: the original hyperspectral data is normalized by using min-max normalization.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] (1) In view of the complex distribution characteristics of the background and the low probability characteristics of abnormal phenomena, the present invention introduces a variational inference method, so that the hyperspectral background distribution can converge to a multivariate normal distribution, thereby effectively avoiding the interference of abnormal samples on the network background learning.

[0031] (2) The present invention constructs a composite loss function from spectral reconstruction loss, adversarial loss, and feature matching loss to ensure the accurate reconstruction of the spectrum, achieve accurate learning of the hyperspectral background, and effectively separate the background from abnormal targets. Among them, the spectral reconstruction loss combines the mean square error (MSE) and spectral angle (SAM) to ensure the overall consistency and detailed matching of the generated spectrum.

[0032] (3) The present invention constructs a feature matching loss in the intermediate layer of two discriminator networks, tightly coupling the data samples with the latent variable generative adversarial network to obtain more stable generated data.

[0033] (4) The method of the present invention has the advantages of unsupervised learning, end-to-end automatic anomaly detection, etc. The variational background inference generative adversarial network model inputs all pixel spectra into the model and autonomously obtains the reconstructed image and reconstruction error, which can further improve the effect of anomaly-background separation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flowchart of the present invention;

[0035] Figure 2 is a structural diagram of the hyperspectral anomaly detection model of the present invention, where Figure 2 (a) is a structural diagram of the encoder, Figure 2 (b) is a structural diagram of the generator, Figure 2 (c) is a structural diagram of the sample discriminator, Figure 2 (d) is a structural diagram of the latent variable discriminator;

[0036] Figure 3 is a structural diagram of the variational background inference generative adversarial network. DETAILED DESCRIPTION OF THE INVENTION

[0037] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives detailed implementation manners and specific operation processes, but the protection scope of the present invention is not limited to the following embodiments.

[0038] In the prior art's anomaly detection methods based on deep learning, the autoencoder network is a deterministic mapping, and it is difficult to handle the changes in background and abnormal samples; various variants of the generative adversarial network have problems such as unstable training, mode collapse, and low quality of generated samples, resulting in a high false alarm rate and low detection accuracy in anomaly detection. In addition, due to the lack of prior knowledge of the target and background, the prior art does not fully consider the statistical distribution characteristics of the background and anomalies; the training samples contain background and abnormal samples, and the abnormal samples will affect the training process of the network, so the detection accuracy of the reconstruction error will be reduced.

[0039] Therefore, the present invention provides a hyperspectral anomaly detection method based on variational background inference generative adversarial network, aiming at the deficiencies of the prior art, and providing a variational background inference generative adversarial network (VBIGAN) anomaly detection method applied to complex hyperspectral backgrounds. This method introduces a variational inference method for the complex distribution characteristics of the background and the low-probability characteristics of anomaly phenomena, enabling the hyperspectral background distribution to converge to a multivariate normal distribution, thus effectively avoiding the interference of abnormal samples on the network background learning. In addition, an adversarial learning method for samples and latent variables GAN is designed to establish a connection between data samples and latent samples to effectively capture the distribution characteristics of the background. The present invention constructs a composite loss function from spectral reconstruction loss, adversarial loss, and feature matching loss to ensure the accurate reconstruction of the spectrum, achieve accurate learning of the hyperspectral background, and effectively separate the background from abnormal targets. Among them, the spectral reconstruction loss incorporates the mean square error (MSE) L MSE and the spectral angle (SAM) L SAM , to ensure the overall consistency and detail matching of the generated spectrum; the present invention constructs a feature matching loss in the middle layer of the two discriminator networks, tightly coupling the data samples with the latent variable GAN to obtain more stable generated data. The method of the present invention has the advantages of unsupervised learning, end-to-end automatic anomaly detection, etc. The VBIGAN model autonomously obtains the reconstructed image and reconstruction error by inputting all pixel spectra into the model, which can further improve the effect of anomaly-background separation.

[0040] The flowchart of the hyperspectral anomaly detection method based on variational background inference generative adversarial network provided by the present invention is as Figure 1 shown. The method includes the following steps:

[0041] Input the hyperspectral image into the trained hyperspectral anomaly detection model, perform image reconstruction and calculate the reconstruction error, and obtain the hyperspectral anomaly detection result based on the reconstruction error and the set anomaly degree threshold.

[0042] Among them, the training process of the hyperspectral anomaly detection model is as follows:

[0043] 1) Obtain the original hyperspectral data, and construct training samples based on the preprocessed original hyperspectral data. The original hyperspectral data includes the spectra of all hyperspectral pixels;

[0044] 2) Establish a variational background inference generative adversarial network. The variational background inference generative adversarial network includes an encoder E, a generator G, a sample discriminator Ds, and a latent variable discriminator Dl. The loss function of the variational background inference generative adversarial network includes variational inference loss, spectral reconstruction loss, adversarial loss, and feature matching loss. The spectral reconstruction loss includes mean square error and spectral angle, and the feature matching loss is constructed in the middle layer of the sample discriminator Ds and the latent variable discriminator Dl;

[0045] 3) Input the training samples into the encoder E to obtain the variance and mean of the variables. According to the variance and mean, perform reparameterization to obtain the sampled data latent variable z;

[0046] 4) Input the latent variable z and the normal distribution sample ξ into the latent variable discriminator Dl for adversarial training. The latent variable discriminator Dl guides the reparameterized data to match the prior distribution, and the prior distribution is the normal distribution sample ξ;

[0047] 5) Input the latent variable z and the normal distribution sample ξ into the generator G to generate two generated samples respectively, which are the spectral features x reconstructed by the generator G from the encoder Gz and the fake spectrum generated by the generator G from the normal distribution sample ξ;

[0048] 6) Input the training samples and the two generated samples generated by the generator G into the sample discriminator Ds to add detailed features;

[0049] 7) Repeat steps 3) - 6) until the loss function of the variational background inference generative adversarial network converges to complete the training of the model.

[0050] In step 1), the preprocessing is specifically: using min - max normalization to normalize the original hyperspectral data.

[0051] In step 4), the normal distribution sample ξ is a standard normal distribution.

[0052] During the training process of the hyperspectral anomaly detection model, the hyperspectral anomaly detection model learns the hyperspectral background distribution characteristics. The hyperspectral anomaly detection model is established based on the variational background inference generative adversarial network. The variational background inference generative adversarial network consists of a sample generative adversarial network and a latent variable generative adversarial network. Each generative adversarial network has an independent discriminator, including the encoder E, the generator G, the sample discriminator Ds, and the latent variable discriminator Dl. The loss function of the variational background inference generative adversarial network is composed of composite losses, including variational inference loss, spectral reconstruction loss, adversarial loss, and feature matching loss.

[0053] The encoder E, the generator G, the sample discriminator Ds, and the latent variable discriminator Dl of the variational background inference generative adversarial network structure are as follows Figure 2As shown, the encoder E consists of three fully connected layers, where μ and σ come from two different fully connected layers and are activated using the Leaky ReLU (Leaky Rectified Linear Unit) function. The generator G consists of four fully connected layers, and the dimension of its output is the same as the number of bands in the hyperspectral image. The last layer is activated using the Sigmoid function. The sample discriminator Ds and the latent variable discriminator Dl also consist of four fully connected layers. The last layer of these two discriminators outputs the discrimination probability using the Sigmoid function, which is used for loss calculation during adversarial training.

[0054] The structure of the variational background inference generative adversarial network is as Figure 3 shown. Figure 3 Figure [figure number] shows the data flow during training. During the training process, the purpose of the encoder is to deceive the latent variable discriminator Dl into believing that the data sampled from the latent variables comes from the real distribution. The sample discriminator Ds can help the generator G generate a more accurate background spectrum.

[0055] The loss function includes variational inference loss, spectral reconstruction loss, adversarial loss, and feature matching loss, which effectively ensures the stable learning of the background distribution characteristics and reduces the possibility of mode collapse.

[0056] The expressions of each loss function are as follows:

[0057]

[0058]

[0059]

[0060]

[0061] Among them, is the total loss of the generator G, L Gs and L Ds are the generation loss and adversarial loss respectively for the adversarial training of the generator G and the sample discriminator Ds. L Gl and L Dl are the generation loss and adversarial loss respectively for the adversarial training of the encoder E and the latent variable discriminator Dl. L MSE is the mean square error, L SAM is the spectral angle, is the total loss of the encoder E and the learned background distribution, L KL is the variational inference loss, is the feature matching loss between the data features taken and the features of the latent variable z extracted by the Dl network. L fm-G is the feature matching loss between the features of two generated samples extracted by the Ds network and the features of the latent variable z extracted by the Dl network. and They are the total losses of the sample discriminator Ds and the latent variable discriminator Dl respectively, and λ1, λ2, λ3, and λ4 are the adjustment parameters of each loss term respectively.

[0062] The training objective of the variational inference loss is to learn the distribution of the latent variable z to sample from the true data distribution. Through reparameterization z = μ + σξ, where ξ = N(0, 1) is the prior distribution, and μ and σ are encoded by the encoder E. The spectral reconstruction loss consists of the mean squared error (MSE) L MSE and the spectral angle mapper (SAM) L SAM together. Its training objective is to make the generated spectrum more similar to the original spectrum. The mean squared error is used to calculate the difference between the generated spectrum and the original spectrum, and the SAM is further introduced as an additional constraint to estimate the similarity between the two spectra and effectively control the direction error. The lower the SAM value, the higher the matching degree between the spectral detail features of the two vectors. The adversarial loss is the adversarial loss between the sample discriminator Ds and the latent variable discriminator Dl constructed by using the optimal transport distance (Wasserstein) loss based on gradient penalty. In the latent variable discriminator Dl, the input contains two distribution samples from the sample ξ = N(0, 1) and the sample z ~ p(z) encoded by E. By minimizing E and maximizing D1 on L Dl the encoder E can accurately represent the hyperspectral background distribution. In the sample discriminator Ds, the input contains the real sample x, two generated samples G(z) and G(ξ). By minimizing G and maximizing Ds on L Df the generator G can generate more realistic spectra. The training purpose of the feature matching loss L fm is to tightly couple the data samples with the latent variable generative adversarial network to obtain more stable generated data. By using the middle layers of the networks of the sample discriminator Ds and the latent variable discriminator Dl, the Euclidean distance between the discriminant features is measured.

[0063] In the present invention, after image reconstruction, the reconstructed hyperspectral image can be obtained through the network trained by the encoder E and the generator G. The calculation formula is as follows:

[0064] X R = G(E(X))

[0065] After reconstruction, the present invention uses the l2 norm criterion between the original spectrum and the generated spectrum to detect abnormal targets, that is, based on the reconstruction error and the set anomaly threshold, the hyperspectral anomaly detection result is obtained. Specifically:

[0066] If the reconstruction error, that is, the anomaly degree, is higher than the anomaly threshold, the pixel is an abnormal pixel; otherwise, it is a background pixel, thereby obtaining the hyperspectral anomaly detection result of the image.

[0067] The calculation formula of the reconstruction error is as follows:

[0068]

[0069] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field according to the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art should fall within the protection scope determined by the claims.

Claims

1. A hyperspectral anomaly detection method based on a variational background inference generative adversarial network, characterized in that It includes the following steps: Input the hyperspectral image into the trained hyperspectral anomaly detection model, perform image reconstruction and calculate the reconstruction error, and obtain the hyperspectral anomaly detection result based on the reconstruction error and the set anomaly threshold. Among them, the training process of the hyperspectral anomaly detection model is as follows: 1) Obtain the original hyperspectral data, and construct training samples based on the preprocessed original hyperspectral data. The original hyperspectral data includes the spectra of all hyperspectral pixels. 2) Establish a variational background inference generative adversarial network. The variational background inference generative adversarial network includes an encoder E, a generator G, a sample discriminator Ds, and a latent variable discriminator Dl. The loss function of the variational background inference generative adversarial network includes a variational inference loss, a spectral reconstruction loss, an adversarial loss, and a feature matching loss. The spectral reconstruction loss includes mean squared error and spectral angle. The feature matching loss is constructed in the middle layer of the sample discriminator Ds and the latent variable discriminator Dl. 3) Input the training samples into the encoder E to obtain the variance and mean of the variables. According to the variance and mean, perform reparameterization to obtain the sampled data latent variable z. 4) Input the latent variable z and the normal distribution sample ξ into the latent variable discriminator Dl for adversarial training. The latent variable discriminator Dl guides the reparameterized data to match the prior distribution, which is the normal distribution sample ξ ; 5) Input the latent variable z and the normal distribution samples ξ into the generator G, and generate two generated samples respectively, which are the spectral features reconstructed by the generator G from the encoder and the fake spectra generated by the generator G from the normal distribution samples ξ ; 6) Input the training samples and two generated samples generated by the generator G into the sample discriminator Ds together to increase the detailed features. 7) Repeat steps 3) to 6) until the loss function of the variational background inference generative adversarial network converges to complete the training of the model. The expressions of each loss function are as follows: Among them, is the total loss of the generator G, and are the generation loss and adversarial loss of the adversarial training between the generator G and the sample discriminator Ds respectively, and are the generation loss and adversarial loss of the adversarial training between the encoder E and the latent variable discriminator Dl respectively, is the mean square error, is the spectral angle, is the total loss of the encoder E and the learned background distribution, is the variational inference loss, is the feature matching loss between the data features extracted by the Ds network and the features of the latent variable z extracted by the Dl network, is the feature matching loss between the features of two generated samples extracted by the Ds network and the features of the latent variable z extracted by the Dl network, and are the total losses of the sample discriminator Ds and the latent variable discriminator Dl respectively, , , and are the adjustment parameters of each loss term respectively.

2. A hyperspectral anomaly detection method for a variational background inference generative adversarial network according to claim 1, characterized in that The encoder E is composed of three fully connected layers and is activated by the Leaky ReLU function.

3. A hyperspectral anomaly detection method based on a variational background inference generative adversarial network according to claim 1, characterized in that The generator G is composed of four fully connected layers. The output dimension of the generator G is the same as the number of bands of the hyperspectral image. The last layer of the generator G is activated by the Sigmoid function.

4. A hyperspectral anomaly detection method based on a variational background inference generative adversarial network according to claim 1, characterized in that, Both the sample discriminator Ds and the latent variable discriminator Dl are composed of four fully connected layers. The last layer of both the sample discriminator Ds and the latent variable discriminator Dl is activated by the Sigmoid function.

5. A hyperspectral anomaly detection method for a variational background inference generative adversarial network according to claim 1, characterized in that The training objective of the variational inference loss is to learn the distribution of the latent variable z to sample the true data distribution.

6. The hyperspectral anomaly detection method of a variational background inference generative adversarial network according to claim 1, wherein The adversarial loss is the adversarial loss of the sample discriminator Ds and the latent variable discriminator Dl constructed by using the optimal transport distance loss based on gradient penalty.

7. A hyperspectral anomaly detection method based on a variational background inference generative adversarial network according to claim 1, characterized in that The training objective of the feature matching loss is to tightly couple the samples and the latent variable generative adversarial network that make up the variational background inference generative adversarial network.

8. A hyperspectral anomaly detection method based on the variational background inference generative adversarial network according to claim 1, characterized in that Normal distribution sample ξ is the standard normal distribution, and its expression is .

9. A hyperspectral anomaly detection method for a variational background inference generative adversarial network according to claim 1, characterized in that The specific preprocessing is: perform normalization processing on the original hyperspectral data by using min-max normalization.

Images