Hyperspectral target detection method based on adversarial autoencoders and attention mechanisms

Abstract

This invention discloses a hyperspectral target detection method based on adversarial autoencoders and attention mechanisms, comprising: performing coarse detection on an input hyperspectral image; obtaining background samples as clean as possible as training data by adjusting the binarization threshold of the coarse detection result image; training a model based on adversarial autoencoders and attention mechanisms using the prepared training data to obtain a model that can accurately reconstruct the background of the hyperspectral image; inputting a test hyperspectral image into the network for reconstruction; calculating the spectral distance between the reconstructed result and the original hyperspectral image pixel by pixel to obtain a distance map; and applying background suppression to the distance map to obtain the final detection result image. This invention introduces a spectral attention mechanism to assign weights to the spectral dimensions, accelerating the training and convergence speed of the network; and overcomes the problem of imbalanced positive and negative samples in hyperspectral images by selecting only background samples as training data to train the image reconstruction model based on adversarial autoencoders.

Description

Technical Field

[0001] This invention relates to a hyperspectral target detection method based on adversarial autoencoders and attention mechanisms, belonging to the technical field of image processing. Background Technology

[0002] Target detection is an important research and application area in hyperspectral remote sensing. Compared to target detection in high-resolution images, which focuses on spatial geometry, target detection in hyperspectral images pays more attention to subtle spectral feature differences between the target of interest and the background. Hyperspectral images are characterized by large data volumes, which means they contain a lot of redundant information, affecting processing speed. When using deep learning-based methods for hyperspectral target detection, the targets are usually small, and there is a problem of imbalanced positive and negative samples, resulting in a lack of training data that affects the accuracy of target detection. Summary of the Invention

[0003] The purpose of this invention is to provide a hyperspectral target detection method based on adversarial autoencoders and attention mechanisms to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a hyperspectral target detection method based on adversarial autoencoders and attention mechanisms, comprising the following steps:

[0005] Step 1: Perform coarse detection on the input hyperspectral image using the CEM hyperspectral target detection algorithm. Adjust the binarization threshold of the coarse detection result image to obtain the purest possible background samples as training data.

[0006] Step 2: Improve the existing network model based on adversarial autoencoders by introducing a spectral attention mechanism. Input the training data into the improved network. First, the spectral attention mechanism module assigns weights to the spectral dimensions of the input hyperspectral data. Then, the weighted data is input into the adversarial autoencoder network. The autoencoder acts as the network's generator. The input data x is processed by the encoder to obtain the hidden vector z, and then by the decoder to obtain the reconstructed result x'. The discriminator determines whether z is real data. The loss function of the adversarial network and the loss function of the autoencoder are combined to continuously optimize the network, making the output reconstructed result x' as close as possible to the input data x, thus obtaining a model that can accurately reconstruct the background of the hyperspectral image. Step 3: Input the test hyperspectral image into the network for reconstruction. Calculate the spectral distance between the reconstructed result and the original hyperspectral image pixel by pixel to obtain a distance map. Apply background suppression to the distance map to obtain the final detection result map.

[0007] Furthermore, in step one, the existing algorithm selected for coarse detection is the CEM algorithm. The CEM algorithm designs an FIR linear filter to minimize the output energy of the filter while satisfying the constraint condition. When binarizing the coarse detection result, the binarization threshold is lowered to achieve the highest coarse detection accuracy under the condition of high false alarm rate, and 80% of the background samples are selected as training data.

[0008] Furthermore, in step two, a spectral attention mechanism is introduced, which uses two 1×1 convolutional layers to focus on the input data x. t ∈R H×W×C x t Representing pixels, R is hyperspectral data, and H, W, C are the spatial dimensions and spectral dimensions of the hyperspectral data. A linear mapping is performed to obtain a query vector with full spatial resolution. and half-channel resolution key vector Then, the query vector is remapped to the key vector through matrix multiplication, the autocorrelation of the spectral dimension is calculated, and the attention coefficients are obtained, denoted as the value vector. To avoid excessive loss of continuity, the channel is set to C / 2. The above process is expressed by the following formula:

[0009]

[0010]

[0011]

[0012] in F represents a 1×1 convolutional layer. R (·) indicates that the size is adjusted using the Reshape function to achieve a match. This represents a matrix multiplication operation. After channel adjustment and sigmoid activation, the weight factor for each channel is obtained. Then, the original data x is... t ∈R H×W×C Multiplying by the weighting factor yields the recalibrated input data:

[0013]

[0014] The aforementioned spectral attention model assigns weights to the spectral dimensions of the input spectral data, enabling the network to focus its attention on the more information-rich spectral dimensions. An adversarial autoencoder is employed as the network's basic architecture. The network is trained to reconstruct hyperspectral data containing only background information. During training, two loss functions are jointly used to achieve higher detection accuracy. The loss functions employed are shown below:

[0015] l total =l a +l e

[0016] In the formula, l total Let l be the total loss function. a For the loss function of adversarial networks, l e For the loss function of the encoder with added target suppression, l a The calculation formula is as follows:

[0017]

[0018] In the formula, G represents the generator, D represents the discriminator, D(x) represents the output of the discriminator when the input is the original image, and D(G(z)) represents the output of the discriminator when the input is the image synthesized by the generator. p represents the expected value of this distribution. data (x) represents the spatial distribution of the real data, p z (z) represents the spatial distribution of the synthetic sample, l e The calculation formula is as follows:

[0019]

[0020] In the formula, h i It is the input hyperspectral image sample, h' i This is the reconstructed hyperspectral image, s i It is to reconstruct the spectrum h' i Plotting the spectral angles of the target spectrum d.

[0021] Furthermore, in step three, the spectral distance d is calculated pixel by pixel for the input hyperspectral data H and the reconstructed result H'. si The formula is expressed as follows:

[0022]

[0023] T represents the matrix transpose.

[0024] d is calculated pixel by pixel. si The final distance map D is obtained. S , and then d si Multiplying by a non-negative number based on the coarse detection result yields the detection result for each pixel, expressed by the following formula:

[0025] d fi =q(y i )d si

[0026] The nonlinear function q(x) is used to suppress the background spectrum while keeping the target spectrum unchanged, and is defined as follows:

[0027]

[0028] Where λ is a parameter that adjusts the inhibition ability, and x is a non-negative number y i The final test result is expressed as follows:

[0029] D F =[d f1 ,d f2 ,...,d fM×N ]

[0030] Where D F For the final distance map, d f1 ,d f2 ,...,d fM×N The distance to each pixel.

[0031] Compared with the prior art, the significant advantages of this invention are:

[0032] (1) This invention proposes a hyperspectral target detection algorithm based on adversarial autoencoders and attention mechanisms. This method can effectively utilize the spectral information in hyperspectral images, obtain an accurate background reconstruction model through learning, and thus achieve accurate hyperspectral image target detection.

[0033] (2) A spectral attention mechanism is introduced to address the problem of excessively long processing time caused by spectral redundancy in hyperspectral images. The spectral attention model focuses on the spectral dimension of the hyperspectral image, that is, by assigning weights, it focuses the network attention on the spectral dimension with higher information content. Through the spectral attention mechanism, the convergence speed of the algorithm is improved and the running time is reduced. Attached Figure Description

[0034] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the overall algorithm flow of the present invention;

[0036] Figure 2 This is a schematic diagram of the spectral attention model adopted in this invention;

[0037] Figure 3 This is a schematic diagram of the counter-autoencoder structure adopted in this invention;

[0038] Figure 4 This is a schematic diagram comparing the target detection results of this invention with other algorithms;

[0039] Figure 5 This is a schematic diagram comparing the ROC curve of this invention with other algorithms. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] See Figure 1 This invention is a hyperspectral target detection method based on adversarial autoencoders and attention mechanisms, including the overall process of background reconstruction network and how to test hyperspectral images to obtain the final detection result map.

[0042] A hyperspectral target detection method based on adversarial autoencoders and attention mechanisms includes the following steps:

[0043] Step 1: Perform coarse detection on the input hyperspectral image. By adjusting the binarization threshold of the coarse detection result image, obtain the purest possible background samples as training data.

[0044] The existing algorithm used for coarse detection is the CEM algorithm. The CEM algorithm designs an FIR linear filter to minimize the output energy of the filter while satisfying the constraint condition. When binarizing the detection results, the binarization threshold is lowered to achieve the highest coarse detection accuracy under the condition of high false alarm rate. 80% of the background samples are selected as training data.

[0045] Step 2: Train the model based on adversarial autoencoders and attention mechanisms. The spectral dimension of the input training data is assigned weights through the spectral attention mechanism module. Then, the network based on adversarial autoencoders is trained to learn to reconstruct the image, resulting in a model that can accurately reconstruct the background of the hyperspectral image.

[0046] like Figure 2-3 As shown, a spectral attention mechanism is introduced, which assigns weights to the spectral dimensions to make the network focus its attention on the spectral dimensions with higher information content; the spectral attention model is as follows. Figure 2 As shown, the specific method is as follows: Two 1×1 convolutional layers are used to process the input data x. t ∈R H×W×C Perform a linear mapping to obtain a query vector with full spatial resolution. and half-channel resolution key vector Then, the query vector is remapped to the key vector through matrix multiplication, the autocorrelation of the spectral dimension is calculated, and the attention coefficients are obtained, denoted as the value vector. Setting the channel to C / 2 is to avoid excessive loss of continuity. The above process can be expressed by the following formula:

[0047]

[0048]

[0049]

[0050] in F represents a 1×1 convolutional layer. R (·) indicates that the size is adjusted using the Reshape function to achieve a match. This represents a matrix multiplication operation. After channel adjustment and Sigmoid activation, the weight factor for each channel can be obtained. Then, the original data x... t ∈R H×W×C Multiplying by the weighting factor yields the recalibrated input data:

[0051]

[0052] The aforementioned spectral attention model assigns weights to the spectral dimensions of the input spectral data, enabling the network to focus its attention on the more information-rich spectral dimensions. An adversarial autoencoder is employed as the network's basic architecture. The network is trained to reconstruct hyperspectral data containing only the background. During training, two loss functions are jointly used to achieve higher detection accuracy. The loss functions used are shown below:

[0053] l total =l a +l e

[0054] In the above formula, l total Let l be the total loss function. a For the loss function of adversarial networks, l e For the loss function of the encoder with added target suppression, l a The calculation formula is as follows:

[0055]

[0056] In the above formula, G and D represent the generator and discriminator, respectively. D(x) represents the output of the discriminator when the input is the original image, and D(G(z)) represents the output of the discriminator when the input is the image synthesized by the generator. p represents the expected value of this distribution. data (x) represents the spatial distribution of the real data, p z (z) represents the spatial distribution of the synthetic sample. e The calculation formula is as follows:

[0057]

[0058] In the above formula, h i It is the input hyperspectral image sample, h' i This is the reconstructed hyperspectral image, s i It is to reconstruct the spectrum h' i Plotting the spectral angles of the target spectrum d.

[0059] Step 3: Input the test hyperspectral image into the network for reconstruction, and calculate the spectral distance between the reconstruction result and the original hyperspectral image pixel by pixel to obtain a distance map;

[0060] Input hyperspectral data H and reconstruction result H' to calculate the spectral distance d pixel by pixel. si The formula is expressed as follows:

[0061]

[0062] d is calculated pixel by pixel. si The final distance map D can be obtained. S , and then d si The final detection result is obtained by multiplying by a non-negative number based on the coarse detection result, expressed by the following formula:

[0063] d fi =q(y i )d si

[0064] The nonlinear function q(x) is used to suppress the background spectrum while keeping the target spectrum unchanged, and is defined as follows:

[0065]

[0066] Where λ is a parameter that adjusts the inhibition ability, the final detection result can be expressed as:

[0067] D F =[d f1 ,d f2 ,...,d fM×N ]

[0068] in Figure 4 This is a schematic diagram comparing the detection results of this invention with other algorithms. Figure 4 The images show the detection results of the algorithm proposed in this invention and five comparative algorithms on the same real hyperspectral dataset. As can be seen from the results, the algorithm proposed in this invention has high detection accuracy and low false alarm rate. Figure 5 This is a schematic diagram of the ROC curves of the algorithm proposed in this invention and the comparison algorithm. The ROC curve is a commonly used evaluation index in the field of hyperspectral target detection. The area under the curve (AUC) can be calculated from this curve as a quantitative indicator to prove that the detection accuracy of this invention is superior to other comparison algorithms.

[0069] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0070] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A hyperspectral target detection method based on adversarial autoencoder and attention mechanism, characterized in that, The method comprises the following steps: Step one: coarse detection is performed on the input hyperspectral image through a hyperspectral target detection algorithm CEM, and a binary threshold of a coarse detection result image is adjusted to obtain as pure background samples as possible as training data; Step two: an existing network model based on an adversarial autoencoder is improved, a spectral attention mechanism is introduced, training data are input into the improved network, the spectral attention mechanism module assigns weights to spectral dimensions of the input hyperspectral data, then the data with weights are input into the adversarial autoencoder network, the autoencoder serves as a generator of the network, hidden vectors z are obtained from an encoder through input data x, then a reconstruction result x' is obtained through a decoder, a discriminator is used to determine whether z is real data, a loss function of the joint adversarial network and a loss function of the autoencoder are constantly optimized to make the output reconstruction result x' as close as possible to the input data x, and a model capable of accurately reconstructing a hyperspectral image background is obtained; In step two, the spectral attention mechanism is introduced, which uses two 1x1 convolutional layers to process the input data x t ∈R H ×W×C , x t represents a pixel point, R is the hyperspectral data, H, W, and C are the spatial dimensions and spectral dimension of the hyperspectral data, and a linear mapping is performed to obtain the full spatial resolution query vector f c q and the semi-channel resolution key vector f c k Then, the query vector is remapped to the key vector through matrix multiplication operation, the autocorrelation of the spectral dimension is calculated, and the attention coefficient is obtained, denoted as the value vector f c v In order to avoid excessive loss of continuity, the channel is set to C / 2, and the above process is represented by the following formula: wherein represents a 1x1 convolutional layer, F R (·) represents adjusting the size to match by the Reshape function, represents a matrix multiplication operation, after the channel adjustment and the Sigmoid activation, a weight factor for each channel is obtained, and then, the original data x t ∈R H×W×C and the weight factor are multiplied to obtain the recalibrated input data: Step three: a test hyperspectral image is input into the network for reconstruction, a spectral distance is calculated between the reconstruction result and the original hyperspectral image pixel by pixel to obtain a distance map, and background suppression is performed on the distance map to obtain a final detection result image.

2. The hyperspectral target detection method based on adversarial autoencoder and attention mechanism according to claim 1, characterized in that: In step one, the selected existing algorithm for coarse detection is the CEM algorithm, the CEM algorithm is designed to be a FIR linear filter, so that the output energy of the filter is minimum under the condition of meeting the constraint condition formula, the binary threshold of the coarse detection result is lowered in the step of binaryzation, so that the highest coarse detection accuracy is achieved under the condition of a high false alarm rate, and 80% of the background samples are selected as training data.

3. The hyperspectral target detection method based on adversarial autoencoder and attention mechanism of claim 1, wherein: Through the above spectral attention model, weights are assigned to spectral dimensions of the input spectral data, so that the network can pay attention to spectral dimensions with more information; the adversarial autoencoder is used as the basic architecture of the network, the network trained based on the adversarial autoencoder learns to reconstruct hyperspectral data with only background, and in the training process, two kinds of loss functions are used together to achieve higher detection accuracy, the loss functions used are as follows: l total ＝l a +l e where l total is the total loss function, l a is the loss function of the adversarial network, l e is the loss function of the encoder with target inhibition added, l a The calculation formula of l is as follows: In the formula, G represents a generator, D represents a discriminator, D(x) represents the output of the discriminator when the input is an original image, and D(G(z)) represents the output of the discriminator when the input is an image synthesized by the generator, represents the expected value of the distribution, p data (x) represents the spatial distribution of real data, p z (z) represents the spatial distribution of synthetic samples, l e The calculation formula is as follows: where h i is the input hyperspectral image sample, h' i is the reconstructed hyperspectral image, s i is the reconstructed spectrum h' i and the spectral angle mapping of the target spectrum d.

4. The hyperspectral target detection method based on adversarial autoencoder and attention mechanism according to claim 1, characterized in that: In step three, the spectral distance d is calculated pixel by pixel for the input hyperspectral data H and the reconstruction result H' si This is expressed in formula as follows: T represents matrix transposition, d is calculated pixel by pixel. si The final distance map D is obtained. S , and then d si Multiplying by a non-negative number based on the coarse detection result yields the detection result for each pixel, expressed by the following formula: d fi = q(y i )d si wherein the nonlinear function q(x) is used to suppress the background spectrum while keeping the target spectrum unchanged, and is defined as follows: where λ is a parameter that adjusts the ability of inhibition, x is a non-negative number y i The final test result is represented as: D F = [d f1 ,d f2 ,...,d fM×N ] where D F is the resulting distance map, d f1 ,d f2 ,...,d fM×N is the distance for each pixel.

Images