Multi-scale twin hyperspectral target detection method and system

Abstract

The invention relates to a multi-scale twin hyperspectral target detection method and system, and belongs to the technical field of deep learning. The method comprises the following steps: S1, preparing initial target and background data of a hyperspectral image; s2, performing adversarial sample generation on the hyperspectral remote sensing image to obtain training sample data; s3, inputting the generated sample into a multi-scale twin target detection network for training; and S4, after training is completed, performing target detection on the test sample to obtain a result. The performance of the method is superior to that of other hyperspectral image target detection methods, the method can generate more sufficient and more diversified training samples, and the method has advantages in the aspect of background suppression compared with other methods.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning and relates to a multi-scale twin hyperspectral target detection method and system. Background Art

[0002] With the development of remote sensing technology, the spatial resolution and spectral resolution of remote sensing images acquired by sensors have also been greatly improved. Among them, hyperspectral remote sensing technology has become one of the cutting-edge technologies for remote sensing earth observation. Among them, hyperspectral target detection focuses on locating and identifying specific target pixels in hyperspectral images, aiming to separate targets of interest from various backgrounds, and usually requires very little prior target spectral information. The unique advantages of hyperspectral target detection have promoted its wide application in various fields, including but not limited to military reconnaissance, mineral exploration, environmental monitoring, and precision agriculture. Compared with traditional border-level detection that relies on spatial information, hyperspectral target detection uses rich spectral information to open up new areas of earth observation.

[0003] In early research, traditional detection methods mainly include three categories: spectral matching, subspace model and statistical distribution hypothesis model. The spectral matching model uses matched filters to suppress the background while highlighting the target; the subspace model algorithm usually detects the target of interest by establishing a subspace model and projecting the pixels to be detected into a subspace related to the target; the statistical distribution hypothesis model usually introduces certain statistical distribution assumptions for the target and the background to construct a generalized likelihood ratio detector. With the development of machine learning theory and methods, popular technologies represented by kernel methods, sparse learning, low-rank learning and metric learning have been widely applied to the problem of hyperspectral target detection. However, these methods often have problems such as low detection efficiency, difficulty in feature selection and poor robustness. In recent years, methods based on deep learning have been widely used, which can effectively separate the target from the background through abstract feature learning. Methods based on deep learning usually deal with the problem of hyperspectral target detection from two perspectives: one is to achieve target classification by constructing supervised training samples, which requires full use of expanded data for training; the other is to use unsupervised deep detectors, the core idea of ​​which is to extract recognition features through unsupervised learning networks and apply them to detect target pixels. Deep models require a large amount of training data to fully exert their advantages. However, in the hyperspectral target detection task, the general training sample enhancement strategy is almost simple and rough, and cannot generate target samples and background samples with obvious uniqueness. In addition, the existing depth-based hyperspectral target detection methods have room for improvement in their ability to integrate features of different scales. In the field of hyperspectral target detection, how to more fully integrate features of different stages and scales is the key to improving target detection accuracy. Summary of the invention

[0004] In view of this, the primary purpose of the present invention is to provide a multi-scale twin hyperspectral target detection method, the core idea of ​​which is the generation of different target and background samples and the learning of target highlighting and background suppressing features. The method consists of an adversarial sample generation network and a multi-scale twin convolutional network. In the sample generation stage, the idea of ​​target-background mutual adversarial learning is adopted to further capture the potential distribution of real samples and adversarially generate more representative target samples and background samples. Subsequently, the generated samples and previous targets are used to train the proposed multi-scale twin convolutional network, and a multi-scale feature extraction strategy is adopted to fully explore the spectral diversity information. In addition, the idea of ​​contrastive learning is introduced to construct positive and negative sample pairs in the twin convolutional network, which can effectively distinguish the differences between positive and negative pairs. In addition, in order to improve the visibility of the target and suppress background noise, we use a dedicated loss function at each stage.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A multi-scale twin hyperspectral target detection method comprises the following steps: S1: preparing initial target and background data of a hyperspectral image; S2: generating adversarial samples for the hyperspectral remote sensing image to obtain training sample data; S3: inputting the generated samples into a multi-scale twin target detection network for training; S4: after the training is completed, performing target detection on the test sample to obtain the result.

[0007] Furthermore, in step S1, initial target and background data of the hyperspectral image are prepared, specifically including: the target data uses the target prior t already existing in the data set, and then, the original pixels of the hyperspectral image are clustered using the target prior to obtain the initial background sample b.

[0008] Furthermore, in step S2, in the training sample generation stage, the target prior t and the background sample xb are respectively input into the generator G1 and the discriminator D1, and then the generated target sample is input into the decoder De1 again to obtain t′, and then input into the discriminator D2 together with the original target prior for discrimination, so as to achieve adversarial generation. Similarly, in the background sample generation stage, the initial background sample b and the generated target sample t′ are respectively input into the generator G1′ and the discriminator D1′, and then the generated background sample is input into the decoder De2 again to obtain b′, and then input into the discriminator D2′ together with the initial background sample for adversarial generation. Finally, the generated target t′ and background sample b′ constitute the training sample.

[0009] Further, in step S2, target samples are generated. First, the existing target prior t is input into the generator G1, and then t is mapped to the latent space through the encoder to obtain its compressed latent feature tz. Subsequently, tz is input into the decoder De1 to generate a new target sample t′ with similar features. In order to ensure that a sufficient number and variety of target samples are generated, Gaussian noise interference is introduced into the generated target sample t′. These perturbed and diversified samples are used as new target inputs for training, prompting the model to adapt to various forms of interference. This process is repeated until a sufficient number of target samples are generated. Then, the spectrum with low cosine similarity to the prior target t is processed using principal component analysis (PCA) to become a background sample xb, reducing the dimension of the background sample to match the dimension of tz. Subsequently, the background sample xb and the latent feature tz are input into the discriminator D1 for target-background adversarial training. In this process, the two generators correspond to the two discriminators, and adversarial training is used to achieve target-background adversarial generation. Specifically, the discriminator D1 corresponds to the generator G1, while the discriminator D2 corresponds to the generator G2 composed of G1 and the decoder De1.

[0010] To ensure that the generated target spectra better represent the distribution of real data, the latent features tz and background spectra are input into the discriminator network D1. The discriminator D1 is optimized to maximize the distinction between the target latent features tz and background samples. The adversarial loss is used to guide the optimization of the network:

[0011]

[0012] Among them, E x～p(x) [f(x)] represents the expected value of function f(x) when the variable x obeys the probability density function distribution.

[0013] The goal of the generator G2 is to generate samples similar to the real target spectrum, and the goal of the discriminator D2 is to distinguish the target spectrum generated by the generator G2 from the real target spectrum. Therefore, the adversarial loss formula between G2 and D2 is expressed as follows:

[0014]

[0015] Same as above, E. t～p(t) [f(t)] represents the expected value of function f(t) when variable t obeys the probability density function distribution;

[0016] In addition, in order to generate more representative and high-quality target samples, the difference between the generated targets and the target priors is calculated, and the mean squared error loss is used to quantify this difference:

[0017]

[0018] Finally, through adversarial training, the generator G2 is able to generate target samples t′ that are significantly different from background samples.

[0019] Further, in step S2, a background sample is generated. First, the initial background sample b obtained by clustering the original hyperspectral image is input into the generator G1′ to obtain the potential feature representation bz of the background sample. Subsequently, bz is input into the discriminator D1′. By performing PCA on the generated target sample, an input t″ of D1′ is obtained. Then, the potential representation bz of the background sample is input into the decoder De2 for decoding to obtain the reconstructed background data b′. Finally, the reconstructed background data b′ and the initial background sample b are input into the discriminator D2′ together for discrimination.

[0020] During the training process, the generator G1′ generates background latent feature representations bz that can deceive the discriminator D1′ as much as possible, while the discriminator D1′ strives to distinguish between the real target samples and the generated background feature representations. In this process, adversarial loss is applied to drive the optimization problem:

[0021]

[0022] Same as above, E. t”～p(t”) [f(t”)] represents the expected value of function f(t”) when variable t” obeys the probability density function distribution;

[0023] The generator G2′ aims to generate background spectrum samples that can deceive the discriminator network D2′. At the same time, the discriminator network d2′ can effectively distinguish the background spectrum samples generated by the generator G2′ from the real background spectrum samples, ensuring the quality and authenticity of the generated samples. The adversarial loss between G2′ and D2′ can be expressed as:

[0024]

[0025] Same as above, E. b～p(b) [f(b)] represents the expected value of function f(b) when variable b obeys the probability density function distribution;

[0026] In order to increase the difference between the reconstructed background samples and the target prior and ensure that the reconstructed samples can truly represent the background, the reconstruction loss L is designed. r ' and DIA loss L a Here, L r The purpose of 'is to minimize the error between the input data b and the reconstructed data b':

[0027]

[0028] In order to increase the difference between the reconstructed background samples and the target prior, the model is prompted to learn more different features to distinguish between the target and the background. We use N to represent the number of samples of the reconstructed background, and calculate the cosine similarity between each reconstructed background spectrum bi′ and the prior target t. The higher the value, the greater the similarity with the prior target. In order to amplify the difference between the target and the reconstructed background and reduce the similarity, the average similarity is used as the DIA loss:

[0029]

[0030] Finally, through adversarial training, the generator G2′ is able to produce background samples b′ that are purer and significantly different from the target samples.

[0031] Further, in step S3, the generated training samples are input into the feature extraction network. After the adversarial sample generation stage, the generated target t′ and background samples b′ constitute the training samples, and respectively constitute positive and negative sample pairs with the target prior t. Then, a multi-scale convolutional twin network is established for target detection. The generated target and background samples are input into the designed multi-scale convolutional twin network for multi-scale feature extraction and fusion. The multi-scale convolutional twin network proposed in the present invention is mainly composed of a multi-scale convolution module (MCM), a convolutional feature extraction module (CDN) and a multi-scale dilated convolution downsampling module (MDCDS).

[0032] Further, the spectral pixels are feature extracted through a multi-scale convolution module and two convolutional feature extraction modules. In the three interconnected modules, each output is seamlessly converted as the input of the next module. At the same time, the original spectral pixels are represented by the spectral features of the scale-void convolution downsampling module to obtain effective semantic information and spatial context. The two results are then fused, and finally, the output is mapped to a range between 0 and 1 for subsequent detection.

[0033] During the training process, the two inputs of the Siamese network are the prior target spectrum and the generated target / background samples. If the input of the upper branch is fixed to the target prior and the input of the lower branch is the target, the label of the training sample is set to 1, otherwise it is set to 0. In addition, we keep the same number of positive and negative samples. In the proposed network, there are two loss functions. The first is the contrastive loss function L c , where for the input sample pair (z0,z i ), the extracted network features are expressed as f(z0), f(z i ). We use the idea of ​​increasing the distance between negative samples and reducing the distance between positive samples, using the following

[0034] The loss function is used to guide model training. The formula is as follows:

[0035]

[0036] Among them, y represents the label of the sample pair, f represents the network output value, and m is a hyperparameter, which represents the distance degree of the negative sample pair.

[0037] Furthermore, in the training process of the multi-scale convolutional twin network, an AW-BCE loss method is proposed. The nonlinear function q(x) is used to suppress the background and highlight the target pixels. Since the similarity of the positive sample pairs is greater than that of the negative sample pairs, q(x) can maintain a higher value for the target pixels and a lower value for the background pixels, which can effectively suppress the background. The exponential suppression function is defined as:

[0038] q(x)=1-e -λx

[0039] AW-BCE loss function L s is defined as follows:

[0040]

[0041] Where N is the number of samples, k i is the weight of the i-th sample, d i is the distance of the i-th sample pair, l i is the label of the i-th sample. k i The calculation method is as follows:

[0042]

[0043] Among them, λ is a hyperparameter set to 10, k i The value of is in the range of [0,1], indicating the weight of the corresponding sample. Finally, BCE(d i ,l i ) represents the binary cross entropy loss function, which sums the weighted losses of all samples and divides them by the number of samples to get the final loss value. The total loss function consists of two components: contrast loss L c and AW-BCE loss L s .

[0044] L=wL c +(1-w)L s

[0045] Where w is a weighting parameter to balance the impact of each component on the overall calculation.

[0046] After the training is completed, a trained hyperspectral image target detection framework is obtained; the input sample is detected through the trained hyperspectral image target detection framework, and a hyperspectral image hyperspectral image target detection result map is output.

[0047] The beneficial effects of the present invention are:

[0048] The proposed multi-scale twin hyperspectral target detection network algorithm based on adversarial sample generation is divided into two stages: sample generation and multi-scale feature extraction. It realizes diversified training samples, solves the problem of background suppression, enhances the model's ability to capture different features in hyperspectral data, enhances the robustness and generalization ability of the network, and significantly improves the target detection effect. Experimental results on four hyperspectral image datasets show that the performance of the proposed AGMS is better than the current advanced hyperspectral image target detection method.

[0049] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0051] Figure 1 is a flow chart of the method of the present invention;

[0052] Figure 2 Schematic diagram of the multi-scale twin network (AGMS) based on adversarial sample generation;

[0053] Figure 3 is a flowchart for generating target samples;

[0054] Figure 4 It is a structural diagram of the multi-scale twin network of the present invention;

[0055] Figure 5 Visualization results of different target detection methods on four hyperspectral image datasets, including (a) ground truth map, (b) ACE. (c) CEM. (d) HCEM. (e) CSCR. (f) DSC. (g) TSCNTD. (h) LCNN-CD. (i) HTD-IRN. (j) AGMS;

[0056] Figure 6 This is a spectrum graph of a sample generated according to the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0058] Figure 1The flowchart of the method of the present invention is a multi-scale twin hyperspectral target detection method and system. As shown in Figure 1, in the data preparation stage, the target data uses the target prior existing in the data set, and uses the target prior to cluster the original pixels of the hyperspectral image to obtain the background sample. The multi-scale twin network generated based on the adversarial sample is shown in Figure 1. Figure 2 As shown, it can enhance the model's ability to capture different features in hyperspectral data. The method consists of two stages: an adversarial sample generation (ASG) network and a multi-scale convolutional twin detection network. First, the idea of ​​generative adversarial learning is used to generate adversarial target-background sample pairs based on a small number of target priors and different backgrounds, thereby generating high-quality training samples. In addition, a differential amplification loss (DIA) and an adaptive weighted binary cross entropy loss (AW-BCE) are proposed. Finally, a multi-scale convolutional twin network model is designed to achieve target detection through contrastive learning. The adversarial sample generation network designed in the present invention is used to generate richer, more diverse and fidelity training samples. The multi-scale convolutional twin network designed in the present invention is used to capture and fuse convolution kernel features at different scales, thereby enhancing the model's ability to capture different features in hyperspectral data. The differential amplification loss and adaptive weighted binary cross entropy loss designed in the present invention improve the background suppression ability of the method. Specifically, the technical solution of the present invention includes the following contents:

[0059] 1. Preparation of initial hyperspectral image target and background data: Generating training samples requires preparation of initial hyperspectral image target and background data. Figure 2 As shown, the target data uses the target prior t that already exists in the data set. Then, the target prior is used to cluster the original pixels of the hyperspectral image to obtain the initial background sample b.

[0060] 2. Adversarial sample generation: Figure 2 As shown in the figure, in the stage of generating target samples, the target prior t and background sample xb are input into the generator G1 and the discriminator D1 respectively, and then the generated target sample is input into the decoder De1 again to obtain t′, and then input into the discriminator D2 together with the original target prior for discrimination, so as to achieve adversarial generation. Similarly, in the stage of generating background samples, the initial background sample b and the generated target sample t′ are input into the generator G1′ and the discriminator D1′ respectively, and then the generated background sample is input into the decoder De2 again to obtain b′, and then input into the discriminator D2′ together with the initial background sample for adversarial generation. Finally, the generated target t′ and background sample b′ constitute the training sample.

[0061] 3. Target sample generation details such as Figure 3As shown. First, the existing target prior t is input into the generator G1, and then t is mapped to the latent space through the encoder to obtain its compressed latent feature tz. Subsequently, tz is input into the decoder De1 to generate a new target sample t′ with similar features. In order to ensure the generation of a sufficient number and variety of target samples, Gaussian noise interference is introduced into the generated target sample t′. These perturbed and diversified samples are used as new target inputs for training, and this process is repeated until a sufficient number of target samples are generated. Then, the spectrum with low cosine similarity to the prior target t is processed using principal component analysis (PCA) to become a background sample xb, reducing the dimension of the background sample to match the dimension of tz. This ensures that the discriminator can effectively process the target and background spectral information. Subsequently, the background sample xb and the latent feature tz are input into the discriminator D1 for target-background adversarial training. In this process, the two generators correspond to the two discriminators, and adversarial training is used to achieve target-background adversarial generation. Specifically, the discriminator D1 corresponds to the generator G1, while the discriminator D2 corresponds to the generator G2 composed of G1 and the decoder De1.

[0062] In order to generate target spectra that better represent the distribution of real data, the latent features tz and background spectra are input into the discriminator network D1. The discriminator D1 is optimized to maximize the distinction between the target latent features tz and background samples. The adversarial loss is used to guide the optimization of the network:

[0063]

[0064] Among them, E x～p(x) [f(x)] represents the expected value of function f(x) when the variable x obeys the probability density function distribution.

[0065] The goal of the generator G2 is to generate samples similar to the real target spectrum to deceive the discriminator D2. The goal of the discriminator D2 is to distinguish the target spectrum generated by the generator G2 from the real target spectrum. Therefore, the adversarial loss formula between G2 and D2 is expressed as follows:

[0066]

[0067] In addition, in order to generate more representative and high-quality target samples, the difference between the generated targets and the target priors is calculated, and the mean squared error loss is used to quantify this difference:

[0068]

[0069] Finally, through adversarial training, the generator G2 is able to generate target samples t′ that are significantly different from background samples.

[0070] 4. Then generate background samples. First, the initial background sample b obtained by clustering the original hyperspectral image is input into the generator G1′ to obtain the potential feature representation bz of the background sample. Subsequently, bz is input into the discriminator D1′. By performing PCA on the generated target sample, an input t″ of D1′ is obtained. Then, the potential representation bz of the background sample is input into the decoder De2 for decoding to obtain the reconstructed background data b′. Finally, the reconstructed background data b′ and the initial background sample b are input into the discriminator D2′ together for discrimination.

[0071] During the training process, the generator G1′ generates background latent feature representations bz that can deceive the discriminator D1′ as much as possible, while the discriminator D1′ strives to distinguish between the real target samples and the generated background feature representations. In this process, adversarial loss is applied to drive the optimization problem:

[0072]

[0073] The generator G2′ aims to generate background spectrum samples that can deceive the discriminator network D2′. At the same time, the discriminator network D2′ can effectively distinguish the background spectrum samples generated by the generator G2′ from the real background spectrum samples, ensuring the quality and authenticity of the generated samples. The adversarial loss between G2′ and D2′ can be expressed as:

[0074]

[0075] In order to increase the difference between the reconstructed background samples and the target prior and ensure that the reconstructed samples can truly represent the background, the reconstruction loss L is designed. r ' and DIA loss L a Here, L r The purpose of 'is to minimize the error between the input data b and the reconstructed data b':

[0076]

[0077] In order to increase the difference between the reconstructed background samples and the target prior, we use N to represent the number of reconstructed background samples and calculate the cosine similarity between each reconstructed background spectrum bi′ and the prior target t. The higher the value, the greater the similarity with the prior target. In order to amplify the difference between the target and the reconstructed background and reduce the similarity, the average similarity is used as the DIA loss:

[0078]

[0079] Finally, through adversarial training, the generator G2′ is able to produce background samples b′ that are purer and significantly different from the target samples.

[0080] 5. After the adversarial sample generation stage, the generated target t′ and background sample b′ form training samples, which form positive and negative sample pairs with the target prior t respectively. The generated target and background samples are input into the designed multi-scale convolutional twin network for multi-scale feature extraction and fusion. The proposed multi-scale convolutional twin network is shown in Figure 4 As shown in the figure, the network mainly consists of a multi-scale convolution module (MCM), a convolutional feature extraction module (CDN) and a multi-scale dilated convolution downsampling module (MDCDS).

[0081] like Figure 4 As shown, the spectral pixels are feature extracted through a multi-scale convolution module and two convolution feature extraction modules. In the three interconnected modules, each output is seamlessly transformed as the input of the next module. At the same time, the original spectral pixels are represented by the spectral features of the scale-void convolution downsampling module. The two results are then fused so that the model can effectively capture the information in the spectral data. Finally, the output is mapped to a range between 0 and 1 for subsequent detection.

[0082] During the training process, the two inputs of the Siamese network are the prior target spectrum and the generated target / background samples. If the input of the upper branch is fixed to the target prior and the input of the lower branch is the target, the label of the training sample is set to 1, otherwise it is set to 0. In addition, we keep the same number of positive and negative samples. In the proposed network, there are two loss functions. The first is the contrastive loss function L c , where for the input sample pair (z0,z i ), the extracted network features are expressed as f(z0), f(z i ). We use the idea of ​​increasing the distance between negative samples and reducing the distance between positive samples, and use the following loss function to guide model training. The formula is as follows:

[0083]

[0084] Among them, y represents the label of the sample pair, f represents the network output value, and m is a hyperparameter, which represents the distance degree of the negative sample pair.

[0085] 6. In the training process of the multi-scale convolutional twin network, an AW-BCE loss method is proposed. The nonlinear function q(x) is used to suppress the background and highlight the target pixels. Since the similarity of the positive sample pairs is greater than the similarity of the negative sample pairs, q(x) can maintain a higher value for the target pixels and a lower value for the background pixels, which can effectively suppress the background. The exponential suppression function is defined as:

[0086] q(x)=1-e -λx

[0087] AW-BCE loss function L s is defined as follows:

[0088]

[0089] Where N is the number of samples, k i is the weight of the i-th sample, d i is the distance of the i-th sample pair, l i is the label of the i-th sample. k i The calculation method is as follows:

[0090]

[0091] Among them, λ is a hyperparameter set to 10, k i The value of is in the range of [0,1], indicating the weight of the corresponding sample. Finally, BCE(d i ,l i ) represents the binary cross entropy loss function, which sums the weighted losses of all samples and divides them by the number of samples to get the final loss value. The total loss function consists of two components: contrast loss L c and AW-BCE loss L s .

[0092] L=wL c +(1-w)L s

[0093] Where w is a weighting parameter to balance the impact of each component on the overall calculation.

[0094] 7. Use the trained target detection framework to identify the input sample and output the target detection result map. Figure 5 The AGMS hyperspectral target detection network described in the present invention uses the method of the present invention and other existing methods ACE, CEM, HCEM, CSCR, DSC, TSCNTD, LCNN-CD, and HTD-IRN to compare the result graph on the hyperspectral natural scene data set. It can be seen that the target area is well detected. At the same time, the area under the curve (AUC) value is calculated, where the larger the area under the curve (AUC) value, the better the comprehensive evaluation of the target detection result. Table 1 gives the index results of different methods in four different test sets:

[0095] Table 1 Comparison of AGMS and various methods on four hyperspectral image datasets (average)

[0096]

[0097] The present invention achieves the best accuracy on all datasets. Figure 6The visualization results of the target prior, generated target samples and generated background samples of the AGMS method on four real hyperspectral image datasets are given. It can be seen that the target samples generated by the method of the present invention retain the spectral curve profile of the target prior to a large extent, and there are certain spectral differences between the target samples; the generated background samples have obvious curve profiles compared with the target samples, indicating that the generated background samples are more discriminative. The method proposed in the present invention can effectively generate fidelity and diverse target and background samples, and has advantages over other methods in target detection and background suppression.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A multi-scale twin hyperspectral target detection method, characterized in that: The following steps are involved: S1: Hyperspectral image initial target and background data preparation; S2: Generate adversarial samples for hyperspectral remote sensing images to obtain training sample data; S3: Input the generated samples into the multi-scale twin object detection network for training; S4: After training is completed, target detection is performed on the test samples to obtain results.

2. The multi-scale twin hyperspectral target detection method according to claim 1, characterized in that: In step S1, initial hyperspectral image target and background data are prepared, wherein the target data uses the target prior existing in the data set, and then the original pixels of the hyperspectral image are clustered using the target prior to obtain the initial background sample.

3. The multi-scale twin hyperspectral target detection method according to claim 1, characterized in that: In step S2, in the stage of generating target samples, the target prior t and the background sample xb are respectively input into the generator G1 and the discriminator D1, and then the generated target sample is input into the decoder De1 again to obtain t′, and then input into the discriminator D2 together with the original target prior for discrimination to achieve adversarial generation. Similarly, in the stage of generating background samples, the initial background sample b and the generated target sample t′ are respectively input into the generator G1′ and the discriminator D1′, and then the generated background sample is input into the decoder De2 again to obtain b′, and then input into the discriminator D2′ together with the initial background sample for adversarial generation. Finally, the generated target t′ and background sample b′ constitute the training sample.

4. The multi-scale twin hyperspectral target detection method according to claim 3, characterized in that: First, the existing target prior t is input into the generator G1, and then t is mapped to the latent space through the encoder to obtain its compressed latent feature tz; Subsequently, tz is input into the decoder De1 to generate a new target sample t′ with similar features. Gaussian noise interference is introduced into the generated target sample t′. These disturbed and diversified samples are used as new target inputs for training. This process is repeated until a sufficient number of target samples are generated. Then, the spectrum with low cosine similarity to the prior target t is processed using principal component analysis (PCA) to become a background sample xb. The dimension of the background sample is reduced to match the dimension of tz. Subsequently, the background sample xb and the potential feature tz are input into the discriminator D1 for target-background adversarial training. In this process, the two generators correspond to the two discriminators, and adversarial training is used to achieve target-background adversarial generation. The discriminator D1 corresponds to the generator G1, and the discriminator D2 corresponds to the generator G2 composed of G1 and the decoder De1. During the training process, the goal of the generator G1 is to extract the latent feature tz of the target spectrum. To ensure that the target spectrum better represents the distribution of real data, the latent feature tz and the background spectrum are input into the discriminator network D1. The discriminator D1 is optimized to maximize the distinction between the target latent feature tz and the background samples. At the same time, the feedback from the discriminator drives the generator G1 to continuously optimize its parameters: Among them, E x～p(x) [f(x)] represents the expected value of function f(x) when variable x obeys the probability density function distribution; The goal of the generator G2 is to generate samples similar to the real target spectrum. At the same time, the goal of the discriminator D2 is to distinguish the target spectrum generated by the generator G2 from the real target spectrum. The adversarial loss formula between G2 and D2 is expressed as follows: Same as above, E. t～p(t) [f(t)] represents the expected value of function f(t) when variable t obeys the probability density function distribution; In addition, the difference between the generated target and the target prior is calculated and the mean squared error loss is used to quantify this difference. By minimizing this loss function, the generator G2 adjusts its parameters to approximate its output so that it is closer to the input target sample: Finally, through adversarial training, the generator G2 is able to generate target samples t′ that are significantly different from background samples.

5. The multi-scale twin hyperspectral target detection method according to claim 3, characterized in that: First, the initial background sample b obtained by clustering the original hyperspectral image is input into the generator G1′ to obtain the potential feature representation bz of the background sample, and then bz is input into the discriminator D1′; by performing PCA on the generated target sample, an input t″ of D1′ is obtained, and then the potential representation bz of the background sample is input into the decoder De2 for decoding to obtain the reconstructed background data b′, and finally, the reconstructed background data b′ and the initial background sample b are input into the discriminator D2′ for discrimination; during the training process, the generator G1′ tries to generate background potential feature representation bz that can deceive the discriminator D1′, while the discriminator D1′ strives to distinguish between the real target sample and the generated background feature representation: Same as above, E. t”～p(t”) [f(t”)] represents the expected value of function f(t”) when variable t” obeys the probability density function distribution; By learning to generate real background samples, the generator G2′ aims to generate background spectrum samples that can deceive the discriminator network D2′; at the same time, the discriminator network D2′ can effectively distinguish the background spectrum samples generated by the generator G2′ from the real background spectrum samples; therefore, the adversarial loss between G2′ and D2′ is expressed as: Same as above, E. b～p(b) [f(b)] represents the expected value of function f(b) when variable b obeys the probability density function distribution; In order to ensure that the reconstructed samples can truly represent the background, we design the reconstruction loss L r ' and DIA loss L a , here, L r The purpose of 'is to minimize the error between the input data b and the reconstructed data b': On the other hand, DIA loses L a The design is to encourage the model to learn more different features to distinguish between the target and the background; N represents the number of samples of the reconstructed background, and the cosine similarity between each reconstructed background spectrum bi′ and the prior target t is calculated; the higher the value, the greater the similarity with the prior target. In order to amplify the difference between the target and the reconstructed background and reduce the similarity, the average similarity is used as the DIA loss: Finally, through adversarial training, the generator G2′ is able to produce background samples b′ that are purer and significantly different from the target samples.

6. The multi-scale twin hyperspectral target detection method according to claim 1, characterized in that: In step S3, after the adversarial sample generation stage, the generated target t′ and background sample b′ constitute training samples, and respectively constitute positive and negative sample pairs with the target prior t; the generated target and background samples are input into the designed multi-scale convolutional twin network for multi-scale feature extraction and fusion; the proposed multi-scale convolutional twin network is mainly composed of a multi-scale convolution module (MCM), a convolutional feature extraction module (CDN) and a multi-scale dilated convolution downsampling module (MDCDS); spectral pixels are feature extracted by a multi-scale convolution module and two convolutional feature extraction modules, and in the three interconnected modules, each output is seamlessly converted as the input of the next module; at the same time, the original spectral pixels are represented by the spectral features of the dilated convolution downsampling module, and effective semantic information and spatial contextual relationships are obtained at the same time, and then the two results are fused so that the model can effectively capture the information in the spectral data; Finally, the output is mapped to a range between 0 and 1 for subsequent detection. During the training process, the two inputs of the twin network are the prior target spectrum and the generated target / background samples. If the input of the upper branch is fixed to the target prior and the input of the lower branch is the target, the label of the training sample is set to 1, otherwise it is set to 0. In addition, we keep the same number of positive and negative samples. In the proposed network, there are two loss functions. The first is the contrast loss function L c , where for the input sample pair (z0,z i ), the extracted network features are expressed as f(z0), f(z i ); We use the idea of ​​increasing the distance between negative samples and reducing the distance between positive samples, and use the following loss function to guide model training. The formula is as follows: Among them, y represents the label of the sample pair, f represents the network output value, and m is a hyperparameter, which represents the distance degree of the negative sample pair.

7. The multi-scale twin hyperspectral target detection method according to claim 6, characterized in that: In the training process of the multi-scale convolutional twin network, an AW-BCE loss method is proposed; the nonlinear function q(x) is used to suppress the background. Since the similarity of positive sample pairs is greater than that of negative sample pairs, q(x) can maintain a higher value of the target pixel and effectively suppress the background. The exponential suppression function is defined as: q(x)=1-e -λx AW-BCE loss function L s is defined as follows: Where N is the number of samples, k i is the weight of the i-th sample, d i is the distance of the i-th sample pair, l i is the label of the i-th sample, k i The calculation method is as follows: Among them, λ is a hyperparameter set to 10, k i The value of is in the range of [0,1], indicating the weight of the corresponding sample. Finally, BCE(d i ,l i ) represents the binary cross entropy loss function, which sums the weighted losses of all samples and divides them by the number of samples to get the final loss value. The total loss function consists of two components: contrast loss L c and AW-BCE loss L s , L=wL c +(1-w)L s Where w is a weighting parameter to balance the impact of each component on the overall calculation.

8. The multi-scale twin hyperspectral target detection method according to claim 7, characterized in that: The loss function is calculated, and the model parameters of the hyperspectral image target detection framework are optimized according to the loss function and the back-propagation process. After the training is completed, a trained hyperspectral target detection framework is obtained; the input sample is detected through the trained hyperspectral target detection framework, and a hyperspectral target detection result map is output.

9. A multi-scale twin hyperspectral target detection system, characterized in that: The system adopts the multi-scale twin hyperspectral target detection method as described in any one of claims 1 to 8.

Images