Hyperspectral target detection method based on butterfly Transform network
By using a butterfly Transformer network in hyperspectral object detection, combining CEM and SAM detectors, spectral cell coding mechanism and Transformer blocks, the problems of degradation in detection performance and difficulty in background suppression in complex scenarios are solved, and efficient spectral information is achieved Detection and background suppression improve detection performance.
Patent Information
- Application Number
- CN202411058125.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-15
AI Technical Summary
The existing hyperspectral object detection methods are difficult to meet actual needs in complex scenarios, especially when the source and target domain imaging sensors of transfer learning are different, the detection performance is degraded and the background is difficult to effectively suppress.
Using a hyperspectral object detection method based on the butterfly Transformer network, the training data is constructed through the joint mask of CEM and SAM detectors, the spectral cell encoding mechanism and Transformer block are designed, the double butterfly Transformer network is built, and the spectrum is extracted using the self-attention mechanism. Features and design attention-based separation and aggregation triple loss functions to improve the separability of targets and backgrounds, and achieve background suppression through exponential functions and power functions.
It effectively reduces the complexity of the model and computing resource requirements, improves the separability of the target and background, realizes efficient spectral information detection and background suppression, and improves detection performance.
Smart Images

Figure CN118968302A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of hyperspectral image target detection, and in particular to a hyperspectral target detection method based on a butterfly Transformer network. Background Art
[0002] Hyperspectral imaging is a powerful technology that can simultaneously capture the morphological and spectral information of objects in a specific area. The hundreds of narrow and continuous spectral bands captured constitute hyperspectral images (HSIs). These spectral bands provide rich information for the spectral characterization of different objects and reflect the differences in the internal structure and composition of objects. This feature enables HSIs to be applied in related fields such as unmixing, detection and classification. Among them, hyperspectral target detection (HTD) is an advantageous application of the rich spectral information of hyperspectral images, and has high application value in military reconnaissance, mineral exploration, environmental monitoring and medical diagnosis.
[0003] Hyperspectral target detection can be regarded as an image binary classification problem that only contains target and background classes. The key is to find an effective detector that can both highlight the target and suppress the background based on spectral differences. Early hyperspectral target detection methods were mainly based on hypothesis models based on statistical distribution and linear mixture models. Adaptive coherence / cosine estimator (ACE) and matched filter (MF) define the target detection task as a binary hypothesis test problem based on Gaussian distribution. Among them, ACE is considered to be one of the most effective detection methods, and the detection results are obtained through generalized likelihood ratio test. Constrained energy minimization (CEM) is the main representative of linear mixture model-based methods. It achieves the effect of highlighting the target and suppressing the background by minimizing the energy of the unknown signal under the constraint of the target signal. However, ideally, the spectral characteristics of the target of interest should be defined by a single target spectrum or target subspace. For the high complexity of the surface cover in real scenes, this often cannot meet the actual needs. Subsequently, the rapid development of machine learning has provided a new solution for hyperspectral target detection. The detector based on sparse theory, joint sparse and collaborative representation (CSCR), has achieved good results by constructing a local background dictionary through double concentric windows. However, these methods usually require extensive research experience to extract and select features, and the limited complexity of the mapping function also limits their nonlinear feature extraction capabilities. In recent years, methods based on deep learning have emerged in the field of hyperspectral image target detection. In the initial research, a supervised hyperspectral target detection method (CNND) based on transfer learning appeared to address the problem of insufficient training samples. However, when the hyperspectral imaging sensors in the source and target domains of the transfer learning are different, the detection performance of the CNND detector will drop significantly. For unsupervised learning, a hyperspectral target detection method based on self-supervised spectral level contrast learning is proposed. This method mines the supervised information of the image itself through a spectral data enhancement method based on odd and even bands, and then uses spectral level contrast learning to complete the learning of spectral similarities and differences, thereby achieving the detection task. Summary of the invention
[0004] In view of the problems existing in the prior art, the present invention discloses a hyperspectral target detection method based on a butterfly Transformer network, which specifically includes the following steps:
[0005] The target and background training data are constructed by joint masking through CEM detector and SAM detector;
[0006] Construct a spectral pixel encoding mechanism and Transformer block to build a double butterfly Transformer network and obtain the global information of the pixel spectrum through the self-attention mechanism;
[0007] A double butterfly Transformer network is constructed based on the butterfly network as the backbone architecture, based on the spectral pixel encoding mechanism and the Transformer block. This network is used to extract the spectral long-distance dependency and self-similarity features of spectral pixels.
[0008] Design an attention-based separation and aggregation triple loss function to improve the network's object and background separability and train a double butterfly Transformer network;
[0009] The trained double butterfly Transformer network is used to extract the representation vector of the difference between the spectrum of each pixel in the hyperspectral image to be tested and the prior target spectrum. The representation vector is input into the classification head of the double butterfly Transformer network to obtain the similarity between the two, thereby obtaining the detection result using only the spectral information.
[0010] An exponential function is used to increase the distance between the target and background pixel values in the spectral detection results, and then a power function is used to map the target pixel value close to the value in the original spectral detection result, thereby completing background suppression.
[0011] Furthermore, when the target and background training data are constructed by joint masking through the CEM detector and the SAM detector: let the hyperspectral image including N pixels be And it consists of L spectral bands, and all spectral pixels in X are arranged in the form of a matrix as X = [x1,...,x N ], let the feature vector of the prior target spectrum be d, and the response of the CEM detector be expressed as:
[0012]
[0013] Where w represents the FIR linear filter, R = (1 / N) XX T represents the correlation matrix;
[0014] For spectral pixel x i , the response of the SAM detector is expressed as:
[0015]
[0016] The results obtained based on the CEM detector and the SAM detector are regarded as two pixel category probability maps y CEM and SAM , where each element represents the possibility of the corresponding pixel at that position being the target. When obtaining the spectral pixel labeled as the background target, different thresholds are used to determine y CEM and SAM The corresponding target map And background image The target sample and the background sample are constructed through the intersection between the target image and the background image, thus forming the training data.
[0017] Furthermore, when constructing the spectral pixel encoding mechanism and the Transformer block: a convolutional layer with a step size of s is designed to divide the overlapping light blocks. The step size s controls the size of the non-overlapping part between adjacent spectral blocks. An additional learnable class embedding is added to replace the average pooling layer structure to obtain an embedding that represents the characteristics of the entire spectral sequence. The obtained overlapping spectral sequence, class embedding, and position embedding are used together as the input of the Transformer block.
[0018] In each self-attention module of the multi-head self-attention mechanism, the spectral sequence is input Through the projection of the learnable feature matrix, three new feature vectors are obtained, namely the query vector Q, the key vector K and the value vector V. The process is described as:
[0019] Q=W Q z,K=W K z,V=W V z
[0020] Where W Q , W K and W V The sub-tables represent the query feature matrix, key feature matrix, and value feature matrix. The dot product of the query vector Q and all key vectors K is calculated and the dot product result is passed through After normalization, the weight of the corresponding value vector V is obtained through the softmax activation function. The weight is dot-producted with the value vector V to obtain the self-attention embedded in the spectral sequence. The process is expressed as:
[0021]
[0022] The multi-head self-attention mechanism is represented as the output of the integrated projection of N self-attention heads:
[0023] MultiHead(Q,K,V)=Concat(head1,...,head N )W o
[0024] head i =Attention(Q,K,V),(i=1,2,...,N)
[0025] The Transformer block projects the embedded spectral sequence obtained by the multi-head self-attention mechanism into a high-dimensional space with the help of the feed-forward layer.
[0026] Furthermore, when constructing a double butterfly Transformer network: the input spectral sequence Z is transmitted to the lower branch channel consisting of a Transformer block, and the process of passing through the Transformer block is recorded as H(·), then the first combination of Z and H(Z) is expressed as:
[0027] P i-1 (Z) = Z + A i-1 H(Z)
[0028] R i-1 (Z) = B i-1 Z+H(Z)
[0029] Among them, A i-1 and B i-1 are two combined coefficient vectors calculated adaptively based on the characteristic response. i-1 (Z) is sent to the upper branch channel and passes through the Transformer block G(·) with the same structure as the Transformer block H(·), then P i (Z) and R i The second combination of (Z) is expressed as:
[0030] P i (Z) = G(P i-1 (Z))+A i R i-1 (Z)
[0031] R i (Z) = B i G(P i-1 (Z))+R i-1 (Z)
[0032] Among them, A i and B i is the corresponding G(P i-1 (Z)) and R i-1 (X) is the combined coefficient vector of P i (Z) and R i (Z) is transmitted to the classification head of the double butterfly Transformer network for spectral feature fusion to obtain the final similarity score. The process is expressed as:
[0033]
[0034] Furthermore, when designing the attention-based separation and aggregation triple loss function: using cosine similarity to measure the target attention F p , Background Attention F n and anchor point F a The distance between the three is expressed as follows:
[0035]
[0036] where F a is the hyperspectral image to be tested X, F p is the product of X and the attention map F output by the butterfly Transformer network, and F n is the product of X and (1-F), and the BSL loss function is expressed as:
[0037]
[0038] BSL max =max(BSL - ,BSL * ), M is the batch size.
[0039] Furthermore, when training the double butterfly Transformer network:
[0040] The prior target spectrum is paired with the training sample to generate the corresponding spectral sequence, and the spectral sequence is transmitted to the double butterfly Transformer network to obtain the embedding corresponding to the class embedding of the two spectral sequences. The two embeddings are connected along the dimension and fed into the classification head of the network. The last layer of the classification head is the Sigmoid activation layer. The output value of the Sigmoid activation function is in the range of 0-1. The final output of the double butterfly Transformer network is in the form of scores or labels. The binary cross entropy BCE function is used as the loss function, which is specifically described as:
[0041]
[0042] In the formula, f i is the output of the Sigmoid layer of the network classification head, and y i Indicates label 1 or 0. The total loss of the double butterfly Transformer network is the sum of the BCE loss and the BSL loss with a weighted coefficient of β, expressed as Loss = Loss BCE +βLoss BSL .
[0043] The trained double butterfly Transformer network is used to extract the representation vector of the difference between the spectrum of each pixel in the hyperspectral image to be tested and the prior target spectrum, and sent to the classification head of the network to obtain the similarity between the two, thereby obtaining the detection result using only the spectral information.
[0044] The exponential function is used to further increase the distance between the target and background pixel values in the spectral detection results, and then the power function is used to map the value of the target pixel close to the value in the original spectral detection result, thereby completing background suppression.
[0045] The spatial information of the hyperspectral image is used as supplementary information, and a two-dimensional spatial filter is used to filter the detection results obtained using only the spectral information to obtain the target detection results of the joint spatial-spectral information.
[0046] Due to the adoption of the above technical scheme, the present invention provides a hyperspectral target detection method based on a butterfly Transformer network. The method effectively solves the problem of high demand for hardware equipment and computing resources caused by the multi-head self-attention mechanism of Transformer, and converts the problem of difficult background suppression into the problem of improving the separability of the target and background. First, a training sample construction method of dual detector joint mask is designed, and the target sample and the background sample are obtained by crossing the target image (background image) determined by the CEM and SAM dual detectors under different threshold conditions, thereby constructing a training data set. Secondly, a butterfly network structure is designed, and a dual butterfly Transformer network is constructed through spectral pixel encoding and Transformer blocks. The network realizes mapping into a network composed of multiple Transformer blocks through fewer Transformer blocks, effectively reducing the complexity of the model and the requirements for computing resources and hardware equipment. Next, a triple loss function of separation and aggregation based on attention is proposed, which converts effective background suppression into improving the separability of the network's target and background. Then, the trained backbone network is used to extract the representation vector of the difference between the pixel spectrum in the hyperspectral image to be tested and the prior target spectrum, and sent to the classification head of the network to obtain the similarity between the two, so as to obtain the detection result using only the spectral information. Finally, the power function method is used to simply but effectively suppress the background of the detection result using only the spectral information, and the spatial information of the hyperspectral image is used as supplementary information through a two-dimensional spatial filter to obtain the final spectral-spatial detection result. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0048] Figure 1 is a flow chart of the method disclosed in the present invention;
[0049] Figure 2 Schematic diagram of spectral pixel encoding and Transformer block
[0050] Figure 3a-Figure 3bGroundtruth annotation and detection result images for the San Diego dataset;
[0051] Table 1 shows the area under the ROC curve of the proposed target detection method and the comparison method on the San Diego dataset, corresponding to the curve values AUC(PD,PF), AUC(PD,τ), AUC(PF,τ), AUC BS and AUC OD . DETAILED DESCRIPTION
[0052] In order to make the technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention:
[0053] like Figure 1 A hyperspectral target detection method based on a butterfly Transformer network is shown, which specifically includes the following steps:
[0054] S1: Target and background training data are constructed by joint masking of dual classical detectors detected by CEM detector and SAM;
[0055] The specific method used in S1 is as follows:
[0056] S11: Assume a hyperspectral image with N pixels And it consists of L spectral bands. All spectral pixels in X can be arranged in the form of a matrix as X = [x1,...,x N ], and let the eigenvector of the prior target spectrum be d. Therefore, the response of the CEM detector can be expressed as:
[0057]
[0058] Where w represents a FIR linear filter, R = (1 / N) XX T It is represented as a correlation matrix.
[0059] S12: For spectral pixel x i , the response of the SAM detector can be expressed as:
[0060]
[0061] S13: The results obtained by the two detectors can be regarded as two pixel category probability maps y CEM and SAM , where each element represents the possibility that the corresponding pixel at that position is the target. When obtaining spectral pixels labeled as targets and backgrounds, different thresholds are used to determine y CEM and SAM The corresponding target map And background image
[0062] S14: Passed and ( and )The intersection of the two constructs the target sample (background sample), thus constituting the training data.
[0063] S2: Construct spectral pixel encoding mechanism and Transformer block;
[0064] The specific method used in S2 is as follows:
[0065] S21: Figure 2 As shown in the figure, a convolutional layer with a step size of s is designed to realize the division of overlapping light blocks, thereby providing a high-quality spectral sequence to improve the performance of the Transformer block. The step size s controls the size of the non-overlapping part between adjacent spectral blocks. In addition to the position embedding that retains the position information, an additional learnable class embedding is added to replace the average pooling layer and other structures to obtain an embedding that can be used to represent the characteristics of the entire spectral sequence. Finally, the obtained overlapping spectral sequence is used as the input of the Transformer block together with the class embedding and the position embedding.
[0066] S22: The multi-head self-attention mechanism is the core of the Transformer block, which can effectively mine the intrinsic correlation between features. Figure 2 As shown, in each self-attention module of the multi-head self-attention mechanism, the input spectral sequence Through the projection of the learnable feature matrix, three new feature vectors are obtained, namely the query vector Q, the key vector K and the value vector V. The process can be described as:
[0067] Q=W Q z,K=W K z,V=W V z
[0068] Where W Q , W K and W V The sub-tables represent the query feature matrix, key feature matrix, and value feature matrix. Then, the dot product of the query vector Q and all key vectors K is calculated, and the dot product result is passed through Normalize, then use the softmax activation function to get the weight of the corresponding value vector V, and finally dot product the weight with the value vector V to get the self-attention embedded in the spectral sequence. The above self-attention process can be expressed as:
[0069]
[0070] The multi-head self-attention mechanism can be seen as N self-attention heads integrated and projected to achieve their output, which can be expressed as:
[0071] MultiHead(Q,K,V)=Concat(head1,...,head N )W o ,
[0072] head i =Attention(Q,K,V),(i=1,2,...,N)
[0073] S23: The Transformer block projects the embedded spectral sequence obtained by the multi-head self-attention mechanism into a high-dimensional space with the help of the feed-forward layer.
[0074] S3: A double butterfly Transformer network is constructed through the butterfly network framework for feature extraction of spectral pixels;
[0075] The specific method used in S3 is as follows: the input spectrum sequence Z is sent to the lower branch channel composed of a Transformer block. The process through the Transformer block is denoted as H(·), then the first combination of Z and H(Z) can be expressed as:
[0076] P i-1 (Z) = Z + A i-1 H(Z)
[0077] R i-1 (Z) = B i-1 Z+H(Z)
[0078] Among them, A i-1 and B i-1 are two combination coefficient vectors, which are adaptively calculated according to the characteristic response. Next, P i-1 (Z) is sent to the upper branch channel and passes through the Transformer block G(·) which has the same structure as the Transformer block H(·). Then, P i (Z) and R i The second combination of (Z) can be expressed as:
[0079] P i (Z) = G(P i-1 (Z))+A i R i-1 (Z)
[0080] R i (Z) = B i G(P i-1 (Z))+R i-1 (Z)
[0081] Among them, A i and B i It also corresponds to G(P i-1 (Z)) and R i-1 (X) is a combination coefficient vector. Then, P i (Z) and R i (Z) is sent to the classification head of the network for spectral feature fusion to obtain the final similarity score. The process can be expressed as:
[0082] F=ClassHead(A i (H(Z)+B i-1 Z)+G(Z+A i-1 H(Z)),
[0083] H(Z)+B i-1 Z+B i (G(Z+A i-1 H(Z))))
[0084] S4: Design an attention-based separation and aggregation (BSL) triple loss function, which aims to make the background attention F n In feature space, the distance from the anchor point F a farther, and makes the target focus F p In feature space, the distance from the anchor point F a Closer, thereby improving target attention F p and background attention F n Specifically, the target attention F p , Background Attention F n and anchor point F a The distance between the three is measured using cosine similarity, which can be formalized as:
[0085]
[0086] where F a is the hyperspectral image to be tested X, F p is the product of X and the attention map F output by the butterfly Transformer network, and F n is the product of X and (1-F). Therefore, the BSL loss function can be expressed as:
[0087]
[0088] BSL max =max(BSL - ,BSL *), and M is the batch size. The BSL loss function can make the feature space distance between the positive sample and the anchor point smaller, and the distance between the negative sample and the anchor point larger, that is, BSL + and BLS * As big as possible, BSL - as small as possible.
[0089] S5: During the training phase, the prior target spectra are paired with the training samples to generate corresponding spectral sequences. These spectral sequences are fed into the double butterfly Transformer network, and finally two embeddings corresponding to the class embeddings of the two spectral sequences are obtained. Then, the two embeddings are concatenated along the dimension and fed into the classification head of the network. The last layer of the classification head is the Sigmoid activation layer. The output value of the Sigmoid activation function is in the range of 0-1, so the final output of the double butterfly Transformer network is in the form of scores (or labels). Therefore, the binary cross entropy (BCE) function is used as the loss function, which is specifically described as:
[0090]
[0091] In the formula, f i is the output of the Sigmoid layer of the network classification head, and y i Indicates a label of 1 or 0.
[0092] Therefore, the total loss of the double butterfly Transformer network is the sum of the BCE loss and the BSL loss with a weighted coefficient of β, expressed as Loss = Loss BCE +βLoss BSL .
[0093] S6: Use the trained double butterfly Transformer network to extract the representation vector of the difference between the spectrum of each pixel in the hyperspectral image to be tested and the prior target spectrum, and send it to the classification head of the network to obtain the similarity between the two, so as to obtain the detection result using only the spectral information.
[0094] S7: Use an exponential function to further increase the distance between the target and background pixel values in the spectral detection result, and then use a power function to map the value of the target pixel close to the value in the original spectral detection result, thereby achieving simple but effective background suppression.
[0095] S8: Using the spatial information of the hyperspectral image as supplementary information, a two-dimensional spatial filter is used to filter the detection result obtained using only the spectral information to obtain the target detection result of the joint spatial-spectral information.
[0096] Next, a hyperspectral target detection method based on a butterfly Transformer network provided by the present invention will be evaluated and assessed on a real hyperspectral image dataset.
[0097] The San Diego dataset was obtained by taking photos of the San Diego Airport area in California, USA, by an airborne visible light\infrared imaging spectrometer. The original image consists of 400×400 pixels, a total of 224 bands, a spectral resolution of 10nm, a spatial resolution of 3.5m, and a wavelength range of 370-2510nm. After removing the low signal-to-noise ratio and water absorption bands, a total of 189 bands remain for detection. The experiment selected an image of size 120×120, and the aircraft in the scene was regarded as the target of interest for detection, which contains 58 target pixels.
[0098] The present invention provides a hyperspectral target detection method based on a butterfly Transformer network. The detection result on a real hyperspectral image is as follows: Figure 3b The receiver operating characteristic curve (ROC) and the area under the curve (AUC) were used to quantitatively analyze the detection results. As shown in Table 1, (bold indicates the best, underline indicates the suboptimal)
[0099]
[0100] Table 1 Accuracy comparison of different methods on the San Diego dataset
[0101] For the San Diego dataset, the area under the curve AUC(PD,PF), AUC(PD,τ), AUC(PF,τ), AUC BS and AUC OD The values are 0.9962, 0.7603, 0.0087, 0.9875 and 1.7478 respectively.
[0102] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A hyperspectral target detection method based on butterfly Transformer network, characterized in that include: The target and background training data are constructed by joint masking through CEM detector and SAM detector; Construct a spectral pixel encoding mechanism and Transformer block to build a double butterfly Transformer network and obtain the global information of the pixel spectrum through the self-attention mechanism; A double butterfly Transformer network is constructed based on the butterfly network as the backbone architecture, based on the spectral pixel encoding mechanism and the Transformer block. This network is used to extract the spectral long-distance dependency and self-similarity features of spectral pixels. Design an attention-based separation and aggregation triple loss function to improve the network's object and background separability and train a double butterfly Transformer network; The trained double butterfly Transformer network is used to extract the representation vector of the difference between the spectrum of each pixel in the hyperspectral image to be tested and the prior target spectrum. The representation vector is input into the classification head of the double butterfly Transformer network to obtain the similarity between the two, thereby obtaining the detection result using only the spectral information. An exponential function is used to increase the distance between the target and background pixel values in the spectral detection results, and then a power function is used to map the target pixel value close to the value in the original spectral detection result, thereby completing background suppression.
2. The hyperspectral target detection method based on butterfly Transformer network according to claim 1 is characterized in that: When constructing target and background training data through joint masking by CEM detector and SAM detector: Suppose the hyperspectral image including N pixels is And it consists of L spectral bands, and all spectral pixels in X are arranged in the form of a matrix as X = [x1,...,x N ], let the feature vector of the prior target spectrum be d, and the response of the CEM detector be expressed as: Where w represents the FIR linear filter, R = (1N) XX T represents the correlation matrix; For spectral pixel x i , the response of the SAM detector is expressed as: The results obtained based on the CEM detector and the SAM detector are regarded as two pixel category probability maps y CEM and SAM , where each element represents the possibility of the corresponding pixel at that position being the target. When obtaining spectral pixels labeled as target and background, different thresholds are used to determine y CEM and SAM The corresponding target map And background image The target sample and the background sample are constructed through the intersection between the target image and the background image, thus forming the training data.
3. The hyperspectral target detection method based on butterfly Transformer network according to claim 1 is characterized in that: When constructing the spectral pixel encoding mechanism and Transformer block: design a convolutional layer with a step size of s to divide the overlapping light blocks. The step size s controls the size of the non-overlapping part between adjacent spectral blocks. Add additional learnable class embeddings to replace the average pooling layer structure to obtain an embedding that represents the characteristics of the entire spectral sequence. The obtained overlapping spectral sequence, class embedding, and position embedding are used together as the input of the Transformer block. In each self-attention module of the multi-head self-attention mechanism, the input spectral sequence Through the projection of the learnable feature matrix, three new feature vectors are obtained, namely the query vector Q, the key vector K and the value vector V. The process is described as: Q=W Q z,K=W K z,V=W V z Where W Q , W K and W V The sub-tables represent the query feature matrix, key feature matrix, and value feature matrix. The dot product of the query vector Q and all key vectors K is calculated and the dot product result is passed through After normalization, the weight of the corresponding value vector V is obtained through the softmax activation function. The weight is dot-producted with the value vector V to obtain the self-attention embedded in the spectral sequence. The process is expressed as: The multi-head self-attention mechanism represents the output of the integrated projection of N self-attention heads: MultiHead(Q,K,V)=Concat(head1,...,head N )W o , head i =Attention(Q,K,V),(i=1,2,...,N) The Transformer block projects the embedded spectral sequence obtained by the multi-head self-attention mechanism into a high-dimensional space with the help of the feed-forward layer.
4. The hyperspectral target detection method based on butterfly Transformer network according to claim 1, characterized in that: When constructing a double butterfly Transformer network: the input spectral sequence Z is transmitted to the lower branch channel consisting of a Transformer block, and the process through the Transformer block is recorded as H(·). Then the first combination of Z and H(Z) is expressed as: P i-1 (Z)=Z+A i-1 H(Z) R i-1 (Z)=B i-1 Z+H(Z) Among them, A i-1 and B i-1 are two combined coefficient vectors calculated adaptively based on the characteristic response. i-1 (Z) is sent to the upper branch channel and passes through the Transformer block G(·) with the same structure as the Transformer block H(·), then P i (Z) and R i The second combination of (Z) is expressed as: P i (Z)=G(P i-1 (Z))+A i R i-1 (Z) R i (Z)=B i G(P i-1 (Z))+R i-1 (Z) Among them, A i and B i is the corresponding G(Pi-1(Z)) and R i-1 (X) is the combined coefficient vector of P i (Z) and R i (Z) is transmitted to the classification head of the double butterfly Transformer network for spectral feature fusion to obtain the final similarity score. The process is expressed as:
5. The hyperspectral target detection method based on butterfly Transformer network according to claim 1, characterized in that: When designing the attention-based separation and aggregation triple loss function: using cosine similarity to measure the target attention F p , Background Attention F n and anchor point F a The distance between the three is expressed as follows: Among them, F a is the hyperspectral image to be tested X, F p is the product of X and the attention map F output by the butterfly Transformer network, and F n is the product of X and (1-F), and the BSL loss function is expressed as: BSL max =max(BSL - ,BSL * ), M is the batch size.
6. The hyperspectral target detection method based on butterfly Transformer network according to claim 1, characterized in that: When training a double butterfly Transformer network: The prior target spectrum is paired with the training sample to generate the corresponding spectral sequence, and the spectral sequence is transmitted to the double butterfly Transformer network to obtain the embedding corresponding to the class embedding of the two spectral sequences. The two embeddings are connected along the dimension and fed into the classification head of the network. The last layer of the classification head is the Sigmoid activation layer. The output value of the Sigmoid activation function is in the range of 0-1. The final output of the double butterfly Transformer network is in the form of scores or labels. The binary cross entropy BCE function is used as the loss function, which is specifically described as: In the formula, f i is the output of the Sigmoid layer of the network classification head, and y i Indicates label 1 or 0. The total loss of the double butterfly Transformer network is the sum of the BCE loss and the BSL loss with a weighted coefficient of β, expressed as Loss = Loss BCE +βLoss BSL .
