A hyperspectral band selection method and device based on Transformer network

Abstract

The present invention relates to the field of hyperspectral remote sensing image processing, and discloses a hyperspectral band selection method and device based on a Transformer network. The method comprises: (1) calculating band position codes; (2) constructing and training a Transformer reconstruction network; (3) measuring the representativeness of candidate bands; (4) measuring the redundancy of candidate bands; (5) designing a band scoring function that takes into account both band representativeness and redundancy to evaluate candidate bands; and (6) selecting the bands with the highest scores among all unselected bands one by one until the number of selected bands reaches a preset value. Starting from the characteristics of hyperspectral images, the present invention combines a Transformer network and a convolutional autoencoder network to explore the potential nonlinear relationship between spectral bands, fully utilize the spatial information of hyperspectral images, and proposes a band selection method that takes into account both band representativeness and redundancy, which can effectively improve the effect of hyperspectral image pixel classification.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a hyperspectral band selection method and device based on a Transformer network. Background Art

[0002] Hyperspectral images consist of hundreds of continuous bands and contain rich spectral information. However, in practical applications, high-dimensional spectral data also brings many challenges, such as information redundancy, heavy computational and storage burdens, and the "Hughes phenomenon." Therefore, it is necessary to develop dimensionality reduction techniques for hyperspectral images. Dimensionality reduction methods for hyperspectral images can generally be divided into two categories: feature extraction and band selection. The former results in the loss of physical features of the original hyperspectral image, while the latter can preserve the physical features of the original hyperspectral image. Therefore, band selection methods have attracted widespread attention from scholars.

[0003] Existing band selection methods at home and abroad can be roughly divided into the following four categories: (1) band selection methods based on sorting; (2) band selection methods based on point-by-point search; (3) band selection methods based on group search; (4) band selection methods based on machine learning algorithms. These methods select the desired band subset from the original hyperspectral image based on specific evaluation criteria. However, the existing band selection methods mainly face three problems:

[0004] (1) Most existing band selection methods simply consider the linear relationship between bands or the nonlinear relationship based on a predefined kernel function, but lack consideration of the potential nonlinear relationship between bands.

[0005] (2) The existing band selection method based on autoencoders cannot take into account the representativeness and redundancy of the bands while utilizing the spatial and spectral information of hyperspectral images, resulting in the inability to achieve breakthroughs in the implementation of downstream tasks.

[0006] (3) The existing band selection method based on autoencoders focuses on the band attention of a certain representation subspace by combining a single attention head, but ignores the information of other representation subspaces, resulting in the learned band attention being unable to accurately reflect the representativeness of the band. Summary of the Invention

[0007] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a hyperspectral band selection method and device based on the Transformer network. With the help of the Transformer network, which has demonstrated superior performance in the field of natural language processing, the potential nonlinear relationship between spectral bands is revealed, the spectral and spatial information of hyperspectral images is fully utilized, and a band selection strategy that can take into account both band representativeness and redundancy is studied. The extraction effect of band subsets with high representativeness and low redundancy is improved, and the problem of low pixel classification accuracy caused by the inability to select the most valuable band subset is solved.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] The present invention provides a hyperspectral band selection method based on a Transformer network, comprising the following steps:

[0010] Step 1) Calculate the band position code:

[0011] Divide the hyperspectral image into blocks, and each hyperspectral image block is used as a sample;

[0012] According to the band position encoding strategy, the band position information is converted into an embedding vector, and the calculation formula is expressed as:

[0013]

[0014] Among them, pos represents the position of a certain band in the set of all bands, i represents the position of the pixel in the band vector, D represents the number of pixels contained in a band vector, BPE(.) represents the band position encoding result of a pixel in a certain band, and the position encoding results of all bands constitute the band position encoding matrix;

[0015] Step 2) Build and train the Transformer reconstruction network:

[0016] Constructing a Transformer reconstruction network; the Transformer reconstruction network includes a Transformer-based band multi-head self-attention module and a reconstruction module based on a convolutional autoencoder;

[0017] The Transformer-based band multi-head self-attention module takes the band position encoding matrix and the sum of the hyperspectral image samples expanded by band as input to obtain the band attention mask. The calculation formula is expressed as:

[0018] a=F TBA (Y;θ t )

[0019] Where a=(a1,a2,...,aL ) represents the band attention mask vector, a i represents the attention mask of the i-th band, L represents the number of bands, Y represents the sum of the band position encoding matrix and the hyperspectral image samples expanded by band, and F TBA (·) represents the Transformer-based band multi-head self-attention module, θ t Represents the trainable parameters in the Transformer-based band multi-head self-attention module;

[0020] The band multi-head self-attention module contains one or more multi-head self-attention layers to suppress the deviation that may be generated by a single attention head. The calculation formula is expressed as:

[0021] F MHSA (M)=Concat(head1,...,head h )W O

[0022] Among them, F MHSA (·) represents the multi-head self-attention layer, M represents the input of the multi-head self-attention layer, Concat(·) represents the concatenation operation, represents the i-th attention head, d = D / h represents the dimension of the key in each attention head, h represents the number of attention heads, and W O 、W Q 、W K and W V Both represent the parameter matrix of the projection;

[0023] The spectral bands in the original hyperspectral image patch are reweighted using the band attention mask vector. The obtained reweighted bands are used as the input of the reconstruction module based on the convolutional autoencoder to reconstruct the original hyperspectral image patch.

[0024] The Transformer reconstruction network is trained with the minimum reconstruction error and sparse band attention mask vector as the optimization goals;

[0025] Step 3) Measure band representativeness:

[0026] The representativeness of each candidate band is measured using the band attention mask vector output by the trained Transformer-based band multi-head self-attention module. During initialization, all bands are considered as candidate bands to form a candidate band set.

[0027] The representative calculation formula of the band is expressed as:

[0028]

[0029] Among them, p(x t) represents the candidate band x t The representativeness of the metric, n represents the number of samples, Represents the candidate band x in the i-th sample t 's attention mask;

[0030] Select the candidate band with the highest representativeness metric value from the candidate band set as the first selected band, construct a selected band subset, and remove it from the candidate band set;

[0031] Step 4) Measure band redundancy:

[0032] The redundancy between the current candidate band and the currently selected band is calculated using the band correlation matrix obtained from the last multi-head self-attention layer in the trained Transformer-based band multi-head self-attention module;

[0033] The band correlation matrix is obtained through the key matrix and the query matrix, and the calculation formula is expressed as:

[0034]

[0035] The calculation formula for the redundancy between the current candidate band and the currently selected band is expressed as:

[0036]

[0037] Among them, d(x t ) represents the candidate band x t The redundancy of q represents the number of currently selected bands, C t,k Represents candidate band x t Correlation with the kth selected band;

[0038] Step 5) Construct a comprehensive evaluation index that takes into account both band representativeness and redundancy. The calculation formula is expressed as:

[0039] s(x t )=p(x t )-β×d(x t )

[0040] Among them, β represents the balance coefficient of the two metrics, s(x t ) represents the candidate band x t The comprehensive score of

[0041] Step 6) Search for the desired band subset:

[0042] Sort the comprehensive scores of all current candidate bands, and select the candidate band with the highest current score to add to the selected band subset;

[0043] Repeat steps 4) to 6) until the number of bands included in the selected band subset reaches a preset value.

[0044] Furthermore, the calculation expression for re-weighting the spectral bands in the original hyperspectral image block using the band attention mask in step 2) is:

[0045]

[0046] Among them, Z represents the reweighted spectral band, X P represents the original hyperspectral image patch, Represents the multiplication sign by band.

[0047] Furthermore, in step 2), the reweighted band is used as the input of the reconstruction module based on the convolutional autoencoder to reconstruct the original hyperspectral image block. The calculation expression of the reconstruction module is:

[0048]

[0049] in, represents the reconstructed hyperspectral image patch, θ C represents the trainable parameters in the reconstruction module based on the convolutional autoencoder, F CAE (·) represents the convolutional autoencoder operation.

[0050] Furthermore, the loss function used to train the Transformer reconstruction network in step 2) is:

[0051]

[0052] Among them, L(θ t ,θ c ) represents the loss function, θ t and θ c represents the trainable parameters, n is the number of samples of the original hyperspectral image block, represents the i-th original hyperspectral image block, represents the i-th reconstructed hyperspectral image patch, represents the square of the F norm, γ represents the balance coefficient, and ||·||1 represents the 1 norm.

[0053] Furthermore, the Transformer-based band multi-head self-attention module in step 2) is composed of 6 identical band attention blocks and a pooling layer in series, and the band attention block includes a multi-head self-attention layer, a first normalization layer, a feedforward layer and a second normalization layer.

[0054] The present invention also provides a hyperspectral band selection device based on a Transformer network, the device comprising:

[0055] A band position code calculation module, used for calculating the band position code;

[0056] Transformer reconstruction network construction and training module, used to build and train the Transformer reconstruction network;

[0057] Band representativeness measurement module, used to measure the representativeness of candidate bands;

[0058] Band redundancy measurement module, used to measure the redundancy between the candidate band and the currently selected band;

[0059] A comprehensive evaluation index construction module that takes into account both band representativeness and redundancy is used to design a band scoring function that takes into account both band representativeness and redundancy;

[0060] The expected band subset search module is used to search for the expected bands with the highest comprehensive scores one by one and add them to the selected band subset;

[0061] The band selection result output module is used to output the selected optimal band subset results.

[0062] The beneficial effects of the present invention are:

[0063] 1) To address the problem in existing technologies that a single attention head cannot accurately mine the representativeness of a band, this paper proposes a Transformer-based band multi-head attention module. By using the multi-head attention mechanism to suppress the deviation that may be caused by a single attention head, it is beneficial to improve the pixel classification accuracy.

[0064] 2) The present invention proposes to effectively combine the Transformer with the convolutional autoencoder to construct a Transformer reconstruction network for hyperspectral image band selection, which can solve the problem that the existing technology is usually unable to exploit the inherent nonlinear relationship between bands, and is conducive to the implementation of downstream tasks.

[0065] 3) In view of the fact that the existing technology cannot make good use of spatial and spectral information while taking into account the representativeness and redundancy of bands, the present invention proposes to use a Transformer reconstruction network that can simultaneously utilize the spatial and spectral information of hyperspectral images to measure the representativeness and redundancy of bands, so as to select a band subset with high representativeness and low redundancy, and further improve the implementation effect of downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a flowchart of the basic steps of an embodiment of the hyperspectral band selection method based on the Transformer network of the present invention.

[0067] Figure 2It is a structural schematic diagram of the hyperspectral image band selection device of the present invention.

[0068] Figure 3 These are real ground object images from the Indian Pines dataset.

[0069] Figure 4 Classification accuracy curves of different band selection methods when using the SVM classifier on the Indian Pines dataset.

[0070] Figure 5 Classification accuracy curves of different band selection methods when using the EPF-Gg classifier on the Indian Pines dataset.

[0071] Figure 6 SVM classification diagram when 15 bands are selected from the Indian Pines dataset using different band selection methods. DETAILED DESCRIPTION

[0072] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0073] like Figure 1 FIG. 1 is a flowchart of the basic steps of an embodiment of the invented hyperspectral image band selection method, which specifically includes the following steps.

[0074] Input: original hyperspectral image X∈R L×W×H , where L represents the number of bands and W×H represents the number of pixels; select the number of bands k.

[0075] Step 1: Calculate the band position code.

[0076] (1) Divide the original hyperspectral image into blocks and represent the hyperspectral image blocks as X P ∈R L×m×m , m×m is the pixel size after blocking, each hyperspectral image block is taken as a sample, and a total of n samples are obtained.

[0077] (2) Design a band position encoding strategy to convert the band position information into an embedding vector. The calculation formula is expressed as:

[0078]

[0079] Among them, pos represents the position of a certain band in the set of all bands, i represents the position of the pixel in the band vector, D represents the number of pixels contained in a band vector, BPE(.) represents the band position encoding result of a pixel in a certain band, and the position encoding results of all bands constitute the band position encoding matrix.

[0080] Step 2: Build and train the Transformer reconstruction network.

[0081] (1) Taking the sum of the band position code and the hyperspectral image samples expanded by band as input, a Transformer-based band multi-head self-attention module is constructed to obtain the band attention mask vector. The calculation formula is expressed as:

[0082] a=F TBA (Y;θ t )

[0083] Where a=(a1,a2,...,a L ) represents the band attention mask vector, a i represents the attention mask of the i-th band, L represents the number of bands, Y represents the sum of the band position encoding and the hyperspectral image samples expanded by band, and F TBA (·) represents the Transformer-based band multi-head self-attention module, θ t Represents the trainable parameters in the Transformer-based band multi-head self-attention module.

[0084] In a specific implementation of this embodiment, the Transformer-based band multi-head self-attention module consists of 6 identical band attention blocks and a pooling layer, where each band attention block is mainly composed of a multi-head self-attention layer and a feedforward layer, and each multi-head self-attention layer and feedforward layer is followed by a normalization layer.

[0085] In each band attention block, the input data is first used as the input of the multi-head self-attention layer. The calculation formula of the multi-head self-attention layer is expressed as:

[0086] F MHSA (M)=Concat(head1,...,head h )W O

[0087] Among them, F MHSA (·) represents the multi-head self-attention layer, M represents the input of the multi-head self-attention layer, Concat(·) represents the concatenation operation, represents the i-th attention head, d = D / h represents the dimension of the key in each attention head, h represents the number of attention heads, and WO 、W Q 、W K and W V Both represent the parameter matrix of the projection.

[0088] Subsequently, the features of all the learned attention heads are further transformed using the feedforward layer. Each feedforward layer contains two linear transformations with a ReLU activation function in between. The calculation formula of the feedforward layer is expressed as:

[0089] F Feed (x) = ReLU(xW1+b1)W2+b2

[0090] Where x is the input of the feedforward layer, W1 and W2 represent weight parameters, b1 and b2 represent bias vectors, and ReLU(·) represents the ReLU activation function.

[0091] (2) The spectral bands in the hyperspectral image block are reweighted using the band attention mask. The calculation formula is expressed as:

[0092]

[0093] Among them, Z represents the reweighted spectral band, X P represents the original hyperspectral image patch, Represents the multiplication sign by band.

[0094] The obtained re-weighted bands are used as input to the reconstruction module based on convolutional autoencoder to reconstruct the original hyperspectral image patch:

[0095]

[0096] in, represents the reconstructed hyperspectral image patch, θ C represents the trainable parameters in the reconstruction module based on the convolutional autoencoder, F CAE (·) represents the convolutional autoencoder operation.

[0097] In this embodiment, the reconstruction module based on the convolutional autoencoder includes a two-dimensional convolutional neural network, batch normalization, a nonlinear activation function layer, a residual connection and other network structures. In this embodiment, the reweighted spectral band set Z is first passed through two-dimensional convolutional layers with output channel sizes of 128, 64 and 32 in sequence, the convolution kernel size is 3×3, and each convolution layer is followed by batch normalization and ELU activation function processing; then, the obtained data is input into a two-dimensional deconvolution layer with a kernel and an output channel size of 3×3 and 64 respectively, combined with batch normalization and ELU activation function processing, and the result of the activation function output is added to the input value of the aforementioned third two-dimensional convolutional layer; the obtained data is input into a two-dimensional deconvolution layer with a kernel and an output channel size of 3×3 and 128 respectively, combined with batch normalization and ELU activation function processing, and the result of the activation function output is added to the input value of the aforementioned second two-dimensional convolutional layer; finally, the obtained data is input into a two-dimensional deconvolution layer with a kernel size of 3×3 and an output channel number equal to the original band number L, combined with batch normalization and Sigmoid activation function processing, to obtain the reconstructed original hyperspectral image block.

[0098] (3) Define the loss function with the optimization objectives of minimizing the reconstruction error and sparse band attention mask vectors:

[0099]

[0100] Among them, L(θ t ,θ c ) represents the loss function, θ t and θ c represents the trainable parameters, n is the number of samples of the original hyperspectral image block, represents the i-th original hyperspectral image block, represents the i-th reconstructed hyperspectral image patch, represents the square of the F norm, γ represents the balance coefficient, and ||·||1 represents the 1 norm.

[0101] By minimizing the loss function, the Transformer is trained to reconstruct the network.

[0102] Step 3: Use the band attention mask vector a to measure the representativeness of the band:

[0103]

[0104] Among them, p(x t ) represents the candidate band x t The representativeness of n is the number of samples. Represents the candidate band x in the i-th sample t Attention mask.

[0105] Step 4: Measure band redundancy.

[0106] The band correlation matrix is obtained by using the key matrix and query matrix in the last multi-head self-attention layer in the Transformer-based band multi-head self-attention module:

[0107]

[0108] The redundancy between the candidate bands and the currently selected bands is calculated using the band correlation matrix;

[0109]

[0110] Among them, d(x t ) represents the candidate band x t The redundancy of q represents the number of currently selected bands, C t,k Represents candidate band x t The correlation with the kth selected band.

[0111] Step 5: Construct a comprehensive evaluation index that takes into account both band representativeness and redundancy. The calculation formula is expressed as:

[0112] s(x t )=p(x t )-β×d(x t )

[0113] Among them, β represents the balance coefficient of the two metrics, s(x t ) represents the candidate band x t The comprehensive score of .

[0114] Step 6: Search for the desired band subset.

[0115] In the initialization phase, the band with the highest representativeness metric value is selected from all bands as the first selected band to construct the initial selected band subset.

[0116] Afterwards, the comprehensive scores of all unselected bands are sorted, and the band with the highest current score is selected to be added to the selected band subset.

[0117] The operation of sorting the currently unselected bands and adding the bands with the highest scores to the selected band subset is repeated until the number of bands contained in the selected band subset reaches a preset value.

[0118] Corresponding to the aforementioned embodiment of the hyperspectral band selection method based on the Transformer network, the present application also provides an embodiment of a hyperspectral band selection device based on the Transformer network, which includes:

[0119] A band position code calculation module, used for calculating the band position code;

[0120] Transformer reconstruction network construction and training module, used to build the Transformer reconstruction network and perform appropriate training;

[0121] Band representativeness measurement module, used to measure the representativeness of candidate bands;

[0122] Band redundancy measurement module, used to measure the redundancy between the candidate band and the currently selected band;

[0123] A comprehensive evaluation index construction module that takes into account both band representativeness and redundancy is used to design a band scoring function that takes into account both band representativeness and redundancy;

[0124] The expected band subset search module is used to search for the expected bands with the highest comprehensive scores one by one and add them to the selected band subset;

[0125] The band selection result output module is used to output the selected optimal band subset results.

[0126] In a specific implementation of the present invention, the band selection device further includes an application module, which uses the band selection result to perform hyperspectral image classification or target detection.

[0127] In a specific implementation of the present invention, the band position code calculation module includes:

[0128] Image block module, used to divide the hyperspectral image into blocks, each hyperspectral image block as a sample;

[0129] The band position encoding module is used to convert the band position information into an embedding vector.

[0130] In a specific implementation of the present invention, the Transformer reconstruction network construction and training module includes:

[0131] A Transformer-based band multi-head self-attention module to obtain band attention mask vectors;

[0132] A reconstruction module based on convolutional autoencoder to reconstruct the original hyperspectral image patches;

[0133] The network training module is used to design the loss function and optimize the parameters in the Transformer reconstruction network.

[0134] Regarding the apparatus in the above embodiment, the specific manner in which each unit or module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0135] As for the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the band position coding calculation module and the Transformer reconstruction network construction and training module may or may not be physically separated. In addition, the various functional modules in the present invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned integrated modules or units can be implemented in the form of hardware or in the form of software functional units, so that some or all of the modules can be selected according to actual needs to achieve the purpose of the present application.

[0136] To verify the effectiveness of the present invention, an experiment was conducted on a real hyperspectral image. The IndianPines dataset is used as an example to illustrate the specific implementation method. The experiment is as follows:

[0137] The image used in this embodiment is the Indian Pines dataset, which is a hyperspectral image taken by the AVIRIS sensor. Figure 3 As shown, the size is 145×145 pixels. Excluding the water vapor absorption band and the band with low signal-to-noise ratio, the remaining 185 bands are involved in the experiment.

[0138] To further verify the application effect of the present invention, the results obtained by the present invention and other methods were used for pixel classification. In order to more intuitively compare the impact of various band selection methods on the accuracy of downstream classification tasks, Figure 4 The curve showing the classification accuracy of different band selection methods as the number of selected bands changes when using the SVM classifier on the Indian Pines dataset is shown. The horizontal axis is the number of selected bands, and the vertical axis is the overall classification accuracy. Figure 4 As shown, for the SVM classifier, when different numbers of bands are selected, the specific embodiment of the present invention can always achieve a classification accuracy that is significantly higher than that of other band selection methods.

[0139] Figure 5 The curve showing the classification accuracy of different band selection methods as the number of selected bands changes when using the EPF-Gg classifier on the Indian Pines dataset. The horizontal axis is the number of selected bands, and the vertical axis is the overall classification accuracy. Figure 5 The results show that the specific example of the present invention can achieve a classification effect that is significantly better than other band selection methods on the EPF-Gg classifier.

[0140] Figure 6The SVM classification diagrams are shown when MVPCA (a), LCMVBCC (b), LCMVBCM (c), OPBS (d) and the specific example of the present invention (e) are used on the Indian Pines dataset to select 15 bands. Figure 6 It can be seen that the specific example of the present invention has the best overall classification effect on various types of ground objects.

[0141] Table 1 Comparison of classification performance of different band selection methods on SVM classifier

[0142] method OA (%) AA (%) 1.MVPCA 64.81 50.83 2.LCMVBCC 58.95 49.74 3.LCMVBCM 66.90 60.98 4.OPBS 72.33 62.97 5.MR 78.42 71.24 6.BS-Net-Conv 78.91 72.27 7. The present invention 81.31 73.39

[0143] Table 1 shows the classification accuracy achieved by different band selection methods using the SVM classifier on the Indian Pines dataset, using k = 15 as an example. The table also shows the results of two quantitative evaluation metrics for pixel classification: OA (Overall Accuracy) and AA (Average Accuracy). The OA refers to the ratio of correctly classified samples to the total number of samples, while the AA refers to the average classification accuracy of samples for each ground feature category. As can be seen from Table 1, the Transformer network-based hyperspectral band selection method proposed in this paper can improve the classification effect of band selection.

[0144] The present invention describes specific embodiments to simplify the present invention. However, it should be understood that the present invention is not limited to the embodiments described, and that various modifications of the present invention are possible without departing from the basic principles, and that equivalent forms of the present invention also fall within the scope defined by the appended claims.

Claims

1. A hyperspectral band selection method based on Transformer network, characterized in that: The steps include: Step 1) Calculate the band position code: Divide the hyperspectral image into blocks, and each hyperspectral image block is used as a sample; According to the band position encoding strategy, the band position information is converted into an embedding vector, and the calculation formula is expressed as: ； Among them, pos represents the position of a certain band in the set of all bands, i represents the position of the pixel in the band vector, D represents the number of pixels contained in a band vector, BPE(.) represents the band position encoding result of a pixel in a certain band, and the position encoding results of all bands constitute the band position encoding matrix; Step 2) Build and train the Transformer reconstruction network: Constructing a Transformer reconstruction network; the Transformer reconstruction network includes a Transformer-based band multi-head self-attention module and a reconstruction module based on a convolutional autoencoder; The Transformer-based band multi-head self-attention module takes the band position encoding matrix and the sum of the hyperspectral image samples expanded by band as input to obtain the band attention mask. The calculation formula is expressed as: ； in, represents the band attention mask vector, represents the attention mask of the i-th band, L represents the number of bands, Represents the sum of the band position encoding matrix and the hyperspectral image samples expanded by band, Represents the Transformer-based band multi-head self-attention module, Represents the trainable parameters in the Transformer-based band multi-head self-attention module; The band multi-head self-attention module contains one or more multi-head self-attention layers to suppress the deviation that may be generated by a single attention head. The calculation formula is expressed as: ； in, represents the multi-head self-attention layer, M represents the input of the multi-head self-attention layer, Represents the splicing operation, represents the i-th attention head, represents the dimension of the key in each attention head, h represents the number of attention heads, 、 、 and Both represent the parameter matrix of the projection; The spectral bands in the original hyperspectral image patch are reweighted using the band attention mask vector. The obtained reweighted bands are used as the input of the reconstruction module based on the convolutional autoencoder to reconstruct the original hyperspectral image patch. The Transformer reconstruction network is trained with the minimum reconstruction error and sparse band attention mask vector as the optimization goals; Step 3) Measure band representativeness: The representativeness of each candidate band is measured using the band attention mask vector output by the trained Transformer-based band multi-head self-attention module. During initialization, all bands are considered as candidate bands to form a candidate band set. The representative calculation formula of the band is expressed as: ； in, Representative candidate band The representativeness of the metric, n represents the number of samples, Represents the candidate band in the i-th sample 's attention mask; Select the candidate band with the highest representativeness metric value from the candidate band set as the first selected band, construct a selected band subset, and remove it from the candidate band set; Step 4) Measure band redundancy: The redundancy between the current candidate band and the currently selected band is calculated using the band correlation matrix obtained from the last multi-head self-attention layer in the trained Transformer-based band multi-head self-attention module; The band correlation matrix is obtained through the key matrix and the query matrix, and the calculation formula is expressed as: ； The calculation formula for the redundancy between the current candidate band and the currently selected band is expressed as: ； in, Representative candidate band The redundancy of q represents the number of currently selected bands. Representative candidate band Correlation with the kth selected band; Step 5) Construct a comprehensive evaluation index that takes into account both band representativeness and redundancy. The calculation formula is expressed as: ； in, represents the balance coefficient of the two metrics, Representative candidate band The comprehensive score of Step 6) Search for the desired band subset: Sort the comprehensive scores of all current candidate bands, and select the candidate band with the highest current score to add to the selected band subset; Repeat steps 4) to 6) until the number of bands included in the selected band subset reaches a preset value.

2. The hyperspectral band selection method based on Transformer network according to claim 1, characterized in that: The calculation expression for re-weighting the spectral bands in the original hyperspectral image block using the band attention mask in step 2) is: ； in, represents the reweighted spectral bands, represents the original hyperspectral image patch, Represents the multiplication sign by band.

3. The hyperspectral band selection method based on Transformer network according to claim 2, characterized in that: In step 2), the reweighted bands are used as the input of the reconstruction module based on the convolutional autoencoder to reconstruct the original hyperspectral image block. The calculation expression of the reconstruction module is: ； in, represents the reconstructed hyperspectral image patch, represents the trainable parameters in the reconstruction module based on the convolutional autoencoder, Represents a convolutional autoencoder operation.

4. The hyperspectral band selection method based on Transformer network according to claim 1, characterized in that: The loss function used to train the Transformer reconstruction network in step 2) is: ； in, represents the loss function, and represents the trainable parameters, is the number of samples of the original hyperspectral image block, represents the i-th original hyperspectral image block, represents the i-th reconstructed hyperspectral image patch, represents the square of the F norm, represents the balance coefficient, Represents the 1-norm.

5. The hyperspectral band selection method based on Transformer network according to claim 1, characterized in that: The Transformer-based band multi-head self-attention module in step 2) is composed of 6 identical band attention blocks and a pooling layer in series. The band attention block includes a multi-head self-attention layer, a first normalization layer, a feedforward layer, and a second normalization layer.

6. A hyperspectral band selection device based on a Transformer network, characterized in that: For implementing the hyperspectral band selection method according to claim 1, the hyperspectral band selection device comprises: A band position code calculation module, used for calculating the band position code; Transformer reconstruction network construction and training module, used to build and train the Transformer reconstruction network; Band representativeness measurement module, used to measure the representativeness of candidate bands; Band redundancy measurement module, used to measure the redundancy between the candidate band and the currently selected band; A comprehensive evaluation index construction module that takes into account both band representativeness and redundancy is used to design a band scoring function that takes into account both band representativeness and redundancy; The expected band subset search module is used to search for the expected bands with the highest comprehensive scores one by one and add them to the selected band subset; The band selection result output module is used to output the selected optimal band subset results.

7. The hyperspectral band selection device based on Transformer network according to claim 6, characterized in that: The band selection device further includes an application module, which uses the band selection result to perform hyperspectral image classification or target detection.

8. The hyperspectral band selection device based on Transformer network according to claim 6, characterized in that: The band position coding calculation module includes: Image block module, used to divide the hyperspectral image into blocks, each hyperspectral image block as a sample; The band position encoding module is used to convert the band position information into an embedding vector.

9. The hyperspectral band selection device based on Transformer network according to claim 6, characterized in that: The Transformer reconstruction network construction and training module includes: A Transformer-based band multi-head self-attention module to obtain band attention mask vectors; A reconstruction module based on convolutional autoencoder to reconstruct the original hyperspectral image patches; The network training module is used to design the loss function and optimize the parameters in the Transformer reconstruction network.

Images