Hyperspectral band selection method based on mask image sensing Transformer
By constructing an adaptive band similarity map and a masked map autoencoder, and combining graph topological prior and Transformer self-attention, the problems of local redundancy and global non-uniformity in band selection in hyperspectral image processing are solved, achieving more comprehensive feature representation and higher classification accuracy.
Patent Information
- Application Number
- CN202611290656.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-25
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2046-08-25
AI Technical Summary
Existing hyperspectral image processing methods suffer from high local redundancy and uneven global distribution in band selection. Traditional methods struggle to effectively combine local topological structures and global dependencies between bands, resulting in incomplete feature representation.
A masked graph-aware Transformer-based approach is adopted. By constructing an adaptive band similarity graph and introducing a masked graph autoencoder with a cross-layer interaction mechanism, combined with graph topology prior information and Transformer multi-head self-attention mechanism, local topology and global context information between bands are extracted to generate a comprehensive score for selecting the optimal band subset.
It effectively characterizes the correlation between bands, selects a subset of bands with high information content and low redundancy, and improves the feature representation capability and classification accuracy of hyperspectral image processing.
Smart Images

Figure CN122799134B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of hyperspectral image processing technology, and more specifically, to a hyperspectral band selection method based on a mask-image-aware Transformer. Background Technology
[0002] Hyperspectral images possess both continuous and detailed spectral information and rich spatial information, accurately reflecting ground features and have been widely used in various fields. However, the large number of bands and strong correlations between bands in hyperspectral images lead to significant redundancy, increasing the difficulty of data processing and affecting subsequent classification and recognition performance. Dimensionality reduction has therefore become a crucial step in hyperspectral image processing.
[0003] Among commonly used dimensionality reduction methods, unsupervised band selection methods can directly select representative subsets from the original bands, better preserving the original spectral characteristics and their physical meaning while reducing dimensionality. Moreover, they do not rely on manual labeling and are more suitable for complex remote sensing application scenarios that lack large-scale labeled samples.
[0004] Traditional unsupervised band selection methods mainly include ranking-based, search-based, and clustering-based methods, but they have limitations such as ignoring inter-band correlations, excessive computational overhead, or difficulty in representing nonlinear high-order relationships. In recent years, convolutional neural network-based methods have been introduced into this field; however, they are limited by fixed receptive fields and regular grid structures, making it difficult to capture the complex topological structures between bands. To address this, existing technologies have introduced graph representation learning-based methods, which abstract bands as graph nodes and construct correlations as edges, using message passing mechanisms to mine local topological structures. However, these methods still rely on static distance metrics or fixed nearest neighbors for graph construction, making it difficult to adaptively characterize the non-uniform distribution characteristics of bands, and the local aggregation mechanism of graph convolution is difficult to efficiently capture the global dependencies between distant bands.
[0005] While attention mechanisms based on the Transformer architecture can capture long-distance global dependencies, they treat bands as independent tokens and rely solely on content similarity when calculating attention. This lacks effective perception of graph topology and is prone to introducing noise interference.
[0006] Therefore, existing methods, which employ a single graph representation learning or traditional Transformer structure, fail to effectively combine the complementary advantages of both, resulting in high local redundancy and uneven global distribution in the selected bands. Summary of the Invention
[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a hyperspectral band selection method based on mask image sensing Transformer.
[0008] To achieve the above objectives, the present invention adopts the following technical solution:
[0009] A hyperspectral band selection method based on mask image-aware Transformer includes the following steps:
[0010] Step S1: Construct an adaptive band similarity map based on local feature density to characterize the local topological relationships between bands and provide reliable structural prior information for subsequent feature learning.
[0011] Step S2: Construct a mask graph autoencoder based on a cross-layer interaction mechanism. The real edges of the masked part are used and the existence prediction of the edges is used as the self-supervised optimization objective. At the same time, a cross-layer feature fusion mechanism is introduced in the mask edge reconstruction task to extract deep local topological structure information between bands.
[0012] Step S3: By introducing graph topology prior information into the Transformer multi-head self-attention mechanism, a graph-aware Transformer reconstruction network is constructed. With the help of the self-supervised reconstruction task of random masked bands, long-distance global context dependency information between bands is extracted.
[0013] Step S4: Integrate local topology and global context information to generate a comprehensive score, and select the optimal band subset in descending order of the scores.
[0014] Furthermore, step S1 includes the following steps:
[0015] Step S11: Expand the original hyperspectral image into a pixel-band matrix and standardize it, using each band as a graph node and the corresponding standardized feature vector as the node feature.
[0016] Step S12: Calculate the distances between band nodes, sort the distances from each band node to the other band nodes in ascending order, and take the distance of the sorted node. The bit distance value is used as its local scale parameter;
[0017] Step S13: The bidirectional nearest neighbor union strategy is used to determine the connection relationship of the band nodes, and the connection weight is calculated by combining the band node spacing and local scale parameters to generate a symmetric weighted adjacency matrix.
[0018] Step S14: Combine the graph nodes, node features, and adjacency matrix to form an adaptive band similarity graph.
[0019] Furthermore, in step S13, the expression for calculating the connection weights is:
[0020]
[0021] In the formula, For band nodes and band nodes The connection weight values between them; For band nodes Centered Nearest neighbor set; For band nodes Centered nearest neighbor set The parameter is the number of nearest neighbors; For band nodes and band nodes The Euclidean distance between them; For band nodes Local scale parameters; For band nodes Local scale parameters; It is a constant; It is an exponential function with the natural constant e as its base; and For band node indexing, , This represents the total number of bands.
[0022] Furthermore, step S2 includes the following steps:
[0023] Step S21: Randomly sample some edges from the real edges of the adaptive band similarity graph as positive samples and remove them to obtain the masked input graph;
[0024] Step S22: Input the masked input graph into the multi-layer graph convolutional encoder, update the node information layer by layer, and map the output of each layer to the same dimension to obtain a multi-layer node embedding set.
[0025] Step S23: Randomly sample negative sample edges from node pairs without topological connections and merge them with positive sample edges to form a candidate predicted edge set;
[0026] Step S24: Perform cross-layer feature element-by-element interactive splicing on the nodes at both ends of the candidate edge to generate candidate edge features that integrate multi-layer topological semantics;
[0027] Step S25: Input the candidate edge features into the multilayer perceptron to predict the edge existence probability, and optimize the parameters of the graph autoencoder with binary cross-entropy loss so that the positive sample prediction approaches 1 and the negative sample prediction approaches 0.
[0028] Furthermore, in step S24, the expression for element-wise interleaving of cross-layer features is:
[0029]
[0030] In the formula, Indicate candidate edges eigenvectors; This indicates the concatenation of feature dimensions; For band nodes In the The embedded representation output by the layer graph convolutional encoder; For band nodes In the The embedded representation output by the layer graph convolutional encoder; This represents the total number of layers in the graph convolutional encoder. , For graph convolutional encoder layer index, ; For element-wise multiplication; and For band node indexing, , This represents the total number of bands.
[0031] Furthermore, step S3 includes the following steps:
[0032] Step S31: Using the band embedding output by the graph encoder as input, randomly mask some bands and replace them with learnable vectors to form the input sequence;
[0033] Step S32: Construct a graph structure attention bias matrix based on the adjacency matrix and inject it into the self-attention calculation process of each layer of the graph-aware Transformer;
[0034] Step S33: Through residual connection, layer normalization and feedforward network layer-by-layer encoding, the final global band representation that integrates global context and graph prior is obtained;
[0035] Step S34: Input the final global band representation into the decoder to reconstruct the masked band, and optimize the graph-aware Transformer with mean square error as the reconstruction loss.
[0036] Furthermore, in step S32, during the self-attention calculation process of each layer of the graph-aware Transformer, the graph-aware attention operation is defined as follows:
[0037]
[0038] In the formula, Indicates the first Layer diagrams perceive the output of self-attention; , and They represent the first The query matrix, key matrix, and value matrix of the layer; This represents the feature dimension of the key vector; To inject weights into the graph bias that is related to the number of layers. For layer index, To perceive the total number of layers in the Transformer encoder; This represents the Softmax normalization function; It is the transpose operator; The attention bias matrix represents the graph structure.
[0039] Furthermore, step S4 includes the following steps:
[0040] Step S41: After normalizing the adjacency matrix by row, the neighborhood feature differences of each band are weighted and aggregated to obtain the local structure contribution score.
[0041] Step S42: Accumulate the graph to perceive the attention weights of other bands to the current band in each layer of the Transformer and obtain the global context contribution score;
[0042] Step S43: Normalize the local structure contribution score and the global context contribution score respectively;
[0043] Step S44: The two normalized scores are weighted and summed to obtain the fused score;
[0044] Step S45: Sort all bands in descending order according to the fusion score, and select the top-ranked bands. One band is selected as the optimal hyperspectral band subset.
[0045] Furthermore, in step S41, the local structure contribution score is obtained as follows:
[0046] For the For each band node, its neighborhood aggregation feature is defined as follows:
[0047]
[0048]
[0049] In the formula, Indicates band node The neighborhood aggregation feature is represented; Band nodes in a band similarity graph and band nodes Normalized adjacency weights between them; Indicates band node Embedded representation; Indicates band node and band nodes The original edge weights between them; Indicates band node and band nodes The original edge weights between them; Indicates the total number of bands; and For band node indexing, ; Represented as the index variable in the summation process, ; It is a constant;
[0050] Definition of the first Local structural contribution score for each band node:
[0051]
[0052] In the formula, Indicates band node Local structural contribution score; This represents the Euclidean norm of a vector.
[0053] Furthermore, in step S42, the global context contribution score is obtained as follows:
[0054]
[0055] In the formula, Indicates band node Global context contribution score; This represents the total number of layers in the graph-aware Transformer encoder; This represents the graph-aware Transformer encoder layer index. ; Indicates the first In layer self-attention, band nodes For band nodes Attention weights; and For band node indexing, , This indicates the total number of bands.
[0056] The beneficial effects of this invention are:
[0057] 1. This invention constructs a band similarity graph by introducing a dynamic scale parameter based on the local density information of band features. This graph construction strategy overcomes the shortcomings of traditional fixed nearest neighbor parameters, which are difficult to fit the non-uniform distribution characteristics of hyperspectral data. It can more reasonably characterize the correlation between bands and provide a reliable graph structure prior for subsequent feature representation learning.
[0058] 2. This invention proposes a graph mask autoencoder-based masked edge reconstruction task to mine local structural representations between bands. Simultaneously, by designing a graph-aware Transformer that explicitly injects graph structure biases into its self-attention mechanism, topological priors are integrated into global context modeling. This dual-architecture design overcomes the limitations of single graph convolution methods in capturing long-distance global information and the lack of local graph topological awareness in traditional Transformers. This allows feature interactions to effectively focus on physically related bands, resulting in more comprehensive band feature representations.
[0059] 3. This invention combines a local structural contribution score, reflecting neighborhood aggregation capability, generated by a graph mask autoencoder, with a global context contribution score, reflecting global dependency, extracted by a graph-aware Transformer, to form a weighted comprehensive evaluation criterion for bands with dual scores. This criterion integrates complementary information from both local and global perspectives, helping to retain key discriminative features and effectively suppress redundant features when assessing band importance, thereby selecting a subset of bands with high information content and low redundancy. Attached Figure Description
[0060] Figure 1 This is a flowchart of a hyperspectral band selection method based on mask image sensing Transformer in this embodiment;
[0061] Figure 2 This is a comparison chart of classification accuracy for different band selection methods when using an SVM classifier on the Pavia University dataset in this embodiment.
[0062] Figure 3 This is a comparison chart of classification accuracy for different band selection methods when using the SVM classifier on the Salinas dataset in this embodiment. Detailed Implementation
[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0064] Example: A hyperspectral band selection method based on masked graph-aware Transformer, which includes four parts: adaptive band similarity graph construction, local topological association modeling based on masked graph autoencoder, global context dependency modeling based on graph-aware Transformer, and dual-score fusion and band optimization. Among them, the adaptive band similarity map is dynamically constructed based on the local density information of band features, providing a reliable structural prior for feature learning; the masked graph autoencoder randomly hides some real edges in the band similarity map, uses a multi-layer graph convolutional encoder to aggregate neighborhood features and perform cross-layer feature interaction, and mines the local topological associations between bands through the masked edge reconstruction task, generating a local structural contribution score that reflects the neighborhood aggregation ability; the graph-aware Transformer adopts a random masked band modeling strategy, transforms the band similarity map into a structural bias and explicitly injects it into the self-attention calculation, captures long-distance global contextual dependencies through the reconstruction task of the masked bands, and generates a global contextual contribution score; finally, a dual-score fusion mechanism is used to normalize and weight the two types of complementary scores, local and global, to comprehensively evaluate and rank the importance of bands and select the optimal subset of bands.
[0065] like Figure 1 As shown, the method includes the following steps:
[0066] Step S1: Construct an adaptive band similarity map based on local feature density to characterize the local topological relationships between bands and provide reliable structural prior information for subsequent feature learning.
[0067] This step aims to transform raw hyperspectral data into graph-structured data. By introducing an adaptive neighborhood scale based on local density, it overcomes the shortcomings of traditional fixed nearest neighbor parameters that are difficult to fit the non-uniform distribution characteristics of hyperspectral data, and provides a reliable graph structure prior for subsequent graph representation learning.
[0068] For the original hyperspectral image data cube , ,in and These represent the height and width of a single-band image, respectively. Indicates the number of bands. It is the set of real numbers.
[0069] Specifically, step S1 includes the following steps:
[0070] Step S11: Hyperspectral data preprocessing and feature node construction.
[0071] The original hyperspectral image data is unfolded into a matrix in the spatial dimension. , ,in Total number of pixels For matrices Standardization by column yields Subsequently, a band feature set is constructed by using each band as a graph node and the corresponding standardized feature vector as the node feature. :
[0072]
[0073] In the formula, ; It is the transpose operator; , indicating the first Standardized feature vectors of each band node .
[0074] Step S12: Adaptive extraction of inter-band distance metrics and local scale parameters.
[0075] The distance between any two band nodes can be measured using Euclidean distance, Mahalanobis distance, or cosine similarity. When using Euclidean distance, the calculation formula is as follows:
[0076]
[0077] In the formula, Indicates band node and band nodes Euclidean distance between them; and Representing the band nodes respectively and band nodes The standardized feature vector; and Representing the band nodes respectively and band nodes In the The normalized value at each pixel; Total number of pixels; For pixel index, ; This represents the Euclidean norm of a vector.
[0078] Sort the distances from the current band node to the other band nodes in ascending order, and take the nth distance after sorting. The distance value of the position is used as the local scale parameter of the node in that band. .
[0079] Step S13: Calculate adaptive connection weights based on bidirectional nearest neighbor union.
[0080] Based on the above distance sorting results, determine the first... Each band node Nearest neighbor set To ensure the symmetry of the graph structure and avoid information loss caused by unidirectional nearest neighbors, a bidirectional nearest neighbor union strategy is used to construct the graph edges, i.e., if... or Then at the band node and band nodes Establish connections between them. Based on this, calculate the band nodes. and band nodes Adaptive connection weights between them:
[0081]
[0082] In the formula, For band nodes and band nodes The connection weight values between them; For band nodes and band nodes The Euclidean distance between them; For band nodes Local scale parameters; For band nodes Local scale parameters; For band nodes Centered nearest neighbor set The parameter is the number of nearest neighbors; For band nodes Centered Nearest neighbor set; This is a preset minimum constant used to prevent the denominator from being zero; It is an exponential function with the natural constant e as its base; and For band node indexing, , This represents the total number of bands.
[0083] For band node pairs that do not meet the above conditions, their connection weight values =0.
[0084] Step S14: Construct a symmetric weighted adaptive band similarity graph based on graph nodes, node features, and adjacency matrix. , ,in For the set of band features, Let be the set of edges. For a weighted adjacency matrix, .
[0085] Step S2: Construct a mask graph autoencoder based on a cross-layer interaction mechanism. The real edges of the masked part are used as the self-supervised optimization objective, and a cross-layer feature fusion mechanism is introduced in the mask edge reconstruction task to extract deep local topological structure information between bands.
[0086] After constructing the adaptive band similarity map, this embodiment constructs a mask map autoencoder based on a cross-layer interactive decoder. By introducing a cross-layer feature fusion mechanism in the mask edge reconstruction task, it completes the learning of local topological relationships in the band map in a self-supervised manner, specifically including the following steps:
[0087] Step S21, from the set of real edges A portion of the samples is randomly sampled as the hidden positive sample edge set. Remove it to obtain the set of retained edges. With the corresponding adjacency matrix The input image after forming the mask .
[0088] Step S22, mask the input image Input multi-layer graph convolutional encoder, the first The node features of the layer are updated as follows:
[0089]
[0090] In the formula, Indicates the first The feature matrix of all band nodes in the layer, with each row corresponding to a band node; Indicates the first Feature matrix of all band nodes in the layer; , is the matrix after adding a self-loop to the adjacent matrix after masking, where express An identity matrix of order 1; Representation matrix The degree matrix is diagonal matrix; Degree matrix The negative one-second power is used for symmetric normalization; Indicates the first Learnable weight matrix for layer graph convolution; Represents a non-linear activation function; This is the layer index of the graph convolutional encoder. , This represents the total number of layers in the graph convolutional encoder.
[0091] Considering that graph convolutional layers of different depths can capture topological information under different receptive fields, the node features output by each layer are uniformly mapped to the same embedding dimension:
[0092]
[0093] In the formula, Indicates the first The layer outputs the projected node embedding matrix. ; Indicates the first The linear projection matrix corresponding to the layer, , For the first Feature dimensions of layer output, To unify the embedding dimensions, a multi-layered node embedding set is ultimately obtained. , .
[0094] Step S23: Randomly sample a portion of the node pairs in the original graph that have no topological connections as a negative sample edge set. , with positive sample edge set Merge into candidate prediction edge set .
[0095] Step S24: Perform cross-layer feature element-wise interaction concatenation on the two endpoints of the candidate edge to obtain candidate edge features containing multiple layers of topological semantics. :
[0096]
[0097] In the formula, Indicate candidate edges The feature vectors are integrated with multi-layer topological semantic information; This indicates the concatenation of feature dimensions; For band nodes In the The embedding representation vector output by the layer graph convolutional encoder; For band nodes In the The embedding representation vector output by the layer graph convolutional encoder; This represents the total number of layers in the graph convolutional encoder. , For graph convolutional encoder layer index, ; For element-wise multiplication; and For band node indexing, , This represents the total number of bands.
[0098] This cross-layer interactive design can integrate the local and high-order structural information contained in convolutional layers of different depth graphs, thereby enhancing the ability to predict edge existence.
[0099] Step S25, select candidate edge features Input the data into a multilayer perceptron and pass it through a sigmoid mapping to predict the probability of an edge existing:
[0100]
[0101] In the formula, Indicate candidate edges The prediction probability in the original band map is close to 1 for positive sample edges and close to 0 for negative sample edges. Represents the mapping function of a multilayer perceptron; This indicates the Sigmoid activation function, which compresses the output to... Interval.
[0102] Based on the edge existence prediction results above, the masked edge reconstruction loss is defined using binary cross-entropy loss:
[0103]
[0104] In the formula, Indicates the loss during mask edge reconstruction; Represents the set of positive sample edges that are masked; Represents the negative sample edge set; and These represent the number of samples in the positive and negative sample edge sets, respectively. Let represent the logarithmic function with base e. In this loss function, the first term is the positive sample edge reconstruction loss, used to encourage the model to recover masked true band connections; the second term is the negative sample discrimination loss, used to suppress the model from misclassifying random node pairs not present in the original graph as true edges. This is achieved by minimizing... The encoder needs to learn band node representations that can support edge existence judgment under the condition that some real edges are hidden, thereby enhancing the model's ability to model local topological relationships of bands.
[0105] Step S3: By introducing graph topology prior information into the Transformer multi-head self-attention mechanism, a graph-aware Transformer reconstruction network is constructed. With the help of the self-supervised reconstruction task of random masked bands, long-distance global contextual dependency information between bands is extracted.
[0106] This step aims to incorporate topological priors into the Transformer self-attention mechanism to model long-range dependencies across bands.
[0107] Specifically, step S3 includes the following steps:
[0108] Step S31: Embed the band output by the image encoder. As input, a portion of the bands is randomly masked and learned vectors are used. Replace to obtain the input sequence , ,in, Indicates band node The embedded representation after masking replacement, if the node of this band is masked, then = ,otherwise = , For band nodes The original embedding, taken from the band embedding .
[0109] Step S32: To address the limitation that the standard Transformer cannot directly utilize graph structure information, a graph structure attention bias matrix is constructed based on the adjacency matrix and injected into the self-attention calculation process of each layer of the graph-aware Transformer.
[0110] Specifically, by adding a self-loop to the adjacency matrix, we obtain:
[0111]
[0112] In the formula, This represents the adjacency matrix after adding self-loops; Represents the weighted adjacency matrix of the adaptive band similarity graph; express An identity matrix of order 1.
[0113] Next, define the corresponding degree matrix:
[0114]
[0115] In the formula, Representing the adjacency matrix The degree matrix is diagonal matrix; Describes the constructor for a diagonal matrix; Representing the adjacency matrix The Middle Line number Column elements, For row index, , For column indexes, .
[0116] Then the adjacency matrix was analyzed. After symmetric normalization, we get:
[0117]
[0118] In the formula, Represents the symmetric normalized adjacency matrix; Degree matrix The negative 1 / 2 power is used for symmetric normalization of rows and columns.
[0119] Based on this, define the graph structure attention bias matrix:
[0120]
[0121] In the formula, The attention bias matrix represents the graph structure. It is a logarithmic function with the natural constant e as its base; This is a preset minimum constant used to prevent the logarithm from tending to negative infinity.
[0122] Finally, the graph structure attention bias matrix is injected into the self-attention calculation process of each layer of the Transformer, then the th layer... The graph-aware attention operation of a layer is defined as follows:
[0123]
[0124] In the formula, Indicates the first Layer diagrams perceive the output of self-attention; , and They represent the first The query matrix, key matrix, and value matrix of the layer; This represents the feature dimension of the key vector; To inject weights into the graph bias that is related to the number of layers. For layer index, To perceive the total number of layers in the Transformer encoder; This represents the Softmax normalization function; This is the transpose operator. In this formula, as the graph-aware Transformer encoder layer deepens, the graph bias injection weight increases accordingly, thereby gradually increasing the injection strength of graph structure priors in the attention calculation process.
[0125] Step S33: Through residual connection, layer normalization and feedforward network layer-by-layer encoding, the final global band representation integrating global context and graph prior is obtained.
[0126] Specifically, let the first The input of the layer is represented as , The process by which this layer updates node features through graph-aware attention, residual connections, layer normalization, and feedforward networks is defined as follows:
[0127]
[0128]
[0129] In the formula, Indicates the first Layer diagram-aware input matrix of Transformer encoder; This represents the intermediate result after self-attention, residual, and layer normalization. Indicates the first Layer diagram-aware Transformer encoder output matrix; Indicates the layer normalization function; This represents the feedforward network function.
[0130] go through After encoding by the layered graph-aware Transformer encoder, the final global band representation, which integrates global context and graph prior, is obtained:
[0131]
[0132] In the formula, , indicating The global band representation matrix encoded by the layer-perceptual Transformer encoder. The feature dimension for band embedding; Indicates band node The representation is based on the fusion of global context information and prior knowledge of the graph structure. .
[0133] Step S34: Input the final global band representation into the decoder to reconstruct the masked band, and optimize the graph-aware Transformer with mean square error as the reconstruction loss.
[0134] Specifically, representing the global band Input to a multilayer perceptron decoder to reconstruct the original embedded representation of the masked bands. :
[0135]
[0136] In the formula, This represents the mapping function of the multilayer perceptron decoder.
[0137] Using the masked band reconstruction loss as the self-supervised optimization objective, the reconstruction loss of the graph-aware Transformer module is defined as follows:
[0138]
[0139] In the formula, Indicates the mask band reconstruction loss; The set of indices representing the masked bands; Indicates the number of masked bands; Indicates the decoder for the first Reconstructed embedding of a masked band; Indicates the first The original embedding of the masked band is used as the reconstruction target; This represents the Euclidean norm of a vector.
[0140] By minimizing the reconstruction loss, the graph-aware Transformer needs to recover the missing band representations using other visible bands and graph structure attention bias, even when some band embeddings are occluded, thereby learning the global dependencies between bands.
[0141] Step S4: Integrate local topology and global context information to generate a comprehensive score, and select the optimal band subset in descending order of the scores.
[0142] This step aims to integrate complementary information from both local and global perspectives to comprehensively evaluate and rank the importance of bands, and then select the top-ranked optimal subset of bands.
[0143] Specifically, the steps include:
[0144] Step S41: Based on graph structure information, the local structure contribution score of the band is obtained by utilizing the aggregation intensity of node neighborhood features. Since the distribution of spectral bands in the feature space is significantly non-uniform, bands in dense regions naturally have more neighboring nodes. To avoid the influence of non-uniform distribution, the adaptive adjacency matrix is first... Normalization is performed row-wise to eliminate the influence of distribution characteristics:
[0145]
[0146] In the formula, Band nodes in a band similarity graph and band nodes Normalized adjacency weights between them; Indicates band node and band nodes The original edge weights between them; Indicates band node and band nodes The original edge weights between them; Represented as the index variable in the summation process, ; This is a preset minimum constant used to prevent the denominator from being zero.
[0147] For band nodes Its neighborhood aggregation feature is represented as:
[0148]
[0149] In the formula, Indicates band node The neighborhood aggregation feature is represented; Indicates band node The embedding representation is taken from the band embedding. ; and For band node indexing, .
[0150] Characterizing band nodes by utilizing the aggregation intensity of node neighborhood features The local structural contribution score is:
[0151]
[0152] In the formula, Indicates band node Local structural contribution score; This represents the Euclidean norm of a vector.
[0153] This score characterizes the ability of a band node to aggregate neighborhood information within the local graph structure. A higher score indicates that the band can establish close topological relationships with key surrounding neighborhoods, thus making a more significant structural contribution to the transmission of local topological information.
[0154] Step S42: Using the attention weights learned by the graph-aware Transformer that reflect the global dependencies between bands, define the band nodes. The global contribution score is:
[0155]
[0156] In the formula, Indicates band node Global context contribution score; This represents the total number of layers in the graph-aware Transformer encoder; This represents the graph-aware Transformer encoder layer index. ; Indicates the first In layer self-attention, band nodes For band nodes Attention weights, representing band nodes Band nodes during global information aggregation The degree of dependence; and For band node indexing, .
[0157] This score reflects the degree to which a band is relied upon by other bands during global context modeling. A higher score indicates a stronger contribution of that band to global information.
[0158] Step S43: Normalize the local structure contribution score and the global context contribution score to eliminate the scale difference in the numerical distribution of the two types of scores caused by different modeling mechanisms.
[0159]
[0160]
[0161] In the formula, and Representing the band nodes respectively Normalized local structural contribution score and global context contribution score; and Representing the band nodes respectively Original local structural contribution score and global context contribution score; and These represent the minimum and maximum values of the local structure contribution scores for all bands, respectively. and These represent the minimum and maximum values of the global context contribution score for all bands, respectively. This is a preset minimum constant used to prevent the denominator from being zero.
[0162] Step S44: To comprehensively consider both local structural contributions and global context contributions, a weighted fusion scoring function for the bands is constructed:
[0163]
[0164] In the formula, Indicates band node A comprehensive evaluation and integrated scoring system; and The weight coefficients for the contributions of the local structure and the global context, respectively, satisfy the following conditions: .
[0165] Step S45, based on the fusion score All bands are sorted in descending order, and the top K bands with the highest scores are selected as the final subset of hyperspectral bands, where K is the preset number of selected bands.
[0166] To further verify the application effect of the present invention, the results obtained by the method proposed in this invention and other methods were used for pixel classification. To more intuitively compare the impact of various band selection methods on the accuracy of downstream classification tasks, Figure 2 The graph shows the classification accuracy of different band selection methods as a function of the number of selected bands when using an SVM classifier on the Pavia University dataset. The horizontal axis represents the number of selected bands, and the vertical axis represents the overall classification accuracy. Figure 2 As shown, for the Pavia University dataset, when selecting different numbers of bands, the present invention can achieve significantly higher classification accuracy than other band selection methods.
[0167] Figure 3 The graph shows the classification accuracy of different band selection methods as a function of the number of selected bands when using an SVM classifier on the Salinas dataset. The horizontal axis represents the number of selected bands, and the vertical axis represents the overall classification accuracy. Figure 3 The results show that the present invention can achieve significantly better classification performance than other band selection methods on the Salinas dataset.
[0168] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
Claims
1. A hyperspectral band selection method based on mask image sensing Transformer, characterized in that, Includes the following steps: Step S1: Construct an adaptive band similarity map based on local feature density to characterize the local topological relationships between bands and provide reliable structural prior information for subsequent feature learning. Step S2: Construct a mask graph autoencoder based on a cross-layer interaction mechanism. The real edges of the masked part are used and the existence prediction of the edges is used as the self-supervised optimization objective. At the same time, a cross-layer feature fusion mechanism is introduced in the mask edge reconstruction task to extract deep local topological structure information between bands. Step S3: By introducing graph topology prior information into the Transformer multi-head self-attention mechanism, a graph-aware Transformer reconstruction network is constructed. With the help of the self-supervised reconstruction task of random masked bands, long-distance global context dependency information between bands is extracted. Step S4: Integrate local topology and global context information to generate a comprehensive score, and select the optimal band subset in descending order of the scores; Step S1 includes the following steps: Step S11: Expand the original hyperspectral image into a pixel-band matrix and standardize it, using each band as a graph node and the corresponding standardized feature vector as the node feature. Step S12: Calculate the distances between band nodes, sort the distances from each band node to the other band nodes in ascending order, and take the distance of the sorted node. The bit distance value is used as its local scale parameter; Step S13: The bidirectional nearest neighbor union strategy is used to determine the connection relationship of the band nodes, and the connection weight is calculated by combining the band node spacing and local scale parameters to generate a symmetric weighted adjacency matrix. Step S14: Combine the graph nodes, node features, and adjacency matrix to form an adaptive band similarity graph; Step S4 includes the following steps: Step S41: After normalizing the adjacency matrix by row, the neighborhood feature differences of each band are weighted and aggregated to obtain the local structure contribution score. Step S42: Accumulate the graph to perceive the attention weights of other bands to the current band in each layer of the Transformer and obtain the global context contribution score; Step S43: Normalize the local structure contribution score and the global context contribution score respectively; Step S44: The two normalized scores are weighted and summed to obtain the fused score; Step S45: Sort all bands in descending order according to the fusion score, and select the top-ranked bands. One band is selected as the optimal hyperspectral band subset.
2. The hyperspectral band selection method based on mask image sensing Transformer according to claim 1, characterized in that, In step S13, the expression for calculating the connection weights is: In the formula, For band nodes and band nodes The connection weight values between them; For band nodes Centered Nearest neighbor set; For band nodes Centered nearest neighbor set The parameter is the number of nearest neighbors; For band nodes and band nodes The Euclidean distance between them; For band nodes Local scale parameters; For band nodes Local scale parameters; It is a constant; It is an exponential function with the natural constant e as its base; and For band node indexing, , This represents the total number of bands.
3. The hyperspectral band selection method based on mask image sensing Transformer according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Randomly sample some edges from the real edges of the adaptive band similarity graph as positive samples and remove them to obtain the masked input graph; Step S22: Input the masked input graph into the multi-layer graph convolutional encoder, update the node information layer by layer, and map the output of each layer to the same dimension to obtain a multi-layer node embedding set. Step S23: Randomly sample negative sample edges from node pairs without topological connections and merge them with positive sample edges to form a candidate predicted edge set; Step S24: Perform cross-layer feature element-by-element interactive splicing on the nodes at both ends of the candidate edge to generate candidate edge features that integrate multi-layer topological semantics; Step S25: Input the candidate edge features into the multilayer perceptron to predict the edge existence probability, and optimize the parameters of the graph autoencoder with binary cross-entropy loss so that the positive sample prediction approaches 1 and the negative sample prediction approaches 0.
4. The hyperspectral band selection method based on mask image sensing Transformer according to claim 3, characterized in that, In step S24, the expression for element-wise interleaving of cross-layer features is: In the formula, Indicate candidate edges eigenvectors; This indicates the concatenation of feature dimensions; For band nodes In the The embedded representation output by the layer graph convolutional encoder; For band nodes In the The embedded representation output by the layer graph convolutional encoder; This represents the total number of layers in the graph convolutional encoder. , For graph convolutional encoder layer index, ; For element-wise multiplication; and For band node indexing, , This represents the total number of bands.
5. The hyperspectral band selection method based on mask image sensing Transformer according to claim 4, characterized in that, Step S3 includes the following steps: Step S31: Using the band embedding output by the graph convolutional encoder as input, randomly mask some bands and replace them with learnable vectors to form the input sequence; Step S32: Construct a graph structure attention bias matrix based on the adjacency matrix and inject it into the self-attention calculation process of each layer of the graph-aware Transformer; Step S33: Through residual connection, layer normalization and feedforward network layer-by-layer encoding, the final global band representation that integrates global context and graph prior is obtained; Step S34: Input the final global band representation into the decoder to reconstruct the masked band, and optimize the graph-aware Transformer with mean square error as the reconstruction loss.
6. The hyperspectral band selection method based on mask image sensing Transformer according to claim 5, characterized in that, In step S32, the graph structure attention bias matrix is injected into the self-attention calculation process of each layer of the graph-aware Transformer. The graph-aware attention operation is defined as follows: In the formula, Indicates the first Layer diagrams perceive the output of self-attention; , and They represent the first The query matrix, key matrix, and value matrix of the layer; This represents the feature dimension of the key vector; To inject weights into the graph bias that is related to the number of layers. For layer index, To perceive the total number of layers in the Transformer encoder; This represents the Softmax normalization function; It is the transpose operator; The attention bias matrix represents the graph structure.
7. The hyperspectral band selection method based on mask image sensing Transformer according to claim 1, characterized in that, In step S41, the local structure contribution score is obtained as follows: For the For each band node, its neighborhood aggregation feature is defined as follows: In the formula, Indicates band node The neighborhood aggregation feature is represented; Band nodes in the band similarity graph and band nodes Normalized adjacency weights between them; Indicates band node Embedded representation; Indicates band node and band nodes The original edge weights between them; Indicates band node and band nodes The original edge weights between them; Indicates the total number of bands; and For band node indexing, ; Represented as the index variable in the summation process, ; It is a constant; Definition of the first Local structural contribution score for each band node: In the formula, Indicates band node Local structural contribution score; This represents the Euclidean norm of a vector.
8. The hyperspectral band selection method based on mask image sensing Transformer according to claim 1, characterized in that, In step S42, the global context contribution score is obtained as follows: In the formula, Indicates band node Global context contribution score; This represents the total number of layers in the graph-aware Transformer encoder; This represents the graph-aware Transformer encoder layer index. ; Indicates the first In layer self-attention, band nodes For band nodes Attention weights; and For band node indexing, , This indicates the total number of bands.
Citation Information
Patent Citations
Hyperspectral image classification method for feature extraction of unsupervised double-mask graph auto-encoder
CN119810657A
Hyperspectral remote sensing image classification method based on self-attention context network
WO2022073452A1
