Hyperspectral image classification method for feature extraction of unsupervised double-mask graph auto-encoder

Abstract

The invention discloses a hyperspectral image classification method for feature extraction of an unsupervised double-mask graph auto-encoder. The method comprises the following steps: step 1, obtaining a hyperspectral remote sensing image; 2, performing data preprocessing on the hyperspectral remote sensing image to obtain an initial image constructed based on space-spectral information; step 3, node branches and edge branches are derived from the initial graph, a double-branch graph mask-reconstruction structure is constructed by means of a graph mask auto-encoder to perform feature processing, and an encoder of the graph mask auto-encoder is used for extracting local and global information features of a graph structure based on a multi-level GCN-Transform parallel encoder structure of jump connection; step 4, converting the features of the graph structure extracted by the last layer of the encoder in the graph mask auto-encoder into pixel features, and using the pixel features to supervise classification of the classification model; the method has the advantages of high precision and high efficiency.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image processing, and in particular relates to a hyperspectral image classification method for unsupervised dual-mask image autoencoder feature extraction. Background Art

[0002] Hyperspectral remote sensing image processing plays an important role in material information detection and is an important topic in the field of remote sensing. Hyperspectral remote sensing images can provide continuous radiation spectral bands, carry rich ground information, and can be used to process different application fields, such as house change detection, crop assessment, geological and mineral resource surveys, etc. (He, Li et al., 2018; Ahmad; Shabbir et al., 2022). In the field of hyperspectral remote sensing image processing, classification is one of the main tasks. The so-called hyperspectral image classification is to determine the category of each hyperspectral image pixel. The existing classification methods are mainly based on the research of convolutional neural network methods, that is, using convolutional neural network models for end-to-end image classification.

[0003] As a deep learning model (Liang, Li, 2016; Zhang, Zou et al. 2022), graph convolutional neural network has been successfully applied to hyperspectral remote sensing image classification. The graph convolutional neural network-based method can process high-dimensional data and quickly aggregate surrounding node information, and can quickly extract deep abstract information.

[0004] The problem with existing hyperspectral image classification methods is that a large number of prior training samples are required to achieve relatively good results. In the application of hyperspectral image classification, a lot of human resources are consumed, affecting the classification efficiency. At the same time, the existing graph neural network structure can only extract local information features, and still lacks the global construction of data. At the same time, nodes and edges are two important attributes of the graph structure, and existing methods find it difficult to extract node and link information from both at the same time.

[0005] This article involves the following references

[0006] [1] He L, Li J, Liu C, Li S. Recent Advances on Spectral–SpatialHyperspectral Image Classification: An Overview and New Guidelines. IEEETrans Geosci Remote Sens. 2018 Mar;56(3):1579–97.

[0007] [2] Ahmad M, Shabbir S, Roy SK, Hong D, Wu X, Yao J, et al.Hyperspectral Image Classification-Traditional to Deep Models: A Survey forFuture Prospects. IEEE J Sel Top Appl Earth Observations Remote Sens. 2022Oct 15;15: 968–99.

[0008] [3] Ye J, He J, Peng X, Wu W, Qiao Y. Attention-Driven Dynamic GraphConvolutional Network for Multi-label Image Recognition. In: Vedaldi A,Bischof H, Brox T, Frahm J-M, editors. Computer Vision - ECCV 2020[Internet]. Cham: Springer International Publishing; 2020 [cited 2021 Apr26]. p. 649–65. (Lecture Notes in Computer Science; vol. 12366). Availablefrom: https: / / link.springer.com / 10.1007 / 978-3-030-58589-1_39

[0009] [4] Liang H, Li Q. Hyperspectral Imagery Classification Using SparseRepresentations of Convolutional Neural Network Features. Remote Sens. 2016Jan 27;8(2):99.

[0010] [5] Zhang H, Zou J, and Zhang L, EMS-GCN: An End-to-end MixhopSuperpixel-based Graph Convolutional Network for Hyperspectral ImageClassification. IEEE Trans. Geosci. Remote Sens. 2022 Mar 30;60: 1-16. Summary of the invention

[0011] The purpose of the present invention is to address the problems existing in the prior art and provide a hyperspectral image classification method based on unsupervised dual-mask graph autoencoder feature extraction, which utilizes a masked autoencoder to mask and reconstruct the edge and node features of the graph structure, and the encoder combines GCN and Transformer to capture local and global features in HSI, thereby achieving high-precision and high-efficiency hyperspectral image classification.

[0012] According to one aspect of the present invention, a hyperspectral image classification method for unsupervised dual mask image autoencoder feature extraction is provided, comprising the following steps:

[0013] Step 1, obtaining a hyperspectral remote sensing image;

[0014] Step 2, preprocessing the hyperspectral remote sensing image to obtain an initial image based on spatial-spectral information;

[0015] Step 3, deriving node branches and edge branches from the initial graph, constructing a dual-branch graph mask-reconstruction structure with the help of a graph mask autoencoder for feature processing, wherein the encoder of the graph mask autoencoder is used for extracting local and global information features of the graph structure based on a multi-stage GCN-Transformer parallel encoder structure with skip connections;

[0016] Step 4: Convert the features of the graph structure extracted by the last layer of the encoder in the graph mask autoencoder into pixel features, and use the pixel features for classification by the supervised classification model.

[0017] In the above scheme, the hyperspectral remote sensing image is first subjected to data preprocessing to obtain an initial graph; in step 3, the initial graph is derived into node branches and edge branches, and in the dual-branch graph mask-reconstruction structure, the dual-branch graph masking and reconstruction are performed with the help of the graph mask autoencoder and decoder; masking is used to simulate data uncertainty, and reconstruction is used to accurately extract image features, and a multi-level GCN-Transformer parallel encoder structure based on jump connections is designed in the graph mask autoencoder to realize local and global information feature extraction of the graph structure, thereby ensuring feature extraction efficiency and accuracy; in step 4, after several layers of mutual learning between encoders and decoders, the deep features extracted by the last layer of the encoder are converted into pixel features for classification by the supervised classification model.

[0018] Furthermore, step 3 includes the following steps:

[0019] Node shielding step: assign a random discard probability value of one to the nodes in the initial graph, assign a value of 0 to the selected nodes according to the discard probability value one, and shield them; the unselected nodes are regarded as visible nodes;

[0020] In the edge shielding step, a node subset is randomly selected from the initial graph. Starting from the node subset, a random walk of a certain length is performed. At each step, the selected edge is determined as a shielded edge or a visible edge according to the random discard probability value.

[0021] By setting the discard probability value 1 to assign values ​​to nodes, the effect of randomly shielding nodes is achieved. By selecting the root node and walking a random length, the shielding judgment of the edge at each step is made by discarding the probability value 2, thus achieving the effect of randomly shielding edges.

[0022] Furthermore, in the dual-branch graph mask-reconstruction structure, a node decoder is provided on the node branch to decode node information, and an edge decoder is provided on the edge branch to decode edge information.

[0023] In the dual-branch graph mask-reconstruction structure, information restoration is achieved by setting decoders on the node branch and the edge branch accordingly without interfering with each other.

[0024] Furthermore, the edge decoder adopts a two-layer multilayer perceptron. Through two layers of nonlinear transformation, the edge decoder learns the mapping from the low-dimensional feature space to the original edge features. The edge decoder is formally defined as:

[0025]

[0026] in , Represents a node.

[0027] Furthermore, the GCN decoder is used in the branch of the node to reconstruct the node. GCN is used to continuously aggregate the information of the node and its adjacent nodes to reduce the number of nodes. The formula is defined as:

[0028]

[0029] in, To reconstruct the node, is the compressed representation obtained from the decoder, is the trainable weight matrix.

[0030] Furthermore, in the branch of the edge, in the branch of the edge, the binary cross entropy loss is selected for edge reconstruction; at the same time, the mean square error loss is used to reconstruct the node; the total loss is expressed as the sum of the binary cross entropy loss and the mean square error loss.

[0031] Furthermore, step 4 includes the following steps:

[0032] The visible nodes and visible edges are input into the node and edge branches respectively to obtain their respective local information features and global information features. The features output by the two branches are then fused. Subsequently, the fused features are used as input features of the next layer of the network. Finally, the accurate extraction of deep features is achieved through the continuous superposition of M layers of encoders.

[0033] M is a hyperparameter that can be set manually. The optimal value is obtained through experimental results. The continuous superposition of M layers of encoders realizes unsupervised feature extraction.

[0034] Furthermore, after extracting features, the relationship matrix is ​​used in classification The node features to be extracted Converted to pixel features for supervised classification ,

[0035]

[0036] In the formula Represents the spatial dimension of the recovered data.

[0037] Furthermore, the data preprocessing in step 2 includes the following steps: after using the simple linear iterative clustering SLIC algorithm, the hyperspectral remote sensing image is clustered into multiple superpixels, where one superpixel represents one node, and an initial graph is constructed by establishing an adjacency matrix to determine whether there are edges between nodes;

[0038] During the Simple Linear Iterative Clustering (SLIC) algorithm, each node is assigned a unique number which serves as a location identifier.

[0039] By using the Transformer's multi-head attention mechanism to extract global information features of hyperspectral images, the local defects of graph convolutional neural networks can be compensated. At the same time, the graph mask autoencoder is used to reconstruct the masked graph attributes to achieve rapid information extraction. Starting from the two attributes of nodes and edges, we construct a dual-branch graph mask autoencoder structure and mine graph information. Therefore, the method of the present invention introduces an effective method based on unsupervised dual-mask graph autoencoders. In view of the problems of existing methods, parallel GCN and Transformer are used as encoders of graph mask autoencoders to extract deep semantic features from local and global perspectives. At the same time, in terms of masking strategy, we adopt a dual-branch approach to simultaneously establish mask-reconstruction for graph nodes and edges.

[0040] According to one aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer executes the steps of the above-mentioned unsupervised dual mask image autoencoder feature extraction method for hyperspectral image classification.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. We innovatively proposed a deep learning framework based on an unsupervised dual-mask image autoencoder, which realizes unsupervised hyperspectral image classification for the first time, alleviating the model's dependence on labeled samples and improving model training and classification efficiency;

[0043] 2. A dual-path encoding-masking-reconstruction strategy is designed. Compared with the traditional single-path graph feature reconstruction method, this design focuses on the entire graph structure, greatly enhancing the robustness of the algorithm and enabling it to adapt to various complex graph structures;

[0044] 3. We designed a multi-layer skip-connected parallel encoder. Our encoder uniquely combines Transformer and GCN, aiming to capture the global and local information features of HSI, thereby effectively extracting deep semantic information at a higher masking ratio, and performing dual-branch data feature mining based on the edge and node features of the image to improve feature extraction and recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a specific schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be noted that the terms front, back, left, right, etc. indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, or the directions or positional relationships in which the product is usually placed when in use. They are only for the convenience of describing the present invention or simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0047] like Figure 1 As shown, according to one aspect of the present invention, a hyperspectral image classification method for unsupervised dual mask image autoencoder feature extraction is provided, comprising the following steps:

[0048] Step 1, obtaining a hyperspectral remote sensing image;

[0049] Step 2, preprocessing the hyperspectral remote sensing image to obtain an initial image based on spatial-spectral information;

[0050] Step 3, deriving node branches and edge branches from the initial graph, constructing a dual-branch graph mask-reconstruction structure with the help of a graph mask autoencoder for feature processing, wherein the encoder of the graph mask autoencoder is used for extracting local and global information features of the graph structure based on a multi-stage GCN-Transformer parallel encoder structure with skip connections;

[0051] Step 4: Convert the features of the graph structure extracted by the last layer of the encoder in the graph mask autoencoder into pixel features, and use the pixel features for classification by the supervised classification model.

[0052] In the above scheme, the hyperspectral remote sensing image is first subjected to data preprocessing to obtain an initial graph; in step 3, the initial graph is derived into node branches and edge branches, and in the dual-branch graph mask-reconstruction structure, the dual-branch graph masking and reconstruction are performed with the help of the graph mask autoencoder and decoder; masking is used to simulate data uncertainty, and reconstruction is used to accurately extract image features, and a multi-level GCN-Transformer parallel encoder structure based on jump connections is designed in the graph mask autoencoder to realize the local and global information feature extraction of the graph structure; in step 4, after several layers of mutual learning between encoders and decoders, the deep features extracted by the last layer of the encoder are converted into pixel features for supervised classification to ensure feature extraction efficiency and accuracy.

[0053] In step 4, the epoch is set to alternately reduce the loss.

[0054] Furthermore, step 3 includes the following steps:

[0055] Node shielding step: assign a random discard probability value of one to the nodes in the initial graph, assign a value of 0 to the selected nodes according to the discard probability value one, and shield them; the unselected nodes are regarded as visible nodes;

[0056] In the edge shielding step, a node subset is randomly selected from the initial graph. Starting from the node subset, a random walk of a certain length is performed. At each step, the selected edge is determined as a shielded edge or a visible edge according to the random discard probability value.

[0057] By setting the discard probability value 1 to assign values ​​to nodes, the effect of randomly shielding nodes is achieved. The unselected nodes retain their original features as visible nodes. Using the node subset as the root node of the random walk, a random length walk is performed from the root node, and the shielding judgment of the edge at each step is performed by discarding the probability value 2, thereby achieving the effect of randomly shielding edges.

[0058] Furthermore, in the dual-branch graph mask-reconstruction structure, a node decoder is provided on the node branch to decode node information, and an edge decoder is provided on the edge branch to decode edge information.

[0059] In the dual-branch graph mask-reconstruction structure, information restoration is achieved by setting decoders on the node branch and the edge branch accordingly without interfering with each other.

[0060] Furthermore, the edge decoder adopts a two-layer multilayer perceptron. Through two layers of nonlinear transformation, the edge decoder learns the mapping from the low-dimensional feature space to the original edge features. The edge decoder is formally defined as:

[0061]

[0062] in , Represents a node.

[0063] Step 3 is as follows: In the experimental setting of the double-mask graph autoencoder, we first use a high-ratio mask to simulate the uncertainty in the graph data. Specifically, we assign a random discard probability value of one to the nodes in the initial graph. According to this value, the selected nodes are assigned a value of 0 to achieve the purpose of shielding the nodes. We call these nodes shielded nodes. The unselected nodes retain their original characteristics. We call these nodes visible nodes. At the same time, in order to simulate potential missing or incorrect connections in the graph structure, we also design a random discard probability value of two for the edges in the graph. First, we randomly extract a subset of nodes from the initial graph, which is called the root node as the starting point of the random walk. Then, starting from the root node, a random walk of a certain length is performed, and each step decides whether to discard the currently selected edge according to the discard probability value of two. The discarded edge is called the shielded edge, and the non-discarded edge is called the visible edge.

[0064] Then, we first feed the visible nodes and visible edges as input data to the encoder. The specific design and implementation details of the encoder will be elaborated in the following step 4. In the decoder part, we design a dual-branch decoder structure, which enables the decoder to focus on both node and edge information at the same time, thereby more comprehensively restoring the structure and properties of the original graph. Specifically, our edge decoder adopts a simple two-layer multi-layer perceptron (MLP).

[0065] Furthermore, the GCN decoder is used in the branch of the node to reconstruct the node. GCN is used to continuously aggregate the information of the node and its adjacent nodes to reduce the number of nodes. The formula is defined as:

[0066]

[0067] in, To reconstruct the node, is the compressed representation obtained from the decoder, is the trainable weight matrix.

[0068] Furthermore, in the branch of the edge, in the branch of the edge, the binary cross entropy loss is selected for edge reconstruction; at the same time, the mean square error loss is used to reconstruct the node; the total loss is expressed as the sum of the binary cross entropy loss and the mean square error loss.

[0069] The binary cross entropy loss is selected for edge reconstruction, which is defined as:

[0070]

[0071] In the formula is the visible edge, represents the reconstructed edge;

[0072] The formula for reconstructing nodes using mean square error loss is:

[0073]

[0074] in Represents visible nodes, represents the reconstruction node, so the total loss can be expressed as

[0075] .

[0076] Furthermore, step 4 includes the following steps:

[0077] The visible nodes and visible edges are input into the node and edge branches respectively to obtain their respective local information features and global information features. The features output by the two branches are then fused. Subsequently, the fused features are used as input features of the next layer of the network. Finally, the accurate extraction of deep features is achieved through the continuous superposition of M layers of encoders.

[0078] M is a hyperparameter that can be set manually. The optimal value is obtained through experimental results. The continuous superposition of M layers of encoders realizes unsupervised feature extraction.

[0079] In step 4:

[0080] Our encoder adopts an innovative architecture that combines the skip-parallel combination of GCN and Transformer, which enables the encoder to capture both local and global information features. In addition, the introduction of skip connections plays a crucial role in alleviating the potential over-smoothing problem caused by stacking multiple GCN layers. In addition, skip connections allow lower-level information to be directly transmitted to higher layers, effectively preserving the semantic information of the original data and preventing information loss and distortion during multi-layer propagation.

[0081] Specifically, we can input visible nodes and visible edges into two independent branches to obtain their respective local information features. and global information features Then the features output by these two branches are fused, and we mark the fusion result as Then, the fusion feature It will be used as the input feature of the next layer of the network. Finally, through the continuous superposition of M layers of encoders, the accurate extraction of deep features can be achieved.

[0082] The standard formula of GCN is

[0083]

[0084] in, represents the output embedding of the lth layer GCN. represents the trainable weight matrix of the lth layer. represents the degree matrix, Count the number of connections each node has with other nodes . Represents the identity matrix.

[0085] Furthermore, after extracting features, the relationship matrix is ​​used in classification The node features to be extracted Converted to pixel features for supervised classification ,

[0086]

[0087] In the formula Represents the spatial dimension of the recovered data.

[0088] Furthermore, the data preprocessing in step 2 includes the following steps: after using the simple linear iterative clustering SLIC algorithm, the hyperspectral remote sensing image is clustered into multiple superpixels, where one superpixel represents one node, and an initial graph is constructed by establishing an adjacency matrix to determine whether there are edges between nodes;

[0089] During the Simple Linear Iterative Clustering (SLIC) algorithm, each node is assigned a unique number which serves as a location identifier.

[0090] Our Transformer model does not require additional positional encoding because each node is assigned a unique number during the SLIC clustering process, which effectively serves as a position identifier. Therefore, the model can directly utilize these number assignments to capture the relative and absolute positional relationships between nodes. Our formula can be expressed as

[0091]

[0092] Which query ,key Sum To enter Linear conversion. , and are three predefined learnable weight matrices. express Dimension.

[0093] We apply different linear projections to transform different weight matrices , and To generate different , and . These attention calculation units generated by different projections are called "heads". Through the multi-head attention mechanism, we can learn and aggregate features from multiple different parameter spaces, thereby enhancing the representation ability of the model. This design allows the model to focus on different aspects at different positions in the input sequence and capture richer and more complex dependencies. The calculation method of multi-head attention is as follows:

[0094]

[0095]

[0096] in, is the number of bulls, is the parameter matrix.

[0097] The key invention of the present invention is to propose a hyperspectral image classification method based on an unsupervised dual-mask graph autoencoder, combining transformer and dual-path autoencoder to deal with the problem of no training samples. This method is based on the mask-reconstruction framework of the graph mask autoencoder, and realizes comprehensive mining of graph structure information by constructing a dual-path mask-reconstruction of the nodes and edges of the graph structure. At the same time, the encoder designs a parallel setting of GCN-Transformer based on jump connections, which can not only extract local information features of the image, but also obtain global information features through node connections, which helps to fully obtain the deep abstract information of the image.

[0098] This specific implementation is written in Python and implemented using the classic deep learning framework PYTORCH. The Python remote sensing image reading and writing functions are the basis for implementation. Call the remote sensing image processing library SPECTRAL, enter the remote sensing image file name to be read, and the remote sensing image is read into the file with a size of In the tensor of , each element in the tensor is the pixel radiation value corresponding to each band, where H is the length of the remote sensing image, is the width of the remote sensing image, and B is the number of remote sensing image bands. The remote sensing image reading and writing function is called to read the hyperspectral remote sensing image to be analyzed into the tensor X. The PYTHON remote sensing image reading and writing function is a well-known technology in the technical field and will not be described in detail here.

[0099] Hyperspectral image classification based on unsupervised dual mask image autoencoder, the specific operations are as follows:

[0100] (1) For the hyperspectral image X, since it consists of hundreds of very narrow continuous spectra, it has rich information but also contains a lot of redundant information. In addition, due to the atmosphere, sensors, environmental interference, human annotation, etc., the spectral information may not be accurate. Therefore, principal component analysis (PCA) is used here to reduce the dimension and denoise the image.

[0101] (2) Then, we introduce the Simple Linear Iterative Clustering (SLIC) algorithm, which is widely favored for its simplicity and ease of implementation. The algorithm can segment the image into a series of compact and uniform superpixels, successfully simplifying the amount of calculation and significantly improving the computational efficiency. Therefore, it is widely used in the field of HSI classification. After applying SLIC, the data is Clustering into multiple superpixels ,in Represents superpixel A superpixel Represents a node , by building an adjacency matrix To determine the node Is there an edge between To construct the initial graph .

[0102] Pixel With Node The relationship between express

[0103]

[0104] in, The value of the i-th row and j-th column represents the current pixel Belongs to this node Then, we divide the matrix Perform space-spectral analysis to obtain the adjacency matrix

[0105]

[0106] If the node and There is a strong correlation between .otherwise To construct the initial graph, we also need to transform the pixel features Convert to Node , in order to facilitate subsequent feature extraction. This is mainly done with the help of the relationship matrix To achieve this, the relation matrix is ​​another representation of the partition matrix M.

[0107]

[0108] if , it means that the pixel Belongs to this node .node It can be expressed as

[0109]

[0110] represents the transposed matrix of the relationship matrix. So far, we have successfully used the node and the adjacency matrix Construct an initial graph based on HSI .

[0111] (3) In the experimental setting of the dual-mask graph autoencoder, we first use high-ratio masks to simulate the uncertainty in the graph data. At the same time, in order to simulate the potential missing or wrong connections in the graph structure, we also design a random drop probability value of two for the edges in the graph.

[0112] (4) After unsupervised feature extraction, we first use the relationship matrix in classification The nodes to be extracted Convert to pixel features Finally, a certain number of target domain samples are selected and classified and evaluated for accuracy using a support vector machine (SVM).

[0113] The beneficial effects of the present invention are verified by comparative experiments as follows:

[0114] The data used in this experiment are Salinas and Trento, Italy, as experimental data sets to verify the effectiveness of the model. The Salinas data set was taken by the AVIRIS sensor in Salinas Valley, California. The data set size is 512×217, the spatial resolution is 3.7 meters, and it contains 224 continuous bands. After removing 20 water absorption bands (108-112, 154-167, and 224 bands), the actual number of bands used for training is 204. There are 16 types of land objects in the study area, including vegetables, bare soils, and vineyard fields. The Trento data set is taken from the rural area south of Trento, Italy. The study area consists of six different types of surface objects. There are six different types of surface objects in the study area. The data set size is 600×166, and the spatial resolution is 1m. The 63 spectral bands are distributed in the visible and near-infrared regions of 0.40 ~ 0.98 μm.

[0115] Classification evaluation index: A quantitative evaluation method is used. A total of 1% of the pixels in data sets 1 and 2 are selected as training samples, and the rest are used as test samples. The evaluation index uses the following two indicators:

[0116] 1) Kappa coefficient:

[0117] The kappa coefficient is an authoritative evaluation indicator used to evaluate classification problems. The larger the kappa coefficient, the higher the accuracy. The kappa coefficient calculation method is:

[0118] The confusion matrix is ​​obtained according to the samples, see Table 1.

[0119] Table 1 Confusion matrix

[0120]

[0121] In Table 1, TTO indicates the number of cases whose labels are category 1 and are predicted to be category 1; TF indicates the number of cases whose actual labels are category 2 but are predicted to be category 1. FT indicates the number of cases whose actual labels are category 1 but are predicted to be category 2. TTT indicates the number of cases whose labels are category 2 and are predicted to be category 2; NCO is the sum of TTO and FT, NCT is the sum of TF and TTT, NRO is the sum of TTO and TF, NRT is the sum of FT and TTT, and N is the total number of samples.

[0122] The calculation formula of the kappa coefficient is:

[0123]

[0124] 2) Overall accuracy:

[0125] Overall Accuracy (OA) is an evaluation index used to evaluate classification problems. The higher the overall accuracy, the higher the detection accuracy. The calculation method of OA is also based on the confusion matrix shown in Table 1. The calculation formula of overall accuracy OA is:

[0126]

[0127] The kappa coefficient and overall accuracy were used to evaluate the change detection capabilities of methods 1 to 5 and the method of the present invention. The evaluation indicators are shown in Table 2.

[0128] Table 2 Comparative test results

[0129]

[0130] As can be seen from Table 2, the method of the present invention can obtain higher overall accuracy and kappa value, indicating that the method of the present invention has stronger feature extraction and classification capabilities. The method of the present invention and the comparative method achieved the best results in terms of overall classification accuracy and kappa value, with a significant improvement.

[0131] It can be concluded that the method of the present invention has higher classification accuracy than traditional and recent deep learning unsupervised classification methods. The present invention makes full use of the unsupervised dual-image mask autoencoder for image classification, combines the GCN-Transformer form, fully considers the multiple representations of samples, comprehensively considers all levels of features, and improves classification accuracy.

[0132] Graph convolutional networks (GCNs) have an excellent ability to quickly mine semantic information from data and can perform deep feature extraction and classification of hyperspectral images (HSI). However, supervised classification methods heavily rely on many high-quality labeled samples, which is often a significant limitation in practical applications. Unsupervised learning, with its unique ability to automatically discover latent structures in data without labeled samples, provides a new approach to address this problem. Therefore, this paper proposes an unsupervised dual-mask graph autoencoder feature extraction network (UDMG) for HSI classification. The method first uses a masked autoencoder to mask and reconstruct the edge and node features of the graph structure. At the same time, the encoder combines GCN and Transformer to capture local and global features in HSI. To verify the performance of UDMG, we use multiple evaluation methods and conduct experiments on three publicly available hyperspectral datasets. The results show that UDMG has better classification performance than the existing state-of-the-art unsupervised feature extraction methods.

[0133] According to one aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer executes the steps of the above-mentioned unsupervised dual mask image autoencoder feature extraction method for hyperspectral image classification.

[0134] The methods described in the above embodiments may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer-readable media may include computer storage media and communication media, and may also include any medium that can transfer a computer program from one place to another. Storage media may be any target medium that can be accessed by a computer.

[0135] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A hyperspectral image classification method based on unsupervised dual mask image autoencoder feature extraction, characterized in that: The steps include: Step 1, obtaining a hyperspectral remote sensing image; Step 2, preprocessing the hyperspectral remote sensing image to obtain an initial image based on spatial-spectral information; Step 3, deriving node branches and edge branches from the initial graph, constructing a dual-branch graph mask-reconstruction structure with the help of a graph mask autoencoder for feature processing, wherein the encoder of the graph mask autoencoder is used for extracting local and global information features of the graph structure based on a multi-stage GCN-Transformer parallel encoder structure with skip connections; Step 4: Convert the features of the graph structure extracted by the last layer of the encoder in the graph mask autoencoder into pixel features, and use the pixel features for classification by the supervised classification model.

2. The hyperspectral image classification method based on unsupervised dual mask image autoencoder feature extraction according to claim 1 is characterized in that: Step 3 includes the following steps: Node shielding step: assign a random discard probability value of one to the nodes in the initial graph, assign a value of 0 to the selected nodes according to the discard probability value one, and shield them; the unselected nodes are regarded as visible nodes; In the edge shielding step, a node subset is randomly selected from the initial graph. Starting from the node subset, a random walk of a certain length is performed. At each step, the selected edge is determined as a shielded edge or a visible edge according to the random discard probability value.

3. The hyperspectral image classification method based on unsupervised dual mask image autoencoder feature extraction according to claim 1 is characterized in that: In the dual-branch graph mask-reconstruction structure, a node decoder is provided on the node branch to decode node information, and an edge decoder is provided on the edge branch to decode edge information.

4. The hyperspectral image classification method based on unsupervised dual mask image autoencoder feature extraction according to claim 3 is characterized in that: The edge decoder adopts a two-layer multilayer perceptron. Through two layers of nonlinear transformation, the edge decoder learns the mapping from the low-dimensional feature space to the original edge features. The edge decoder is formally defined as: in , Represents a node.

5. The hyperspectral image classification method based on unsupervised dual mask image autoencoder feature extraction according to claim 1 is characterized in that: The GCN decoder is used in the branch of the node to reconstruct the node. GCN is used to continuously aggregate the information of the node and its adjacent nodes to reduce the number of nodes. The formula is defined as: in, To reconstruct the node, is the compressed representation obtained from the decoder, is the trainable weight matrix.

6. The hyperspectral image classification method based on unsupervised dual mask image autoencoder feature extraction according to claim 1, characterized in that: In the branch of the edge, the binary cross entropy loss is selected for edge reconstruction; at the same time, the mean square error loss is used to reconstruct the node; the total loss is expressed as the sum of the binary cross entropy loss and the mean square error loss.

7. The hyperspectral image classification method based on unsupervised dual mask image autoencoder feature extraction according to claim 1 is characterized in that: Step 4 includes the following steps: The visible nodes and visible edges are input into the node and edge branches respectively to obtain their respective local information features and global information features. The features output by the two branches are then fused. Subsequently, the fused features are used as input features of the next layer of the network. Finally, the accurate extraction of deep features is achieved through the continuous superposition of M layers of encoders.

8. The hyperspectral image classification method based on unsupervised dual mask image autoencoder feature extraction according to claim 7 is characterized in that: After extracting features, the relationship matrix is ​​used in classification The node features to be extracted Converted to pixel features for supervised classification , In the formula Represents the spatial dimension of the recovered data.

9. The hyperspectral image classification method based on unsupervised dual mask image autoencoder feature extraction according to claim 1, characterized in that: The data preprocessing in step 2 includes the following steps: after using the simple linear iterative clustering SLIC algorithm, the hyperspectral remote sensing image is clustered into multiple superpixels, one superpixel represents one node, and the initial graph is constructed by establishing an adjacency matrix to determine whether there are edges between nodes; During the Simple Linear Iterative Clustering (SLIC) algorithm, each node is assigned a unique number which serves as a location identifier.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the computer executes the steps of the hyperspectral image classification method of unsupervised dual mask image autoencoder feature extraction as described in any one of claims 1 to 9.

Images