Hyperspectral image classification method based on element pseudo-label multi-scale image convolutional network

Abstract

The invention relates to a hyperspectral image classification method based on a meta-pseudo-label multi-scale image convolution network, and the method comprises the steps: building a meta-pseudo-label multi-scale feature search image convolution network (MPL-MFSGCN), dividing the HSI into superpixel images of different scales through employing an SLIC superpixel segmentation technology, and converting the superpixel images into image structures; secondly, constructing a multi-scale feature search graph convolutional network MFSGCN architecture, optimizing a superpixel feature weight combination by using a neural network search algorithm, and enhancing the discrimination of feature expression; a semi-supervised teacher-student model sharing the same graph convolutional network is adopted to generate meta-pseudo tags, and a student network is finely adjusted to improve the robustness of the network; the key features can be automatically captured under the small sample condition, the classification precision higher than that of an advanced method is obtained compared with a common HSI data set, and the classification accuracy is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image processing, and in particular relates to a hyperspectral image classification method based on a meta-pseudo-label multi-scale graph convolutional network. Background Art

[0002] Hyperspectral remote sensing technology obtains the spectral information of ground objects by obtaining the spectral reflectance of ground objects in different bands. Compared with traditional remote sensing technology, hyperspectral technology can provide more spectral band information, so it can more accurately describe the spectral characteristics of ground objects. Hyperspectral data usually consists of hundreds of continuous narrow bands, each band corresponds to a spectral reflectance value. This rich spectral information provides more feature dimensions for ground object classification, which helps to improve the accuracy of classification. By analyzing and processing hyperspectral data, ground objects can be accurately classified and identified.

[0003] Traditional hyperspectral classification methods are mainly based on manually designed feature extraction and classifiers. These methods usually select and extract some discriminative spectral features, such as statistical features of spectral reflectance, spectral index, etc., and then use traditional machine learning algorithms to perform classification tasks. However, these methods often rely on the experience of domain experts to select and design features, and cannot fully explore the potential information in hyperspectral data. Therefore, when dealing with complex land object classification tasks, traditional hyperspectral classification methods often face performance bottlenecks.

[0004] In recent years, with the rise of deep learning methods, researchers have begun to introduce deep learning into hyperspectral image classification, including stacked autoencoders, deep belief networks, convolutional neural networks (CNNs), generative adversarial networks, recurrent neural networks, and graph convolutional networks (GCNs). Deep learning technology has achieved remarkable results in hyperspectral image classification. Among them, GCN is based on graph structure and can capture the relationship and connection information in the data, providing a novel processing idea for hyperspectral data. Although GCN already has a certain adaptive feature extraction capability, the importance distribution of features at different scales is not balanced, resulting in the model being unable to fully utilize information at all scales. At the same time, the ability of the GCN model depends on effectively learning the complex structure and attribute information of the graph data, but in the case of scarce labels, it cannot get rid of the problem of relying on the expertise of domain experts, cannot effectively capture key features, and the available training samples are usually very limited.

[0005] Therefore, designing a hyperspectral image classification method based on meta-pseudo-label multi-scale graph convolutional network that can cope with the shortcomings of hyperspectral images and fully improve the ability to extract features of irregular objects has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] In order to solve the above problems, the present invention provides a hyperspectral image classification method based on a meta-pseudo-label multi-scale graph convolutional network.

[0007] The technical solution of the present invention is a hyperspectral image classification method based on a meta-pseudo-label multi-scale graph convolutional network, comprising the following steps: s1, using convolutional neural network CNN to extract pixel-level features of the original hyperspectral image; s2, divide the original hyperspectral image into unlabeled dataset and labeled dataset, and perform superpixel segmentation at four scales; s3, merge smaller superpixels to form larger superpixels, and obtain a multi-layer superpixel structure graph, which includes the graph feature structure at each scale and the hierarchical matrix H of the relationship between superpixels at different scales. Then, the superpixel points are used as nodes of the graph, and adjacent superpixels are connected as edges; s4, construct a multi-scale feature search graph convolutional network MFSGCN, and use the graph convolutional neural network GCN to fuse superpixel feature maps of different scales; s5. After obtaining the final fusion features through step s4, the meta-pseudo-label method is introduced for model training to establish a semi-supervised teacher-student model that shares the same graph convolutional network structure. The teacher network to be trained is pre-trained using labeled training data. The pre-trained teacher network generates prediction results based on unlabeled data to obtain corresponding pseudo-labels. The losses of pseudo-labels, labeled data, and unlabeled data in the student network are calculated. The network parameters are iteratively updated by minimizing the expected loss, and the teacher network parameters and the student network loss function are optimized to obtain a student network with the smallest loss and the highest classification accuracy. s6, calculate the verification loss and verification accuracy of the student network, fine-tune the student network based on the calculation results, and then repeat the calculation verification and fine-tune network operations until the calculation results meet the stopping threshold, obtain the best teacher-student model, and complete the multi-scale feature search graph convolutional network MPL-MFSGCN based on meta-pseudo labels; s7, use the trained network model for classification mapping, compare it with other commonly used classification models, and analyze the accuracy of the classification results.

[0008] As a further improvement of the present invention, the superpixel segmentation is based on the SLIC segmentation algorithm to perform superpixel segmentation on the hyperspectral image. The expression of the superpixel after segmentation is: ,in, represents the center of the i-th superpixel, represents the average value of pixels in the i-th superpixel.

[0009] As a further improvement of the present invention, the step of fusing superpixel feature maps includes: a) Extract pixel-level features, perform dot multiplication on the features after convolution of the original image and the relationship after superpixel segmentation, so as to combine the pixel features extracted by convolution with the spatial relationship of superpixels to obtain a more representative feature representation as the first input of the graph convolution network in the encoder; b) Multi-scale graph structure weight search: Use the neural structure search algorithm NAS to find the network structure with the best performance for a specific task, and use the gradient descent method to optimize the neural network structure parameters; c) Encoder feature weight search: The encoder is based on the computational expression of each layer of the graph convolutional network. The encoder is divided into two parts to capture the hierarchical information of the data. The first part performs preliminary encoding and feature learning. After three layers of graph convolution operations, the computational expression of the graph convolutional network is transformed, and the feature map of the first part is integrated into the second part. The feature maps with the same three dimensions are calculated, and finally the weight parameters are introduced into the feature map of the second part of the encoder. , , , and obtain the final deep Figure X 1 Expression of d) Decoder feature weight search. The decoder setting consists of two parts corresponding to the encoder. In the decoding structure of the first part, the final deep layer Figure X 1 As the input of the entire decoder, it is decomposed into a shallow graph after the graph convolution operation; in the second part, the nuances and framework of the input data are restored by transforming the feature map of the main part of the decoder. Finally, the feature map of the second part of the decoder introduces additional weight parameters , , After weighting, the shallow feature map is finally restored.

[0010] As a further improvement of the present invention, the expression of the more representative feature expression is: ,in, is the normalized transposed matrix of the first level matrix H after superpixel segmentation, W j is the convolution kernel of the jth layer, is the feature of the original image after convolution; the calculation expression of the graph convolution network is , where X represents the input feature map, is the normalized adjacency matrix, W is the weight matrix of the graph convolution layer of the structural linear transformation, B represents the bias term, represents the square root of the inverse of the logarithmic matrix, σ is the softmax activation function; the formula after the first part of the encoder is transformed as follows: , , in, represents the transformation results obtained by different layers in the first part of the encoder, Represents the output feature map of each layer; The expression of the feature map of the first part of the encoder is, , , , in, Represents the transformation results corresponding to each layer of the second part of the encoder, represents the feature map; the deep graph expression is ; The expression of the first part of the decoder is, , , in, The transformation results representing the association of the main part of the decoder with the different layers, Indicates that the connection operation is implemented on the main part of the decoder and the first part of the feature map of the encoder, ⊕ indicates connection; The expression of the second part of the decoder is, , , , in, It is the transformation result after the second part of the decoder establishes connections with different layers. It is the result of the concatenation operation between the feature map of the second part of the decoder and the feature map of the first value in the first part of the encoder; The expression of the shallow feature map is .

[0011] As a further improvement of the present invention, the teacher-student model generates a teacher network and a student network based on a multi-scale feature search graph convolutional network, and generates parameters corresponding to the two networks. and , and obtain the probability distribution expression of the sample categories obtained by the teacher network on the unlabeled data , the probability distribution of the categories predicted by the student network on labeled data and unlabeled data are and ,in, and Respectively represent the categories predicted for the corresponding labeled data and unlabeled data.

[0012] As a further improvement of the present invention, the pseudo label generation generates pseudo labels by inputting an unlabeled data set into a teacher network; the network parameter update is based on a cross entropy loss expression, , Where N is the total number of samples given, log represents the operation of natural logarithm, C is the number of categories, represents the label value of the jth category in the pseudo label of the i-th sample, Indicates that the student network is in the sample The predicted probability for category j above; The training loss of the student network under the pseudo labels generated by the teacher network is expressed as ; By minimizing the expected loss, the optimal student network parameters trained with pseudo labels are expressed as .

[0013] As a further improvement of the present invention, the network parameter update is based on the dependency relationship between the optimal parameters of the student network and the teacher network parameters, and the optimal student network parameters are converted into , and then introduce the meta-learning mechanism to The one-step gradient update is used to approximate the multi-step parameter optimization process, and the loss of the optimized student network is obtained. The relationship is expressed as, , , in, is the learning rate of the student network, the parameters of the student network Treated as a fixed value.

[0014] After adopting the above method, the hyperspectral image is segmented into superpixels through the SLIC segmentation algorithm, which has low time complexity and memory consumption. While maintaining image details, it can generate compact superpixel segmentation results with clear boundaries, which is helpful to merge smaller superimages to obtain multi-layer superpixel structure maps; pixel-level features are extracted through MFSGCN, and a representation with a pixel-based topological structure is obtained, and a connection relationship is established with subsequent superpixel-level graph features; the structural space of the neural network is searched through the NAS algorithm to find the network structure with the best performance for a specific task, and the gradient-based optimization strategy is used to find the optimized graph structure in the space to reduce training time and complexity; the encoder feature weight search reflects the role of the NAS algorithm in encoder feature extraction, and the optimal weights can be searched for the deepest level graph structure features, so that the deepest level features benefit from information from different scales; the decoder feature weight search restores the shallow feature map to assist model training and improve accuracy.

[0015] By establishing a teacher-student model based on the same graph convolutional network, the existing trained GCN model can predict unlabeled data and use these prediction results as pseudo labels. After the introduction of pseudo labels, the model can make more full use of unlabeled data for training in the absence of additional labeled data, thereby improving classification performance. By updating the network parameters and iteratively optimizing the teacher network parameters and the student network loss function, a student network with minimal loss and highest classification accuracy is obtained. By utilizing the self-learning characteristics of the model and the gradually accumulated pseudo-label information, the model can gradually improve its recognition ability in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Shown is a flow chart of the graph convolutional network classification method of the present invention.

[0017] Figure 2 Shown is a comparison chart of the classification results of the present invention based on the PU data set.

[0018] Figure 3 Shown is a comparison chart of the classification results of the present invention based on the SA data set.

[0019] Figure 4 Shown is a comparison chart of the classification results of the present invention based on the KSC data set.

[0020] Figure 5 Shown is a comparison chart of the overall accuracy of the present invention based on different training samples of the PU dataset.

[0021] Figure 6 Shown is a comparison chart of the overall accuracy of the present invention based on different training samples of the SA dataset.

[0022] Figure 7Shown is a comparison chart of the overall accuracy of the present invention based on different training samples of the KSC dataset.

[0023] Figure 8 Shown is a color comparison diagram of the classification results based on the PU data set of the present invention.

[0024] Fig. 9 Shown is a color comparison diagram of the classification results based on the SA data set of the present invention.

[0025] Fig.10 Shown is a color comparison diagram of the classification results of the present invention based on the KSC data set.

[0026] Fig.11 Shown is a color comparison chart of the overall accuracy of the present invention based on different training samples of the PU data set.

[0027] Fig.12 Shown is a color comparison chart of the overall accuracy of the present invention based on different training samples of the SA data set.

[0028] Fig.13 Shown is a color comparison chart of the overall accuracy of the present invention based on different training samples of the KSC dataset. DETAILED DESCRIPTION

[0029] like Figure 1-10 A hyperspectral image classification method based on a meta-pseudo-label multi-scale graph convolutional network is shown, comprising the following steps: s1, using convolutional neural network CNN to extract pixel-level features of the original hyperspectral image; s2, divide the original hyperspectral image into unlabeled dataset and labeled dataset, and perform superpixel segmentation at four scales; s3, merge smaller superpixels to form larger superpixels, and obtain a multi-layer superpixel structure graph, which includes the graph feature structure at each scale and the hierarchical matrix H of the relationship between superpixels at different scales. Then, the superpixel points are used as nodes of the graph, and adjacent superpixels are connected as edges; s4, construct a multi-scale feature search graph convolutional network MFSGCN, and use the graph convolutional neural network GCN to fuse superpixel feature maps of different scales; s5. After obtaining the final fusion features through step s4, the meta-pseudo-label method is introduced for model training to establish a semi-supervised teacher-student model that shares the same graph convolutional network structure. The teacher network to be trained is pre-trained using labeled training data. The pre-trained teacher network generates prediction results based on unlabeled data to obtain corresponding pseudo-labels. The losses of pseudo-labels, labeled data, and unlabeled data in the student network are calculated. The network parameters are iteratively updated by minimizing the expected loss, and the teacher network parameters and the student network loss function are optimized to obtain a student network with the smallest loss and the highest classification accuracy. s6, calculate the verification loss and verification accuracy of the student network, fine-tune the student network based on the calculation results, and then repeat the calculation verification and fine-tune network operations until the calculation results meet the stopping threshold, obtain the best teacher-student model, and complete the multi-scale feature search graph convolutional network MPL-MFSGCN based on meta-pseudo labels; s7, use the trained network model for classification mapping, compare it with other commonly used classification models, and analyze the accuracy of the classification results.

[0030] The superpixel segmentation is based on the SLIC segmentation algorithm to perform superpixel segmentation on the hyperspectral image. The expression of the superpixel after segmentation is: ,in, represents the center of the i-th superpixel, represents the average value of pixels in the i-th superpixel.

[0031] The step of fusing superpixel feature maps comprises: a) Extract pixel-level features, perform dot multiplication on the features after convolution of the original image and the relationship after superpixel segmentation, so as to combine the pixel features extracted by convolution with the spatial relationship of superpixels to obtain a more representative feature representation as the first input of the graph convolution network in the encoder; b) Multi-scale graph structure weight search: Use the neural structure search algorithm NAS to find the network structure with the best performance for a specific task, and use the gradient descent method to optimize the neural network structure parameters; c) Encoder feature weight search: The encoder is based on the computational expression of each layer of the graph convolutional network. The encoder is divided into two parts to capture the hierarchical information of the data. The first part performs preliminary encoding and feature learning. After three layers of graph convolution operations, the computational expression of the graph convolutional network is transformed, and the feature map of the first part is integrated into the second part. The feature maps with the same three dimensions are calculated, and finally the weight parameters are introduced into the feature map of the second part of the encoder. , , , and obtain the final deep Figure X 1 Expression of d) Decoder feature weight search. The decoder setting consists of two parts corresponding to the encoder. In the decoding structure of the first part, the final deep layer Figure X 1 As the input of the entire decoder, it is decomposed into a shallow graph after the graph convolution operation; in the second part, the nuances and framework of the input data are restored by transforming the feature map of the main part of the decoder. Finally, the feature map of the second part of the decoder introduces additional weight parameters , , After weighting, the shallow feature map is finally restored.

[0032] The expression of the more representative feature representation is ,in, is the normalized transposed matrix of the first level matrix H after superpixel segmentation, W j is the convolution kernel of the jth layer, is the feature of the original image after convolution; the calculation expression of the graph convolution network is , where X represents the input feature map, is the normalized adjacency matrix, W is the weight matrix of the graph convolution layer of the structural linear transformation, B represents the bias term, represents the square root of the inverse of the logarithmic matrix, σ is the softmax activation function; the formula after the first part of the encoder is transformed as follows: , , in, represents the transformation results obtained by different layers in the first part of the encoder, Represents the output feature map of each layer; The expression of the feature map of the first part of the encoder is, , , ,in, Represents the transformation results corresponding to each layer of the second part of the encoder, represents the feature map; the deep graph expression is ; The expression of the first part of the decoder is, , ,in, The transformation results representing the association of the main part of the decoder with the different layers, Indicates that the connection operation is implemented on the main part of the decoder and the first part of the feature map of the encoder, ⊕ indicates connection; The expression of the second part of the decoder is, , , ,in, It is the transformation result after the second part of the decoder establishes connections with different layers. It is the result of the concatenation operation between the feature map of the second part of the decoder and the feature map of the first value in the first part of the encoder; The expression of the shallow feature map is .

[0033] The teacher-student model generates a teacher network and a student network based on a multi-scale feature search graph convolutional network, and generates parameters corresponding to the two networks. and , and obtain the probability distribution expression of the sample categories obtained by the teacher network on the unlabeled data , the probability distribution of the categories predicted by the student network on labeled data and unlabeled data are and ,in, and Respectively represent the categories predicted for the corresponding labeled data and unlabeled data.

[0034] The pseudo label generation generates pseudo labels by inputting unlabeled data sets into the teacher network; the network parameter update is based on the cross entropy loss expression, , where N is the total number of samples given, log represents the operation of natural logarithm, and C is the number of categories. represents the label value of the jth category in the pseudo label of the i-th sample, Indicates that the student network is in the sample The predicted probability for category j above; The training loss of the student network under the pseudo labels generated by the teacher network is expressed as ; By minimizing the expected loss, the optimal student network parameters trained with pseudo labels are expressed as .

[0035] The network parameter update is based on the dependency relationship between the optimal parameters of the student network and the teacher network parameters. The optimal student network parameters are converted into , and then introduce the meta-learning mechanism to The one-step gradient update is used to approximate the multi-step parameter optimization process, and the loss of the optimized student network is obtained. The relationship is expressed as, , ,in, is the learning rate of the student network, the parameters of the student network Treated as a fixed value.

[0036] The SLIC segmentation algorithm is used to perform superpixel segmentation on hyperspectral images, which has low time complexity and memory consumption. While maintaining image details, it can generate compact superpixel segmentation results with clear boundaries, which is helpful to merge smaller superimages to obtain multi-layer superpixel structure maps; MFSGCN is used to extract pixel-level features, and a representation with a pixel-based topological structure is obtained, and a connection relationship is established with subsequent superpixel-level graph features; the NAS algorithm is used to search the structural space of the neural network to find the network structure with the best performance for a specific task, and a gradient-based optimization strategy is used to find the optimized graph structure in the space to reduce training time and complexity; the encoder feature weight search reflects the role of the NAS algorithm in encoder feature extraction, and the optimal weights can be searched for the deepest level graph structure features, so that the deepest level features benefit from information from different scales; the decoder feature weight search is used to restore the shallow feature map to assist model training and improve accuracy.

[0037] By establishing a teacher-student model based on the same graph convolutional network, the existing trained GCN model can predict unlabeled data and use these prediction results as pseudo labels. After the introduction of pseudo labels, the model can make more full use of unlabeled data for training in the absence of additional labeled data, thereby improving classification performance. By updating the network parameters and iteratively optimizing the teacher network parameters and the student network loss function, a student network with minimal loss and highest classification accuracy is obtained. By utilizing the self-learning characteristics of the model and the gradually accumulated pseudo-label information, the model can gradually improve its recognition ability in complex scenarios.

[0038] Table 1 Classification accuracy of PU dataset (%) category SS-CNN RSEN Oct-MCNN-HS A2S2K-ResNet CEGCN MSSGU MFSGCN MPL-MFSGCN 1 79.14±2.26 93.00±0.93 75.31±3.74 78.78±1.12 93.91±1.71 91.49±5.35 91.44±4.32 97.54±1.17 2 90.62±2.89 97.69±0.52 96.50±1.74 93.63±0.71 98.73±0.41 98.88±0.44 99.66±0.17 99.25±0.31 3 45.11±4.47 75.98±2.44 73.33±9.35 72.41±5.83 91.04±8.08 99.58±0.50 97.25±2.35 99.33±0.88 4 97.90±3.49 93.24±0.84 100.00±0.00 96.61±1.49 99.37±0.30 98.40±2.51 92.25±5.46 98.58±2.44 5 97.68±1.45 62.27±14.58 97.50±5.00 96.93±1.53 96.95±3.21 99.08±0.92 97.89±1.92 99.87±0.06 6 86.40±6.06 74.32±1.89 98.26±2.14 96.22±0.63 99.90±0.10 99.94±0.11 99.67±0.58 99.99±0.01 7 55.17±5.74 38.64±3.78 97.14±5.72 84.93±5.69 91.72±7.35 94.07±5.22 90.47±2.28 98.49±1.83 8 68.67±2.42 70.34±2.63 68.02±9.29 65.99±3.89 92.02±4.94 82.91±7.59 88.38±6.99 90.73±3.91 9 90.09±7.05 68.65±2.71 100.00±0.00 96.39±2.80 99.51±0.65 100.00±0.00 95.87±3.79 99.23±1.33 OA 84.13±2.11 86.88±0.88 89.53±1.81 88.23±0.67 96.81±0.84 96.01±1.11 96.07±0.96 98.21±0.47 AA 78.98±1.25 76.74±2.35 82.34±4.84 86.88±1.04 95.91±1.56 96.04±0.58 94.76±1.51 98.11±0.72 kappa 78.57±2.91 80.52±4.57 85.98±2.49 84.16±0.93 95.75±1.11 94.71±1.49 94.80±1.26 97.62±0.62 Table 2 Classification accuracy of SA dataset (%) category SS-CNN RSEN Oct-MCNN-HS A2S2K-ResNet CEGCN MSSGU MFSGCN MPL-MFSGCN 1 99.46±0.81 98.05±1.55 100.00±0.00 98.42±2.02 99.85±0.30 100.00±0.00 99.91±0.08 99.99±0.02 2 88.33±5.50 97.54±2.56 100.00±0.00 97.95±0.62 99.99±0.01 98.82±2.08 99.80±0.21 100.00±0.00 3 93.51±3.65 59.61±0.96 100.00±0.00 99.61±0.40 98.10±1.56 96.61±4.84 98.84±1.35 98.84±1.31 4 93.04±5.22 99.25±0.43 100.00±0.00 96.50±1.28 96.71±2.38 95.39±2.57 96.45±2.78 96.77±1.68 5 91.81±7.28 99.21±0.26 100.00±0.00 98.62±0.40 98.64±2.38 99.39±0.56 97.87±2.02 99.57±0.45 6 99.59±0.32 100.00±0.00 100.00±0.00 99.40±0.25 99.94±0.09 99.97±0.03 99.92±0.15 99.99±0.01 7 90.81±9.03 97.54±0.64 96.15±4.72 99.67±0.48 99.02±1.77 95.89±3.97 97.14±3.34 99.93±0.06 8 73.85±4.14 86.89±1.48 90.67±4.98 87.56±1.14 96.22±1.56 95.13±3.82 97.99±1.36 99.44±0.37 9 90.09±2.27 99.97±0.04 100.00±0.00 99.18±0.34 99.66±0.37 94.72±4.00 98.99±0.82 99.88±0.14 10 91.62±1.77 97.08±0.70 98.82±2.35 98.29±0.37 98.85±0.91 94.74±5.59 97.63±1.85 99.28±0.30 11 78.54±12.02 71.60±3.12 100.00±0.00 96.32±1.92 98.43±1.82 96.95±4.07 94.08±10.64 97.72±3.18 12 82.52±4.76 97.29±3.93 100.00±0.00 98.17±0.97 99.46±0.47 91.27±5.75 94.97±6.36 98.58±1.26 13 96.32±3.64 99.96±0.09 92.00±16.00 98.68±1.18 99.33±0.74 98.83±2.05 99.33±0.42 97.76±3.06 14 89.94±8.54 68.30±7.11 95.90±8.20 87.49±6.35 97.24±4.01 92.28±2.19 76.25±16.70 95.53±2.21 15 68.64±5.54 75.88±2.69 88.75±6.13 87.54±0.76 97.62±2.00 98.29±2.12 97.66±2.00 97.96±2.08 16 96.81±0.97 97.42±2.09 100.00±0.00 99.93±0.07 100.00±0.00 99.94±0.09 99.36±1.17 99.86±0.29 OA 84.59±1.18 90.55±0.66 95.08±0.63 94.64±0.25 98.33±0.17 96.44±1.16 97.41±0.79 99.15±0.28 AA 89.06±0.94 90.35±0.48 97.10±1.79 96.46±0.35 98.69±0.31 96.76±0.64 96.64±1.31 98.82±0.35 kappa 82.78±1.33 89.46±0.73 94.52±0.71 94.03±0.28 98.14±0.20 96.03±1.30 97.12±0.88 99.05±0.31 Table 3 Classification accuracy of KSC dataset (%) category SS-CNN RSEN Oct-MCNN-HS A2S2K-ResNet CEGCN MSSGU MFSGCN MPL-MFSGCN 1 35.20±14.34 65.99±6.12 39.89±26.53 90.00±16.22 99.14±1.03 99.78±0.45 85.53±13.59 91.80±10.84 2 13.32±4.98 21.08±0.85 19.14±9.38 92.30±5.14 96.51±2.68 85.96±15.20 79.74±21.56 96.27±4.68 3 18.65±5.75 4.72±2.71 24.49±14.12 71.27±8.75 71.73±15.84 71.39±17.45 74.45±19.87 96.72±4.36 4 21.69±8.72 2.40±1.43 9.28±4.97 35.52±14.49 84.29±10.03 74.51±20.82 75.81±11.58 89.69±6.51 5 10.43±3.63 36.48±3.92 21.42±13.03 54.75±20.23 84.80±14.54 74.03±11.31 61.99±30.69 81.89±22.30 6 11.65±4.71 64.49±3.80 9.68±6.01 54.93±15.46 77.45±10.41 52.82±7.08 62.88±18.27 89.75±15.66 7 11.35±8.58 11.46±5.08 23.03±19.52 58.44±4.77 72.06±14.70 83.08±20.73 84.31±19.59 63.20±18.42 8 13.07±9.91 40.14±5.79 35.80±14.94 76.62±9.09 76.88±19.16 87.19±14.36 73.06±9.69 75.60±8.70 9 38.11±12.53 49.80±6.65 22.89±17.50 91.24±7.47 91.54±11.45 95.44±5.42 93.99±7.33 94.81±10.26 10 25.66±9.34 27.03±2.38 15.74±8.95 96.78±1.44 93.94±6.04 86.15±8.24 92.01±9.58 89.76±9.00 11 68.97±29.52 15.48±1.49 68.39±25.27 97.53±4.10 96.49±4.41 93.90±5.80 89.94±9.56 98.78±0.90 12 26.01±5.63 35.15±5.02 32.18±7.00 76.84±13.21 88.64±17.13 86.21±16.46 93.33±5.88 80.45±12.71 13 59.05±9.35 69.11±2.62 62.35±9.86 99.07±0.56 95.49±4.27 90.11±10.00 85.67±6.92 97.14±2.84 OA 34.71±3.46 42.48±0.94 36.26±3.33 78.46±4.13 86.34±3.17 82.98±1.53 79.17±3.78 88.95±3.70 AA 27.16±3.46 34.10±0.70 30.26±3.65 76.56±2.79 86.84±2.07 83.12±1.71 80.98±3.24 88.56±4.03 kappa 27.79±3.17 36.49±0.98 29.49±3.69 76.17±4.47 84.83±3.50 81.11±1.73 76.87±4.10 87.73±4.12 As shown in Table 1-3, and Figure 2-13As shown in the figure, the hyperspectral data used for verification of this method are Pavia University, Salinas and Kennedy Space Center, corresponding to PU dataset, SA dataset and KSC dataset respectively; the area photographed in the PU dataset is located in Pavia, a city in northern Italy, and the features mainly include grass, trees, asphalt roads, etc., with a total of 9 feature categories; the area photographed in the SA dataset is located in the agricultural area of ​​California, USA, with a total of 16 feature categories; the area photographed in the KSC dataset is located in the Kennedy Space Center in Florida, USA, which is mainly a wetland ecosystem, including a variety of different waters and vegetation types, with large spectral differences, suitable for natural ecological research and water body monitoring, with a total of 13 feature categories. In order to evaluate the effectiveness under different training sample sizes, training samples were extracted from the three datasets at a ratio of 0.3%, 0.5%, 1%, 3% and 5% for testing.

[0039] Figure 2 , 8 The marks aj in the middle correspond to false color images, classification reference images, SS-CNN classification images, RSEN classification images, Oct-MCNN-HS classification images, A2S2K-ResNet classification images, CEGCN classification images, MSSGU classification images, MFSGCN classification images, and MPL-MFSGCN classification images. Compared with the reference images and other comparison methods, the MPL-MFSGCN classification images of the classification method of the present invention are more outstanding in detail processing. For gravel and shadow categories, other methods are difficult to distinguish correctly, and there is a problem of misclassifying gravel categories as asphalt. The MPL-MFSGCN algorithm can significantly improve the classification accuracy of these two categories. In the classification accuracy table of Table 1, the MPL-MFSGCN model shows the best performance in the three indicators of OA, AA and kappa, confirming its high classification accuracy and consistency. Compared with MFSGCN without the MPL mechanism, the model improves OA by about 2 percentage points, and the AA and Kappa coefficients are also improved by nearly 3 percentage points, indicating that MPL-MFSGCN has significant advantages in small sample scenarios. For single category accuracy, MPL-MFSGCN shows the highest accuracy in all four categories with a lower standard deviation, indicating good classification effect and stability in most categories.

[0040] Figure 3 , 9 The object marked aj corresponds to Figure 2 , 8Similarly, the classification results of the SA dataset are shown. Compared with the other six comparison objects, the MPL-MFSGCN method can clearly distinguish the uncultivated vineyard and grape categories that are easily confused in the upper left corner of the image, and reduce the discrete patch phenomenon; in addition, in terms of the difficulty in distinguishing the romaine lettuce-6 weeks and romaine lettuce-7 weeks with a small number of samples, MPL-MFSGCN can accurately identify various types of ground objects with different sample sizes; in the classification accuracy table of Table 2, MPL-MFSGCN achieved the best results in all three evaluation indicators, with low standard deviations of 0.28, 0.35 and 0.31 respectively, and its OA, AA and kappa values ​​of 99.15%, 98.82% and 99.05% respectively, showing strong robustness. Moreover, the classification accuracy of the MPL-MFSGCN method is very high in 16 categories, and the accuracy in 8 categories reaches more than 99%; compared with MFSGCN, in the difficult-to-identify 14th category, the MPL-MFSGCN with the introduction of the MPL mechanism improves the classification accuracy to at least 95%, an increase of 19.28 percentage points compared with MFSGCN.

[0041] Figure 4 , 10 The object marked aj corresponds to Figure 2 , 8 Similarly, the classification result diagram of the KSC dataset is shown. Due to the small number of samples and uneven distribution of objects in the KSC dataset, the visual classification effect of the first three comparison objects is poor, and the other GCN-based methods also have more misclassifications. In contrast, MPL-MFSGCN visually achieves more accurate classification of various objects. In the classification accuracy table in Table 3, the performance of the MPL-MFSGCN model in the three indicators of OA, AA, and kappa is 88.95%, 88.56%, and 87.73%, which are better than other models; compared with the CEGCN method with the second best accuracy, MPL-MFSGCN is about 2 percentage points higher in all three indicators; compared with the MFSGCN method without the MPL mechanism, MPL-MFSGCN is 9.78 percentage points higher in OA, 7.58 percentage points higher in AA, and 10.86 percentage points higher in kappa. In terms of category accuracy, the MPL-MFSGCN model achieved the highest classification accuracy in four categories, especially in the face of difficult-to-distinguish categories 3 and 6, MPL-MFSGCN achieved significant classification results, and the classification accuracy was much higher than that of the comparison objects. The MPL-MFSGCN method proposed in this invention combines multi-feature learning with graph convolutional networks, optimizes the learning process in a small sample environment, and thus shows significant superiority over traditional supervised and semi-supervised learning methods in hyperspectral image classification tasks.

[0042] exist Figure 5-7, 11-13, the bar graphs correspond to the bottom marks from top to bottom in each ratio extraction. For the PU dataset, SA dataset, and KSC dataset, MPL-MFSGCN shows the best performance in the training sample ratio range of 0.3% to 5%, verifying its strong robustness; especially when the training samples are very few, such as 0.3%, the OA of MPL-MFSGCN is significantly higher than other methods, showing its ability to effectively extract information, learn and make predictions for small sample data, and the effect is more significant in sparse data environments. As the training sample ratio increases to 5%, the performance of all methods has improved, but MPL-MFSGCN still maintains the leading classification accuracy, indicating that when there is more training data, MPL-MFSGCN has a good performance that is not inferior to other methods. MPL-MFSGCN has excellent processing and learning capabilities for different amounts of training data.

Claims

1. A hyperspectral image classification method based on a meta-pseudo-label multi-scale graph convolutional network, characterized by: The following steps are included: s1, using convolutional neural network CNN to extract pixel-level features of the original hyperspectral image; s2, divide the original hyperspectral image into unlabeled dataset and labeled dataset, and perform superpixel segmentation at four scales; s3, merge smaller superpixels to form larger superpixels, and obtain a multi-layer superpixel structure graph, which includes the graph feature structure at each scale and the hierarchical matrix H of the relationship between superpixels at different scales. Then, the superpixel points are used as nodes of the graph, and adjacent superpixels are connected as edges; s4, construct a multi-scale feature search graph convolutional network MFSGCN, and use the graph convolutional neural network GCN to fuse superpixel feature maps of different scales; s5. After obtaining the final fusion features through step s4, the meta-pseudo-label method is introduced for model training to establish a semi-supervised teacher-student model that shares the same graph convolutional network structure. The teacher network to be trained is pre-trained using labeled training data. The pre-trained teacher network generates prediction results based on unlabeled data to obtain corresponding pseudo-labels. The losses of pseudo-labels, labeled data, and unlabeled data in the student network are calculated. The network parameters are iteratively updated by minimizing the expected loss, and the teacher network parameters and the student network loss function are optimized to obtain a student network with the smallest loss and the highest classification accuracy. s6, calculate the verification loss and verification accuracy of the student network, fine-tune the student network based on the calculation results, and then repeat the calculation verification and fine-tune network operations until the calculation results meet the stopping threshold, obtain the best teacher-student model, and complete the multi-scale feature search graph convolutional network MPL-MFSGCN based on meta-pseudo labels; s7, use the trained network model for classification mapping, compare it with other commonly used classification models, and analyze the accuracy of the classification results.

2. The hyperspectral image classification method based on meta-pseudo-label multi-scale graph convolutional network according to claim 1, characterized in that: The superpixel segmentation is based on the SLIC segmentation algorithm to perform superpixel segmentation on the hyperspectral image. The expression of the superpixel after segmentation is: ,in, represents the center of the i-th superpixel, represents the average value of pixels in the i-th superpixel.

3. The hyperspectral image classification method based on meta-pseudo-label multi-scale graph convolutional network according to claim 1, characterized in that: The step of fusing superpixel feature maps comprises: a) Extract pixel-level features, perform dot multiplication on the features after convolution of the original image and the relationship after superpixel segmentation, so as to combine the pixel features extracted by convolution with the spatial relationship of superpixels to obtain a more representative feature representation as the first input of the graph convolution network in the encoder; b) Multi-scale graph structure weight search: Use the neural structure search algorithm NAS to find the network structure with the best performance for a specific task, and use the gradient descent method to optimize the neural network structure parameters; c) Encoder feature weight search: The encoder is based on the computational expression of each layer of the graph convolutional network. The encoder is divided into two parts to capture the hierarchical information of the data. The first part performs preliminary encoding and feature learning. After three layers of graph convolution operations, the computational expression of the graph convolutional network is transformed, and the feature map of the first part is integrated into the second part. The feature maps with the same three dimensions are calculated, and finally the weight parameters are introduced into the feature map of the second part of the encoder. , , , and get the final deep graph X 1 Expression of d) Decoder feature weight search, the decoder setting consists of two parts corresponding to the encoder. In the decoding structure of the first part, the final deep graph X 1 As the input of the entire decoder, it is decomposed into a shallow graph after the graph convolution operation; in the second part, the nuances and framework of the input data are restored by transforming the feature map of the main part of the decoder. Finally, the feature map of the second part of the decoder introduces additional weight parameters , , After weighting, the shallow feature map is finally restored.

4. The hyperspectral image classification method based on meta-pseudo-label multi-scale graph convolutional network according to claim 3 is characterized by: The expression of the more representative feature representation is ,in, is the normalized transposed matrix of the first level matrix H after superpixel segmentation, W j is the convolution kernel of the jth layer, is the feature of the original image after convolution; the calculation expression of the graph convolution network is , where X represents the input feature map, is the normalized adjacency matrix, W is the weight matrix of the graph convolution layer of the structural linear transformation, B represents the bias term, represents the square root of the inverse of the logarithmic matrix, σ is the softmax activation function; the formula after the first part of the encoder is transformed as follows: , , in, represents the transformation results obtained by different layers in the first part of the encoder, Represents the output feature map of each layer; The expression of the feature map of the first part of the encoder is, , , , in, Represents the transformation results corresponding to each layer of the second part of the encoder, represents the feature map; the deep graph expression is ; The expression of the first part of the decoder is, , , in, The transformation results representing the association of the main part of the decoder with the different layers, Indicates that the connection operation is implemented on the main part of the decoder and the first part of the feature map of the encoder, ⊕ indicates connection; The expression of the second part of the decoder is, , , , in, It is the transformation result after the second part of the decoder establishes connections with different layers. It is the result of the concatenation operation between the feature map of the second part of the decoder and the feature map of the first value in the first part of the encoder; The expression of the shallow feature map is .

5. The hyperspectral image classification method based on meta-pseudo-label multi-scale graph convolutional network according to claim 1, characterized in that: The teacher-student model generates a teacher network and a student network based on a multi-scale feature search graph convolutional network, and generates parameters corresponding to the two networks. and , and obtain the probability distribution expression of the sample categories obtained by the teacher network on the unlabeled data , the probability distribution of the categories predicted by the student network on labeled data and unlabeled data are and ,in, and Respectively represent the categories predicted for the corresponding labeled data and unlabeled data.

6. The hyperspectral image classification method based on meta-pseudo-label multi-scale graph convolutional network according to claim 5, characterized in that: The pseudo label generation generates pseudo labels by inputting an unlabeled dataset into a teacher network; The network parameter update is based on the cross entropy loss expression, , Where N is the total number of samples given, log represents the operation of natural logarithm, C is the number of categories, represents the label value of the jth category in the pseudo label of the i-th sample, Indicates that the student network is in the sample The predicted probability for category j above; The training loss of the student network under the pseudo labels generated by the teacher network is expressed as ; By minimizing the expected loss, the optimal student network parameters trained with pseudo labels are expressed as .

7. The hyperspectral image classification method based on meta-pseudo-label multi-scale graph convolutional network according to claim 6 is characterized by: The network parameter update is based on the dependency relationship between the optimal parameters of the student network and the teacher network parameters. The optimal student network parameters are converted into , and then introduce the meta-learning mechanism to The one-step gradient update is used to approximate the multi-step parameter optimization process, and the loss of the optimized student network is obtained. The relationship is expressed as, , , in, is the learning rate of the student network, the parameters of the student network Treated as a fixed value.

Images