A graph convolutional fusion network for hyperspectral image classification

Abstract

The present invention discloses a graph convolution fusion network for hyperspectral image classification, comprising: image data preprocessing, mainly comprising utilizing methods such as block partitioning, linear discriminant analysis and simple linear iterative clustering to divide segmentation and block data of different scales, and constructing a pixel-level graph structure based on the segmentation and block data; constructing a classification network, comprising a spectral conversion module, a block data graph convolution branch, a block data convolution branch, a segmentation data graph convolution branch, a block data graph convolution feature processing module, a segmentation data feature processing module and a feature fusion module; the block data graph convolution feature processing module improves pixel-level feature expression and enhances classification accuracy by combining a large convolution kernel convolution layer with neighborhood aggregation; the segmentation data feature processing module improves pixel-level feature expression and enhances classification accuracy by utilizing feature similarity weight aggregation; the feature fusion module learns the intrinsic connection between features through void convolution, improves feature fusion effect and enhances classification accuracy. The present invention designs a new convolution and graph convolution fusion neural network to perform hyperspectral image classification. First, block partitioning, linear discriminant analysis and simple linear iterative clustering are used to divide the segmentation and block data of different scales, and a pixel-level graph structure is constructed based on the segmentation and block data. Then, the spectral features of the segmentation and block data are extracted and their spectral dimensions are reduced through the spectral conversion module. Then, convolution and graph convolution are used to extract the spatial features of the segmentation and block data. Then, the block data graph convolution feature processing module is used to process the block data graph convolution features, and the segmentation data feature processing module is used to process the segmentation data features. Finally, the feature fusion module is used to adaptively fuse the features, and the fused features are finally classified.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral remote sensing image processing, and specifically to a graph convolutional fusion network for hyperspectral image classification. Background Art

[0002] Hyperspectral images (HSI) have seen significant applications in various remote sensing fields in recent years, including target detection, agricultural monitoring, and marine safety. Unlike conventional two-dimensional digital images, hyperspectral images are three-dimensional cube data, consisting of a two-dimensional digital image and a one-dimensional spectral dimension. Spectral bands contain rich geographic information, making feature selection and extraction particularly important for hyperspectral pixel classification. Commonly used classification methods for hyperspectral images include K-Nearest Neighbor (K-NN), Extreme Learning Machine (ELM), and Support Vector Machine (SVM). They all utilize spectral information for classification. K-NN is the simplest classifier in machine learning, while ELM is a novel fast learning algorithm. For a single-hidden-layer neural network, ELM can randomly initialize input weights and biases and obtain the corresponding output weights. Support Vector Machine (SVM) is a widely used machine learning algorithm that seeks to best approximate the model through training with small sample data, achieving an optimal balance between model complexity and learning ability. In addition, principal component analysis (PCA) can reduce noise interference by compressing spectral bands through frequency band selection, which can retain important features and is particularly suitable for hyperspectral image classification.

[0003] In recent years, image classification based on deep learning methods has become increasingly popular among researchers. The fundamental idea behind deep learning image classification is to extract abstract features from raw image data by densely connecting multiple layers of neurons, using simple linear equations and activation functions to obtain nonlinear features. Through training on large amounts of raw data, specific network weight parameters are formed, thereby improving image classification accuracy. Hyperspectral images have spectral dimensions of dozens or even hundreds of bands. Chen et al. first introduced the concept of deep learning to hyperspectral image classification, proposing a stacked autoencoder (SAE) model that extracts raw data information using autoencoders and finally classifies it using support vector machines (SVMs). However, this method only utilizes spectral information while neglecting the use of spatial information.

[0004] To further utilize the spatial features of hyperspectral images, a hyperspectral algorithm based on a convolutional neural network (CNN) has been proposed. CNNs are widely used in hyperspectral image classification due to their powerful ability to extract features from fixed-shape data blocks. While CNNs have been proven to be effective at extracting pixel-level features, the shared weights and fixed shape of the convolution kernel make them difficult to adapt to data with complex feature shapes.

[0005] Graph convolutional neural networks are also widely used in the field of hyperspectral image classification due to their powerful ability to extract graph structural features. However, most existing graph convolution methods use superpixel partitioning to reduce the number of nodes, which inevitably causes the loss of pixel-level features. At the same time, the aggregation of other node information will inevitably introduce erroneous information. Summary of the Invention

[0006] Purpose of the invention: In order to better integrate the advantages of convolutional neural networks and graph convolutional neural networks, and improve the ability of graph convolutional neural networks to aggregate neighborhood information, a graph convolutional fusion network for hyperspectral image classification is provided. A multi-branch hyperspectral image classification network is designed, which extracts spatial features with different characteristics by constructing local graph structures of different shapes and scales, and then improves the feature representation through feature processing modules designed for different characteristics. Finally, the feature fusion module is used to adaptively fuse features and classify them.

[0007] Technical solution: To achieve the above objectives, the present invention provides a graph convolutional fusion network for hyperspectral image classification, comprising the following steps:

[0008] S1: Standardize the hyperspectral image data (raw data) HSI to obtain standardized data

[0009] S2: Standardize the data The pixels to be classified and the pixels with a distance of no more than 2 from them are constructed into a 5×5 block data Normalize the data The pixels to be classified and the pixels with a distance of no more than 7 from them are constructed into a 15×15 block data.

[0010] S3: Standardized data Apply the linear discriminant analysis method to perform dimensionality reduction operation and obtain dimensionality reduction data

[0011] S4: Dimensionality reduction data Perform fine simple linear clustering to obtain a fine segmentation map, and perform coarse simple linear clustering to obtain a coarse segmentation map;

[0012] S5: Based on refined segmentation map and normalized data Get segmentation data Based on rough segmentation map and normalized data Get segmentation data

[0013] S6: construct a classification network, including S61 spectral conversion module, S62 block data graph convolution branch, S63 block data convolution branch, S64 segmentation data graph convolution branch, S65 block data graph convolution feature processing module, S66 segmentation data feature processing module and S67 feature fusion module;

[0014] S7: Data-based and Calculate the corresponding Laplace matrix and Laplacian matrix From the adjacency matrix A i Calculated, the adjacency matrix A i The calculation formula is:

[0015]

[0016] Among them, X i f is the pixel block Z i Reduce the spatial dimension to a one-dimensional two-dimensional matrix form, a and b are X i f The spatial coordinates of the midpoint.

[0017] Laplacian matrix The calculation formula is:

[0018]

[0019] Where I is the shape and A i The same identity matrix, D i A i degree matrix of ;

[0020] S8: Data and The labeled part and the corresponding Laplacian matrix and They are fed into the classification network as input data for training, and the network output results of the training set pixels are compared with their labels to calculate the cross entropy loss L. The Adam optimizer is used to backpropagate the network parameters to reduce the cross entropy loss L. The training is continued until the cross entropy loss L converges and the network parameters are saved.

[0021] The calculation formula of cross entropy loss L is:

[0022]

[0023] Among them, L i is the cross entropy loss of the i-th sample, N is the number of samples, M is the number of categories, and p ic is the predicted probability that the i-th sample belongs to the c-th category, y ic is the true label of the cth category of the i-th sample;

[0024] S9: Data and The unlabeled part and the corresponding Laplacian matrix and They are input into the classification network for classification prediction and the classification results of the entire hyperspectral image are output.

[0025] Furthermore, the classification network in step S6 specifically consists of a spectrum conversion module S61, two block data graph convolution branches S62, two block data convolution branches S63, two segmentation data graph convolution branches S64, two block data graph convolution feature processing modules S65, two segmentation data feature processing modules S66, and a feature fusion module S67. The specific process is as follows:

[0026] (1) Data and The data are then sent to the S61 spectrum conversion module to extract the spectral features and reduce the spectral dimension.

[0027] (2) Change the and The acquired data and the corresponding Laplace matrix and The data is fed into two different S62 block graph convolution branches and two S63 block data convolution branches in parallel to extract spatial features with different characteristics;

[0028] (3) The features extracted by the two S62 block data graph convolution branches in the previous step are respectively sent to the two S65 block data graph convolution feature processing modules to obtain the feature X 1g and X 2g ;

[0029] (4) The features extracted by the two S63 block data convolution branches in step (2) are average pooled to obtain the feature X 1c and X 2c ;

[0030] (5) Change the step (1) from and The acquired data and the corresponding Laplace matrix and The data is fed into two different S64 segmentation graph convolution branches in parallel to extract spatial features;

[0031] (6) The features extracted by the two S64 segmentation data graph convolution branches in the previous step are respectively sent to the two S66 segmentation data feature processing modules to obtain the feature X s1 and X s2 .

[0032] Furthermore, the spectrum conversion module S61 in step S6 is specifically composed of two submodules of the same structure connected in series, each submodule is composed of three network layers connected in series, and includes the following steps:

[0033] S611: 1×1 convolutional layer, extracting spectral features from the input data and reducing its spectral dimension;

[0034] S612: Batch normalization layer, which performs normalization on the extracted features. The specific formula is:

[0035]

[0036] Among them, T (l) is the input data of the batch normalization layer, E(·) represents the mean function, Var(·) represents the variance function, ε is a correction to prevent the denominator from being too small, γ and β are learnable parameters used to adjust the mean and variance so that the batch normalized data conforms to the original distribution as much as possible. is the output data of the batch normalization layer;

[0037] S613: Mish activation function, which performs nonlinear mapping on features. The specific formula is:

[0038] f(x)=x×tanh(ln(1+e x ))

[0039] Among them, x represents the pixel feature in the data, and f(x) is the output value of the activation function.

[0040] The specific formula of the S61 spectrum conversion module is:

[0041] X spe =Mish(BN(Mish(BN(XW sp1 ))W sp2 ))

[0042] Among them, X represents the input data of the classification network, X spe is the output data of the spectrum conversion module, W sp1 and W sp2 is the parameter matrix of the spectral conversion module, BN(·) represents the batch normalization operation, and Mish(·) represents the Mish activation function.

[0043] Furthermore, the data graph convolution branch of block S62 in step S6 is specifically composed of three submodules with similar structures connected in series, each submodule is composed of three network layers connected in series, including the following steps:

[0044] S621: graph convolution layer, extracts spatial features from input data;

[0045] S622: Batch normalization layer, which performs normalization on the extracted features;

[0046] S623: Mish activation function, which performs nonlinear mapping on features.

[0047] The specific formula of the submodule is:

[0048]

[0049] Represents the data input to the lth submodule, is the data output by the lth submodule, W (l) is the graph convolution parameter matrix in the lth submodule, is the Laplacian matrix corresponding to the input data.

[0050] Furthermore, the data convolution branch of block S63 in step S6 is specifically composed of three submodules with similar structures connected in series, each submodule is composed of three network layers connected in series, including the following steps:

[0051] S631: Convolutional layer, extracts spatial features from input data;

[0052] S632: Batch normalization layer, which performs normalization on the extracted features;

[0053] S633: Mish activation function, which performs nonlinear mapping on features.

[0054] The specific formula of the submodule is:

[0055]

[0056] X (l) Represents the data input to the lth submodule, X (l+1) is the data output by the lth submodule, X (l+1) (x, y, k) is the point with coordinates (x, y, k) in the output data. k represents a channel dimension of the output data. The features of this dimension are extracted by the kth convolution kernel in the convolution layer. W i，k，j (l) is the parameter matrix of the kth convolution kernel corresponding to the jth channel in the i-th convolution layer in the l-th submodule, s is the side length of the convolution kernel, It is one of the sampling points in the convolutional layer input data, and j represents a channel dimension of the input data.

[0057] Furthermore, the S64 segmentation data graph convolution branch in step S6 is specifically composed of three sub-modules with similar structures connected in series, each sub-module is composed of three network layers connected in series, and the sub-module structure is the same as the S62 block data graph convolution branch sub-module.

[0058] Furthermore, the data graph convolution feature processing module of step S65 in step S6 can be subdivided into two submodules, and the final output is the sum of the outputs of the two submodules. The two submodules are:

[0059] S651: Convolution feature processing submodule, which converts the pixel block features of the input module into pixel-level features using convolution operation;

[0060] S652: A neighborhood aggregation submodule converts the pixel block features of the input module into pixel-level features using a neighborhood aggregation operation.

[0061] Furthermore, the segmentation data feature processing module in step S66 is specifically composed of a feature similarity weight aggregation operation. The specific formula of the similarity weight aggregation operation is:

[0062]

[0063] X sgc represents the final pixel-level features of the segmented data, P is the number of pixels in the segmented data, X f (a) is a pixel in the segmented data, X f (t) is the target pixel.

[0064] Furthermore, the feature fusion module S67 in step S6 specifically learns the relationship between features through a dilated convolution and performs a fusion operation and a final classification. The specific formula of the operation is:

[0065]

[0066] Represents the classification result, X h Represents one of the input features to be fused, W dcov Represents the parameters of the dilated convolution, W lin represents the linear layer parameters, and σ(·) represents the softmax activation function.

[0067] Furthermore, the specific steps of the convolution feature processing submodule S651 in step S65 are as follows:

[0068] S6511: Deep convolution layer, which uses large convolution kernels to convert the pixel block features of the input module into pixel-level features;

[0069] S6512: Batch normalization layer, which normalizes the converted pixel-level features;

[0070] S6513: Mish activation function, which performs nonlinear mapping on features.

[0071] The specific formula of the S651 convolution feature processing submodule is:

[0072]

[0073] Among them, X Cov Represents the output data of the convolutional feature processing submodule, W Cov，j is the parameter matrix of the jth convolution kernel (the convolution kernel calculated on the jth channel), X out It is the input data of the convolutional feature processing submodule.

[0074] Furthermore, in step S65, the neighborhood aggregation submodule S652 performs neighborhood aggregation operation on the features mainly according to the Laplace matrix, and the specific formula is:

[0075]

[0076] Among them, X Agg Represents the output data of the neighborhood aggregation submodule, It is the input data of the neighborhood aggregation submodule.

[0077] Beneficial effects: (1) The present invention enhances the ability of graph convolutional networks to effectively extract features from complex shapes by constructing a graph structure based on segmented data.

[0078] (2) By constructing a graph structure with different characteristics based on multi-scale segmentation data and tile data, the present invention improves the overall consistency of the graph structure, thereby improving the classification accuracy.

[0079] (3) The present invention introduces two different feature processing methods to process local features with different characteristics, thereby improving pixel-level feature representation and enhancing classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is a flowchart of image data preprocessing of a graph convolutional fusion network for hyperspectral image classification disclosed in the present invention.

[0081] Figure 2 This is a schematic diagram of a graph convolutional fusion network for hyperspectral image classification disclosed in the present invention. DETAILED DESCRIPTION

[0082] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0083] The present invention discloses a graph convolution fusion network for hyperspectral image classification, which mainly includes image data preprocessing, using block partitioning, linear discriminant analysis and simple linear iterative clustering methods to divide segmentation and block data of different scales, and constructing a pixel-level graph structure based on the segmentation and block data; constructing a classification network, including a spectral conversion module, a block data graph convolution branch, a block data convolution branch, a segmentation data graph convolution branch, a block data graph convolution feature processing module, a segmentation data feature processing module and a feature fusion module; the block data graph convolution feature processing module improves pixel-level feature expression and enhances classification accuracy by combining a large convolution kernel convolution layer with neighborhood aggregation; the segmentation data feature processing module improves pixel-level feature expression and enhances classification accuracy by utilizing feature similarity weight aggregation; the feature fusion module learns the intrinsic connection between features through void convolution, improves feature fusion effect and enhances classification accuracy.

[0084] Based on the above method, this embodiment applies the above method to hyperspectral image classification. The specific process is as follows:

[0085] like Figure 1 As shown:

[0086] S1: Standardize the hyperspectral image data (raw data) HSI to obtain standardized data

[0087] S2: Standardize the data The pixels to be classified and the pixels with a distance of no more than 2 from them are constructed into a 5×5 block data Normalize the data The pixels to be classified and the pixels with a distance of no more than 7 from them are constructed into a 15×15 block data.

[0088] S3: Standardized data Apply the linear discriminant analysis method to perform dimensionality reduction operation and obtain dimensionality reduction data

[0089] S4: Dimensionality reduction data Perform fine simple linear clustering to obtain a fine segmentation map, and perform coarse simple linear clustering to obtain a coarse segmentation map;

[0090] S5: Based on refined segmentation map and normalized data Get segmentation data Based on rough segmentation map and normalized data Get segmentation data

[0091] like Figure 2 As shown:

[0092] S6: construct a classification network, including S61 spectral conversion module, S62 block data graph convolution branch, S63 block data convolution branch, S64 segmentation data graph convolution branch, S65 block data graph convolution feature processing module, S66 segmentation data feature processing module and S67 feature fusion module;

[0093] The classification network in S6 is specifically composed of an S61 spectral conversion module, two S62 block data graph convolution branches, two S63 block data convolution branches, two S64 segmentation data graph convolution branches, two S65 block data graph convolution feature processing modules, two S66 segmentation data feature processing modules and an S67 feature fusion module. The specific process is as follows:

[0094] (1) Data and The data are then sent to the S61 spectrum conversion module to extract the spectral features and reduce the spectral dimension.

[0095] (2) Change the and The acquired data and the corresponding Laplace matrix and The data is fed into two different S62 block graph convolution branches and two S63 block data convolution branches in parallel to extract spatial features with different characteristics;

[0096] (3) The features extracted by the two S62 block data graph convolution branches in the previous step are respectively sent to the two S65 block data graph convolution feature processing modules to obtain the feature X 1g and X 2g ;

[0097] (4) The features extracted by the two S63 block data convolution branches in step (2) are average pooled to obtain the feature X 1c and X 2c ;

[0098] (5) Change the step (1) from and The acquired data and the corresponding Laplace matrix and The data is fed into two different S64 segmentation graph convolution branches in parallel to extract spatial features;

[0099] (6) The features extracted by the two S64 segmentation data graph convolution branches in the previous step are respectively sent to the two S66 segmentation data feature processing modules to obtain the feature X s1 and X s2 .

[0100] The S61 spectrum conversion module consists of two submodules with the same structure connected in series. Each submodule is composed of three network layers connected in series, including the following steps:

[0101] S611: 1×1 convolutional layer, extracting spectral features from the input data and reducing its spectral dimension;

[0102] S612: Batch normalization layer, which performs normalization on the extracted features. The specific formula is:

[0103]

[0104] Among them, T (l) is the input data of the batch normalization layer, E(·) represents the mean function, Var(·) represents the variance function, ∈ is a correction to prevent the denominator from being too small, γ and β are learnable parameters used to adjust the mean and variance so that the batch normalized data conforms to the original distribution as much as possible. is the output data of the batch normalization layer;

[0105] S613: Mish activation function, which performs nonlinear mapping on features. The specific formula is:

[0106] f(x)=x×tanh(ln(1+e x ))

[0107] Among them, x represents the pixel feature in the data, and f(x) is the output value of the activation function.

[0108] The specific formula of the S61 spectrum conversion module is:

[0109] X spe =Mish(BN(Mish(BN(XW sp1 ))W sp2 ))

[0110] Among them, X represents the input data of the classification network, X spe is the output data of the spectrum conversion module, W sp1 and W sp2 is the parameter matrix of the spectral conversion module, BN(·) represents the batch normalization operation, and Mish(·) represents the Mish activation function.

[0111] The S62 block data graph convolution branch is specifically composed of three sub-modules with similar structures connected in series. Each sub-module is composed of three network layers connected in series, including the following steps:

[0112] S621: graph convolution layer, extracts spatial features from input data;

[0113] S622: Batch normalization layer, which performs normalization on the extracted features;

[0114] S623: Mish activation function, which performs nonlinear mapping on features.

[0115] The specific formula of the submodule is:

[0116]

[0117] Represents the data input to the lth submodule, is the data output by the lth submodule, W (l) is the graph convolution parameter matrix in the lth submodule, is the Laplacian matrix corresponding to the input data.

[0118] The S63 block data convolution branch is specifically composed of three sub-modules with similar structures connected in series. Each sub-module is composed of three network layers connected in series, including the following steps:

[0119] S631: Convolutional layer, extracts spatial features from input data;

[0120] S632: Batch normalization layer, which performs normalization on the extracted features;

[0121] S633: Mish activation function, which performs nonlinear mapping on features.

[0122] The specific formula of the submodule is:

[0123]

[0124] X (l) Represents the data input to the lth submodule, X (l+1) is the data output by the lth submodule, X (l+1) (x, y, k) is the point with coordinates (x, y, k) in the output data. k represents a channel dimension of the output data. The features of this dimension are extracted by the kth convolution kernel in the convolution layer. W i，k，j (l) is the parameter matrix of the kth convolution kernel corresponding to the jth channel in the i-th convolution layer in the l-th submodule, s is the side length of the convolution kernel, It is one of the sampling points in the convolutional layer input data, and j represents a channel dimension of the input data.

[0125] The S64 segmentation data graph convolution branch is specifically composed of three sub-modules with similar structures connected in series. Each sub-module is composed of three network layers connected in series. The sub-module structure is the same as the S62 block data graph convolution branch sub-module.

[0126] The S65 block data graph convolution feature processing module can be divided into two sub-modules. The final output is the sum of the outputs of the two sub-modules. The two sub-modules are:

[0127] S651: Convolution feature processing submodule, which converts the pixel block features of the input module into pixel-level features using convolution operation;

[0128] S652: A neighborhood aggregation submodule converts the pixel block features of the input module into pixel-level features using a neighborhood aggregation operation.

[0129] The specific steps of the S651 convolution feature processing submodule are:

[0130] S6511: Deep convolution layer, which uses large convolution kernels to convert the pixel block features of the input module into pixel-level features;

[0131] S6512: Batch normalization layer, which normalizes the converted pixel-level features;

[0132] S6513: Mish activation function, which performs nonlinear mapping on features.

[0133] The specific formula of the S651 convolution feature processing submodule is:

[0134]

[0135] Among them, X Cov Represents the output data of the convolutional feature processing submodule, W Cov,j is the parameter matrix of the jth convolution kernel (the convolution kernel calculated on the jth channel), X out It is the input data of the convolutional feature processing submodule.

[0136] The S652 neighborhood aggregation submodule mainly performs neighborhood aggregation operations on features according to the Laplace matrix. The specific formula is:

[0137]

[0138] Among them, X Agg Represents the output data of the neighborhood aggregation submodule, It is the input data of the neighborhood aggregation submodule.

[0139] S66 segmentation data feature processing module is specifically composed of feature similarity weight aggregation operation. The specific formula of similarity weight aggregation operation is:

[0140]

[0141] X sgc represents the final pixel-level features of the segmented data, P is the number of pixels in the segmented data, X f (a) is a pixel in the segmented data, X f (t) is the target pixel.

[0142] The S67 feature fusion module is specifically composed of a dilated convolution to learn the relationship between features and perform fusion operations and final classification. The specific formula of the operation is:

[0143]

[0144] Represents the classification result, X h Represents one of the input features to be fused, W dcov Represents the parameters of the dilated convolution, W lin represents the linear layer parameters, and σ(·) represents the softmax activation function.

Claims

1. A graph convolutional fusion network for hyperspectral image classification, characterized by: The steps include: S1: Standardize the hyperspectral image data (raw data) HSI to obtain standardized data S2: Standardize the data The pixels to be classified and the pixels with a distance of no more than 2 from them are constructed into a 5×5 block data Normalize the data The pixels to be classified and the pixels with a distance of no more than 7 from them are constructed into a 15×15 block data. S3: Standardized data Apply the linear discriminant analysis method to perform dimensionality reduction operation and obtain dimensionality reduction data S4: Dimensionality reduction data Perform fine simple linear clustering to obtain a fine segmentation map, and perform coarse simple linear clustering to obtain a coarse segmentation map; S5: Based on refined segmentation map and normalized data Get segmentation data Based on rough segmentation map and normalized data Get segmentation data S6: construct a classification network, including S61 spectral conversion module, S62 block data graph convolution branch, S63 block data convolution branch, S64 segmentation data graph convolution branch, S65 block data graph convolution feature processing module, S66 segmentation data feature processing module and S67 feature fusion module; S7: Data-based and Calculate the corresponding Laplace matrix and S8: Data and The labeled part and the corresponding Laplacian matrix and The training data is fed into the classification network together as input data, and the network output results of the training set pixels are compared with their labels to calculate the cross entropy loss L. The Adam optimizer is used to backpropagate the network parameters to reduce the cross entropy loss L. The training is continued until the cross entropy loss L converges and the network parameters are saved. S9: Data and The unlabeled part and the corresponding Laplacian matrix and They are input into the classification network for classification prediction and the classification results of the entire hyperspectral image are output.

2. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The classification network in step S6 specifically consists of a spectrum conversion module S61, two block data graph convolution branches S62, two block data convolution branches S63, two segmentation data graph convolution branches S64, two block data graph convolution feature processing modules S65, two segmentation data feature processing modules S66 and one feature fusion module S67. The specific process is as follows: (1) Data and The data are sent to the S61 spectrum conversion module to extract the spectral features and reduce the spectral dimension; (2) Change the and The acquired data and the corresponding Laplace matrix and The data is fed into two different S62 block graph convolution branches and two S63 block data convolution branches in parallel to extract spatial features with different characteristics; (3) The features extracted by the two S62 block data graph convolution branches in the previous step are respectively sent to the two S65 block data graph convolution feature processing modules to obtain the feature X 1g and X 2g ; (4) The features extracted by the two S63 block data convolution branches in step (2) are average pooled to obtain the feature X 1c and X 2c ; (5) Change the step (1) from and The acquired data and the corresponding Laplace matrix and The data is fed into two different S64 segmentation graph convolution branches in parallel to extract spatial features; (6) The features extracted by the two S64 segmentation data graph convolution branches in the previous step are respectively sent to the two S66 segmentation data feature processing modules to obtain the feature X s1 and X s2 .

3. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The spectrum conversion module S61 in step S6 is specifically composed of two submodules of the same structure connected in series, each submodule is composed of three network layers connected in series, and includes the following steps: S611: 1×1 convolutional layer, extracting spectral features from the input data and reducing its spectral dimension; S612: Batch normalization layer, which performs normalization on the extracted features; S613: Mish activation function, which performs nonlinear mapping on features.

4. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The data graph convolution branch of block S62 in step S6 is specifically composed of three submodules with similar structures connected in series, each submodule is composed of three network layers connected in series, including the following steps: S621: graph convolution layer, extracts spatial features from input data; S622: Batch normalization layer, which performs normalization on the extracted features; S623: Mish activation function, which performs nonlinear mapping on features.

5. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The data convolution branch of block S63 in step S6 is specifically composed of three submodules with similar structures connected in series, each submodule is composed of three network layers connected in series, and includes the following steps: S631: Convolutional layer, extracts spatial features from input data; S632: Batch normalization layer, which performs normalization on the extracted features; S633: Mish activation function, which performs nonlinear mapping on features.

6. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: In the step S6, the S64 segmentation data graph convolution branch is specifically composed of three sub-modules with similar structures connected in series, each sub-module is composed of three network layers connected in series, and the sub-module structure is the same as the S62 block data graph convolution branch sub-module.

7. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The data graph convolution feature processing module of block S65 in step S6 can be subdivided into two submodules. The final output is the sum of the outputs of the two submodules. The two submodules are: S651: Convolution feature processing submodule, which converts the pixel block features of the input module into pixel-level features using convolution operation; S652: A neighborhood aggregation submodule converts the pixel block features of the input module into pixel-level features using a neighborhood aggregation operation.

8. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The segmentation data feature processing module S66 in step S6 is specifically composed of a feature similarity weight aggregation operation. The specific formula of the similarity weight aggregation operation is: X sgc represents the final pixel-level features of the segmented data, P is the number of pixels in the segmented data, X f (a) is a pixel in the segmented data, X f (t) is the target pixel.

9. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The feature fusion module S67 in step S6 specifically learns the relationship between features through a dilated convolution and performs a fusion operation and a final classification. The specific formula of the operation is: Represents the classification result, X h Represents one of the input features to be fused, W dcov Represents the parameters of the dilated convolution, W lin represents the linear layer parameters, and σ(·) represents the softmax activation function.

10. The graph convolutional fusion network for hyperspectral image classification according to claim 7, characterized in that: The specific steps of the convolution feature processing submodule S651 in step S65 are: S6511: Deep convolution layer, which uses large convolution kernels to convert the pixel block features of the input module into pixel-level features; S6512: Batch normalization layer, which normalizes the converted pixel-level features; S6513: Mish activation function, which performs nonlinear mapping on features; The specific formula of the S651 convolution feature processing submodule is: where X Cov Represents the output data of the convolutional feature processing submodule, W Cov,j is the parameter matrix of the jth convolution kernel (the convolution kernel calculated on the jth channel), X out It is the input data of the convolutional feature processing submodule.

11. The graph convolutional fusion network for hyperspectral image classification according to claim 7, characterized in that: In step S65, the neighborhood aggregation submodule S652 mainly performs neighborhood aggregation operation on features according to the Laplace matrix. The specific formula is: where X Agg Represents the output data of the neighborhood aggregation submodule, It is the input data of the neighborhood aggregation submodule.

Images