Graph convolution fusion network for hyperspectral image classification

Abstract

The invention discloses an image convolution fusion network for hyperspectral image classification, which comprises the following steps: image data preprocessing: dividing segmentation and block data of different scales by using methods such as block division, linear discriminant analysis and simple linear iterative clustering, and constructing a pixel-level image structure based on the segmentation and block data; constructing a classification network, wherein the classification network comprises a spectrum conversion module, a block data graph convolution branch, a block data convolution branch, a segmentation data graph convolution branch, a block data graph convolution feature processing module, a segmentation data feature processing module and a feature fusion module; the block data graph convolution feature processing module is used for improving pixel-level feature expression and improving classification precision through a mode of combining a large convolution kernel convolution layer and neighborhood aggregation; the segmentation data feature processing module is used for improving pixel-level feature expression by using feature similarity weight aggregation and improving classification precision; and the feature fusion module is used for improving the feature fusion effect and improving the classification precision through the internal relation between cavity convolution learning features. According to the method, classification work of hyperspectral images is carried out by designing a brand new convolution and graph convolution fusion neural network, and the method comprises the following steps: firstly, dividing segmentation and block data of different scales by using methods such as block division, linear discriminant analysis and simple linear iterative clustering, and constructing a pixel-level graph structure based on the segmentation and block data; then spectral features of the segmented and block data are extracted through a spectrum conversion module, spectral dimensions of the segmented and block data are reduced, spatial features of the segmented and block data are extracted through convolution and graph convolution, and then block data graph convolution features are processed through a block data graph convolution feature processing module; and processing segmented data features by using a segmented data feature processing module, finally adaptively fusing the features by using a feature fusion module, and finally classifying the fused features.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral remote sensing image processing, and specifically to a graph convolutional fusion network for hyperspectral image classification. Background Art

[0002] Hyperspectral images (HSI) have been widely used in various remote sensing fields such as target detection, agricultural monitoring, and marine safety in recent years. Hyperspectral images are different from conventional two-dimensional digital images. They are three-dimensional cube data, consisting of two-dimensional digital images and one-dimensional spectral dimensions. Spectral bands contain rich geographic feature information. Therefore, feature selection and feature extraction are particularly important for hyperspectral pixel classification. In the past, the commonly used classification methods for hyperspectral images included: K-NN, Extreme Learning Machine (ELM), and Support Vector Machine (SVM). They all use spectral information for classification. K-NN is the simplest classifier in the field of machine learning; and extreme learning machine is a new type of fast learning algorithm. For a single hidden layer neural network, ELM can randomly initialize input weights and biases and obtain corresponding output weights. Support vector machine (SVM) pursues the best approximation of the model through small sample data training to obtain the best balance between model complexity and learning ability. It is one of the widely used machine learning algorithms. In addition, principal component analysis (PCA) can reduce noise interference by compressing spectral bands through frequency band selection, which can retain important features and is particularly suitable for hyperspectral image classification.

[0003] In recent years, image classification based on deep learning methods has been increasingly favored by researchers. The basic idea of ​​deep learning image classification is to obtain nonlinear features through dense connections of multiple layers of neurons using simple linear equations and activation functions, extract abstract features from raw image data, and form specific network weight parameters through training with a large amount of raw data, thereby improving the classification accuracy of images. The spectral dimensions of hyperspectral images can reach dozens or even hundreds of bands. Chen et al. first introduced the concept of deep learning into hyperspectral image classification and proposed a model based on stacked autoencoders (SAE), which used autoencoders to extract raw data information and finally used SVM classification. However, this method only used spectral information but ignored the use of spatial information.

[0004] In order to further utilize the spatial feature information of hyperspectral images, a hyperspectral algorithm based on convolutional neural network (CNN) was proposed. Convolutional neural network is widely used in hyperspectral image classification due to its powerful feature extraction ability for fixed-shape data blocks. Convolutional neural network has been proven to be able to effectively extract pixel-level features, but the weight sharing and fixed shape of the convolution kernel make it difficult to adapt to data with complex feature shapes.

[0005] Graph convolutional neural networks are also widely used in the field of hyperspectral image classification due to their powerful ability to extract graph structure features. However, most existing graph convolution methods use superpixel division to reduce the number of nodes, which inevitably causes the loss of pixel-level features. At the same time, the aggregation of other node information will inevitably introduce erroneous information. Summary of the invention

[0006] Purpose of the invention: In order to better integrate the advantages of convolutional neural networks and graph convolutional neural networks and improve the ability of graph convolutional neural networks to aggregate neighborhood information, a graph convolutional fusion network for hyperspectral image classification is provided, which designs a multi-branch hyperspectral image classification network, extracts spatial features of different characteristics by constructing local graph structures of different shapes and scales, and then improves the feature representation through feature processing modules designed for different characteristics, and finally uses the feature fusion module to adaptively fuse features and classify.

[0007] Technical solution: To achieve the above purpose, the present invention provides a graph convolution fusion network for hyperspectral image classification, comprising the following steps:

[0008] S1: Standardize the hyperspectral image data (raw data) HSI to obtain standardized data

[0009] S2: Standardize the data The pixels to be classified and the pixels whose distance to them is no more than 2 are constructed into a 5×5 block data. Standardize the data The pixels to be classified and the pixels whose distance is no more than 7 are constructed into a 15×15 block data.

[0010] S3: Standardized data Apply the linear discriminant analysis method to perform dimensionality reduction operations and obtain dimensionality reduction data

[0011] S4: Dimensionality reduction data Perform fine simple linear clustering to obtain a fine segmentation map, and perform coarse simple linear clustering to obtain a coarse segmentation map;

[0012] S5: Based on fine segmentation map and standardized data Get segmentation data Based on rough segmentation map and standardized data Get segmentation data

[0013] S6: construct a classification network, including S61 spectral conversion module, S62 block data graph convolution branch, S63 block data convolution branch, S64 segmentation data graph convolution branch, S65 block data graph convolution feature processing module, S66 segmentation data feature processing module and S67 feature fusion module;

[0014] S7: Data-based and Calculate the corresponding Laplace matrix and Laplacian Matrix From the adjacency matrix A i Calculated, the adjacency matrix A i The calculation formula is:

[0015]

[0016] Among them, X i f is the pixel block Z i Reduce the spatial dimension to a one-dimensional two-dimensional matrix form, a and b are X i f The spatial coordinates of the midpoint.

[0017] Laplacian Matrix The calculation formula is:

[0018]

[0019] Where I is the shape and A i The same identity matrix, D i A i The degree matrix of

[0020] S8: Data and The labeled part and the corresponding Laplacian matrix and They are input into the classification network as input data for training, and the network output results of the training set pixels are compared with their labels to calculate the cross entropy loss L. The Adam optimizer is used to backpropagate the network parameters to reduce the cross entropy loss L. The training is continued until the cross entropy loss L converges and the network parameters are saved.

[0021] The calculation formula of cross entropy loss L is:

[0022]

[0023] Among them, L i is the cross entropy loss of the i-th sample, N is the number of samples, M is the number of categories, and p ic is the predicted probability that the i-th sample belongs to the c-th class, y ic is the true label of the cth category of the i-th sample;

[0024] S9: Data and The unlabeled part and the corresponding Laplacian matrix and They are input into the classification network for classification prediction and the classification result of the entire hyperspectral image is output.

[0025] Furthermore, the classification network in step S6 specifically consists of a S61 spectrum conversion module, two S62 block data graph convolution branches, two S63 block data convolution branches, two S64 segmentation data graph convolution branches, two S65 block data graph convolution feature processing modules, two S66 segmentation data feature processing modules and one S67 feature fusion module. The specific process is:

[0026] (1) Data and The spectrum is sent to the S61 spectrum conversion module to extract the spectrum features and reduce the spectrum dimension;

[0027] (2) The and The acquired data and the corresponding Laplace matrix and The data are sent to two different S62 block graph convolution branches and two S63 block data convolution branches in parallel to extract spatial features with different characteristics;

[0028] (3) The features extracted by the two S62 block data graph convolution branches in the previous step are respectively sent to the two S65 block data graph convolution feature processing modules to obtain the feature X 1g and X 2g ;

[0029] (4) The features extracted by the two S63 block data convolution branches in step (2) are averaged and pooled to obtain feature X 1c and X 2c ;

[0030] (5) Replace the step (1) with and The acquired data and the corresponding Laplace matrix and The data is sent to two different S64 segmentation graph convolution branches in parallel to extract spatial features;

[0031] (6) The features extracted by the two S64 segmentation data graph convolution branches in the previous step are respectively sent to the two S66 segmentation data feature processing modules to obtain the feature X s1 and X s2 .

[0032] Furthermore, the spectrum conversion module S61 in step S6 is specifically composed of two submodules of the same structure connected in series, each submodule is composed of three network layers connected in series, and includes the following steps:

[0033] S611: 1×1 convolutional layer, extracting spectral features of input data and reducing its spectral dimension;

[0034] S612: Batch normalization layer, which normalizes the extracted features. The specific formula is:

[0035]

[0036] Among them, T (l) is the input data of the batch normalization layer, E(·) represents the mean function, Var(·) represents the variance function, ε is a correction to prevent the denominator from being too small, γ and β are learnable parameters used to adjust the mean and variance so that the batch normalized data conforms to the original distribution as much as possible. is the output data of the batch normalization layer;

[0037] S613: Mish activation function, which performs nonlinear mapping on features. The specific formula is:

[0038] f(x)=x×tanh(ln(1+e x ))

[0039] Among them, x represents the pixel feature in the data, and f(x) is the output value of the activation function.

[0040] The specific formula of the S61 spectrum conversion module is:

[0041] X spe =Mish(BN(Mish(BN(XW sp1 ))W sp2 ))

[0042] Among them, X represents the input data of the classification network, X spe is the output data of the spectrum conversion module, W sp1 and W sp2 is the parameter matrix of the spectral conversion module, BN(·) represents the batch normalization operation, and Mish(·) represents the Mish activation function.

[0043] Furthermore, the data graph convolution branch of block S62 in step S6 is specifically composed of three submodules with similar structures connected in series, each submodule is composed of three network layers connected in series, and includes the following steps:

[0044] S621: graph convolution layer, extracting spatial features from input data;

[0045] S622: batch normalization layer, normalizing the extracted features;

[0046] S623: Mish activation function, which performs nonlinear mapping on features.

[0047] The specific formula of the submodule is:

[0048]

[0049] Represents the data input to the lth submodule, is the data output by the lth submodule, W (l) is the graph convolution parameter matrix in the lth submodule, is the Laplacian matrix corresponding to the input data.

[0050] Furthermore, the data convolution branch of block S63 in step S6 is specifically composed of three submodules with similar structures connected in series, each submodule is composed of three network layers connected in series, and includes the following steps:

[0051] S631: convolutional layer, extracting spatial features from input data;

[0052] S632: batch normalization layer, normalizing the extracted features;

[0053] S633: Mish activation function, which performs nonlinear mapping on features.

[0054] The specific formula of the submodule is:

[0055]

[0056] X (l) Represents the data input to the lth submodule, X (l+1) is the data output by the lth submodule, X (l+1) (x, y, k) is the point with coordinates (x, y, k) in the output data. k represents a channel dimension of the output data. The features of this dimension are extracted by the kth convolution kernel in the convolution layer. W i，k，j (l) is the parameter matrix of the kth convolution kernel corresponding to the jth channel in the ith convolution layer in the lth submodule, s is the side length of the convolution kernel, It is one of the sampling points in the convolutional layer input data, and j represents a channel dimension of the input data.

[0057] Furthermore, in the step S6, the segmented data graph convolution branch S64 is specifically composed of three sub-modules with similar structures connected in series, each sub-module is composed of three network layers connected in series, and the sub-module structure is the same as the block data graph convolution branch sub-module S62.

[0058] Furthermore, the data graph convolution feature processing module of block S65 in step S6 can be subdivided into two submodules, and the final output is the sum of the outputs of the two submodules, and the two submodules are:

[0059] S651: a convolution feature processing submodule, which converts the pixel block features of the input module into pixel-level features by using a convolution operation;

[0060] S652: A neighborhood aggregation submodule converts the pixel block features of the input module into pixel-level features using a neighborhood aggregation operation.

[0061] Furthermore, the segmentation data feature processing module S66 in step S6 is specifically composed of feature similarity weight aggregation operation. The specific formula of similarity weight aggregation operation is:

[0062]

[0063] X sgc represents the final pixel-level features of the segmented data, P is the number of pixels in the segmented data, and X f (a) is a pixel in the segmented data, X f (t) is the target pixel.

[0064] Furthermore, the feature fusion module S67 in step S6 specifically learns the relationship between features through a dilated convolution and performs a fusion operation and a final classification. The specific formula of the operation is:

[0065]

[0066] Represents the classification result, X h Represents one of the input features to be fused, W dcov represents the parameters of the dilated convolution, W lin represents the linear layer parameters, and σ(·) represents the softmax activation function.

[0067] Furthermore, the specific steps of the convolution feature processing submodule S651 in step S65 are:

[0068] S6511: Deep convolution layer, which uses large convolution kernels to convert the pixel block features of the input module into pixel-level features;

[0069] S6512: Batch normalization layer, which normalizes the converted pixel-level features;

[0070] S6513: Mish activation function, which performs nonlinear mapping on features.

[0071] The specific formula of the S651 convolution feature processing submodule is:

[0072]

[0073] Among them, X Cov Represents the output data of the convolutional feature processing submodule, W Cov，j is the parameter matrix of the jth convolution kernel (the convolution kernel calculated on the jth channel), X out It is the input data of the convolutional feature processing submodule.

[0074] Furthermore, in the step S65, the neighborhood aggregation submodule S652 mainly performs neighborhood aggregation operation on the features according to the Laplace matrix, and the specific formula is:

[0075]

[0076] Among them, X Agg Represents the output data of the neighborhood aggregation submodule, It is the input data of the neighborhood aggregation submodule.

[0077] Beneficial effects: (1) The present invention enhances the ability of graph convolutional networks to effectively extract features from complex shapes by constructing a graph structure based on segmented data.

[0078] (2) The present invention constructs a graph structure with different characteristics based on multi-scale segmentation data and tile data. We improve the overall consistency of the graph structure, thereby improving the classification accuracy.

[0079] (3) The present invention introduces two different feature processing methods to process local features with different characteristics, thereby improving the pixel-level feature representation and enhancing the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is a flowchart of image data preprocessing of a graph convolutional fusion network for hyperspectral image classification disclosed in the present invention.

[0081] Figure 2 This is a schematic diagram of a graph convolutional fusion network for hyperspectral image classification disclosed in the present invention. DETAILED DESCRIPTION

[0082] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0083] The invention discloses a graph convolution fusion network for hyperspectral image classification, which mainly includes image data preprocessing, using block partitioning, linear discriminant analysis and simple linear iterative clustering methods to divide segmentation and block data of different scales, and building a pixel-level graph structure based on the segmentation and block data; building a classification network, including a spectral conversion module, a block data graph convolution branch, a block data convolution branch, a segmentation data graph convolution branch, a block data graph convolution feature processing module, a segmentation data feature processing module and a feature fusion module; the block data graph convolution feature processing module improves pixel-level feature expression and classification accuracy by combining a large convolution kernel convolution layer with neighborhood aggregation; the segmentation data feature processing module improves pixel-level feature expression and classification accuracy by using feature similarity weight aggregation; the feature fusion module learns the intrinsic connection between features through hole convolution, improves feature fusion effect and classification accuracy.

[0084] Based on the above method, this embodiment applies the above method to hyperspectral image classification, and the specific process is as follows:

[0085] like Figure 1 As shown:

[0086] S1: Standardize the hyperspectral image data (raw data) HSI to obtain standardized data

[0087] S2: Standardize the data The pixels to be classified and the pixels whose distance to them is no more than 2 are constructed into a 5×5 block data. Standardize the data The pixels to be classified and the pixels whose distance is no more than 7 are constructed into a 15×15 block data.

[0088] S3: Standardized data Apply the linear discriminant analysis method to perform dimensionality reduction operations and obtain dimensionality reduction data

[0089] S4: Dimensionality reduction data Perform fine simple linear clustering to obtain a fine segmentation map, and perform coarse simple linear clustering to obtain a coarse segmentation map;

[0090] S5: Based on fine segmentation map and standardized data Get segmentation data Based on rough segmentation map and standardized data Get segmentation data

[0091] like Figure 2 As shown:

[0092] S6: construct a classification network, including S61 spectral conversion module, S62 block data graph convolution branch, S63 block data convolution branch, S64 segmentation data graph convolution branch, S65 block data graph convolution feature processing module, S66 segmentation data feature processing module and S67 feature fusion module;

[0093] The classification network in S6 is specifically composed of an S61 spectral conversion module, two S62 block data graph convolution branches, two S63 block data convolution branches, two S64 segmentation data graph convolution branches, two S65 block data graph convolution feature processing modules, two S66 segmentation data feature processing modules and an S67 feature fusion module. The specific process is:

[0094] (1) Data and The spectrum is sent to the S61 spectrum conversion module to extract the spectrum features and reduce the spectrum dimension;

[0095] (2) The and The acquired data and the corresponding Laplace matrix and The data are sent to two different S62 block graph convolution branches and two S63 block data convolution branches in parallel to extract spatial features with different characteristics;

[0096] (3) The features extracted by the two S62 block data graph convolution branches in the previous step are respectively sent to the two S65 block data graph convolution feature processing modules to obtain the feature X 1g and X 2g ;

[0097] (4) The features extracted by the two S63 block data convolution branches in step (2) are averaged and pooled to obtain feature X 1c and X 2c ;

[0098] (5) Replace the step (1) with and The acquired data and the corresponding Laplace matrix and The data is sent to two different S64 segmentation graph convolution branches in parallel to extract spatial features;

[0099] (6) The features extracted by the two S64 segmentation data graph convolution branches in the previous step are respectively sent to the two S66 segmentation data feature processing modules to obtain the feature X s1 and X s2 .

[0100] The S61 spectrum conversion module is specifically composed of two sub-modules of the same structure connected in series. Each sub-module is composed of three network layers connected in series, including the following steps:

[0101] S611: 1×1 convolutional layer, extracting spectral features of input data and reducing its spectral dimension;

[0102] S612: Batch normalization layer, which normalizes the extracted features. The specific formula is:

[0103]

[0104] Among them, T (l) is the input data of the batch normalization layer, E(·) represents the mean function, Var(·) represents the variance function, ∈ is a correction to prevent the denominator from being too small, γ and β are learnable parameters used to adjust the mean and variance so that the batch normalized data conforms to the original distribution as much as possible. is the output data of the batch normalization layer;

[0105] S613: Mish activation function, which performs nonlinear mapping on features. The specific formula is:

[0106] f(x)=x×tanh(ln(1+e x ))

[0107] Among them, x represents the pixel feature in the data, and f(x) is the output value of the activation function.

[0108] The specific formula of the S61 spectrum conversion module is:

[0109] X spe =Mish(BN(Mish(BN(XW sp1 ))W sp2 ))

[0110] Among them, X represents the input data of the classification network, X spe is the output data of the spectrum conversion module, W sp1 and W sp2 is the parameter matrix of the spectral conversion module, BN(·) represents the batch normalization operation, and Mish(·) represents the Mish activation function.

[0111] The S62 block data graph convolution branch is specifically composed of three sub-modules with similar structures connected in series. Each sub-module is composed of three network layers connected in series, including the following steps:

[0112] S621: graph convolution layer, extracting spatial features from input data;

[0113] S622: batch normalization layer, normalizing the extracted features;

[0114] S623: Mish activation function, which performs nonlinear mapping on features.

[0115] The specific formula of the submodule is:

[0116]

[0117] Represents the data input to the lth submodule, is the data output by the lth submodule, W (l) is the graph convolution parameter matrix in the lth submodule, is the Laplacian matrix corresponding to the input data.

[0118] The S63 block data convolution branch is specifically composed of three sub-modules with similar structures connected in series. Each sub-module is composed of three network layers connected in series, including the following steps:

[0119] S631: convolutional layer, extracting spatial features from input data;

[0120] S632: batch normalization layer, normalizing the extracted features;

[0121] S633: Mish activation function, which performs nonlinear mapping on features.

[0122] The specific formula of the submodule is:

[0123]

[0124] X (l) Represents the data input to the lth submodule, X (l+1) is the data output by the lth submodule, X (l+1) (x, y, k) is the point with coordinates (x, y, k) in the output data. k represents a channel dimension of the output data. The features of this dimension are extracted by the kth convolution kernel in the convolution layer. W i，k，j (l) is the parameter matrix of the kth convolution kernel corresponding to the jth channel in the ith convolution layer in the lth submodule, s is the side length of the convolution kernel, It is one of the sampling points in the convolutional layer input data, and j represents a channel dimension of the input data.

[0125] The S64 segmentation data graph convolution branch is specifically composed of three sub-modules with similar structures connected in series. Each sub-module is composed of three network layers connected in series. The sub-module structure is the same as the S62 block data graph convolution branch sub-module.

[0126] The S65 block data graph convolution feature processing module can be divided into two sub-modules. The final output is the sum of the outputs of the two sub-modules. The two sub-modules are:

[0127] S651: a convolution feature processing submodule, which converts the pixel block features of the input module into pixel-level features by using a convolution operation;

[0128] S652: A neighborhood aggregation submodule converts the pixel block features of the input module into pixel-level features using a neighborhood aggregation operation.

[0129] The specific steps of the S651 convolution feature processing submodule are:

[0130] S6511: Deep convolution layer, which uses large convolution kernels to convert the pixel block features of the input module into pixel-level features;

[0131] S6512: Batch normalization layer, which normalizes the converted pixel-level features;

[0132] S6513: Mish activation function, which performs nonlinear mapping on features.

[0133] The specific formula of the S651 convolution feature processing submodule is:

[0134]

[0135] Among them, X Cov Represents the output data of the convolutional feature processing submodule, W Cov,j is the parameter matrix of the jth convolution kernel (the convolution kernel calculated on the jth channel), X out It is the input data of the convolutional feature processing submodule.

[0136] The S652 neighborhood aggregation submodule mainly performs neighborhood aggregation operations on features according to the Laplace matrix. The specific formula is:

[0137]

[0138] Among them, X Agg Represents the output data of the neighborhood aggregation submodule, It is the input data of the neighborhood aggregation submodule.

[0139] S66 Segmentation data feature processing module is specifically composed of feature similarity weight aggregation operation. The specific formula of similarity weight aggregation operation is:

[0140]

[0141] X sgc represents the final pixel-level features of the segmented data, P is the number of pixels in the segmented data, and X f (a) is a pixel in the segmented data, X f (t) is the target pixel.

[0142] The S67 feature fusion module is specifically composed of a dilated convolution to learn the relationship between features and perform fusion operations and final classification. The specific formula of the operation is:

[0143]

[0144] Represents the classification result, X h Represents one of the input features to be fused, W dcov represents the parameters of the dilated convolution, W lin represents the linear layer parameters, and σ(·) represents the softmax activation function.

Claims

1. A graph convolutional fusion network for hyperspectral image classification, characterized in that: The steps include: S1: Standardize the hyperspectral image data (raw data) HSI to obtain standardized data S2: Standardize the data The pixels to be classified and the pixels whose distance to them is no more than 2 are constructed into a 5×5 block data. Standardize the data The pixels to be classified and the pixels whose distance is no more than 7 are constructed into a 15×15 block data. S3: Standardized data Apply the linear discriminant analysis method to perform dimensionality reduction operations and obtain dimensionality reduction data S4: Dimensionality reduction data Perform fine simple linear clustering to obtain a fine segmentation map, and perform coarse simple linear clustering to obtain a coarse segmentation map; S5: Based on fine segmentation map and standardized data Get segmentation data Based on rough segmentation map and standardized data Get segmentation data S6: construct a classification network, including S61 spectral conversion module, S62 block data graph convolution branch, S63 block data convolution branch, S64 segmentation data graph convolution branch, S65 block data graph convolution feature processing module, S66 segmentation data feature processing module and S67 feature fusion module; S7: Data-based and Calculate the corresponding Laplace matrix and S8: Data and The labeled part and the corresponding Laplacian matrix and The pixels are input into the classification network as input data for training, and the network output results of the training set pixels are compared with their labels to calculate the cross entropy loss L. The Adam optimizer is used to back-propagate the network parameters to reduce the cross entropy loss L. The training is continued until the cross entropy loss L converges and the network parameters are saved. S9: Data and The unlabeled part and the corresponding Laplacian matrix and They are input into the classification network for classification prediction and the classification result of the entire hyperspectral image is output.

2. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The classification network in step S6 specifically consists of a spectrum conversion module S61, two block data graph convolution branches S62, two block data convolution branches S63, two segmentation data graph convolution branches S64, two block data graph convolution feature processing modules S65, two segmentation data feature processing modules S66 and one feature fusion module S67. The specific process is as follows: (1) Data and The spectrum is sent to the S61 spectrum conversion module to extract the spectrum features and reduce the spectrum dimension; (2) The and The acquired data and the corresponding Laplace matrix and The data are sent to two different S62 block graph convolution branches and two S63 block data convolution branches in parallel to extract spatial features with different characteristics; (3) The features extracted by the two S62 block data graph convolution branches in the previous step are respectively sent to the two S65 block data graph convolution feature processing modules to obtain the feature X 1g and X 2g ; (4) The features extracted by the two S63 block data convolution branches in step (2) are averaged and pooled to obtain feature X 1c and X 2c ; (5) Replace the step (1) with and The acquired data and the corresponding Laplace matrix and The data is sent to two different S64 segmentation graph convolution branches in parallel to extract spatial features; (6) The features extracted by the two S64 segmentation data graph convolution branches in the previous step are respectively sent to the two S66 segmentation data feature processing modules to obtain the feature X s1 and X s2 .

3. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The spectrum conversion module S61 in step S6 is specifically composed of two submodules of the same structure connected in series, each submodule is composed of three network layers connected in series, and includes the following steps: S611: 1×1 convolutional layer, extracting spectral features of input data and reducing its spectral dimension; S612: batch normalization layer, normalizing the extracted features; S613: Mish activation function, which performs nonlinear mapping on features.

4. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The data graph convolution branch of block S62 in step S6 is specifically composed of three submodules with similar structures connected in series, each of which is composed of three network layers connected in series, including the following steps: S621: graph convolution layer, extracting spatial features from input data; S622: batch normalization layer, normalizing the extracted features; S623: Mish activation function, which performs nonlinear mapping on features.

5. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The data convolution branch of block S63 in step S6 is specifically composed of three submodules with similar structures connected in series, each of which is composed of three network layers connected in series, including the following steps: S631: convolutional layer, extracting spatial features from input data; S632: batch normalization layer, normalizing the extracted features; S633: Mish activation function, which performs nonlinear mapping on features.

6. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: In the step S6, the S64 segmentation data graph convolution branch is specifically composed of three sub-modules with similar structures connected in series, each sub-module is composed of three network layers connected in series, and the sub-module structure is the same as the S62 block data graph convolution branch sub-module.

7. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The data graph convolution feature processing module of block S65 in step S6 can be subdivided into two submodules, and the final output is the sum of the outputs of the two submodules. The two submodules are: S651: a convolution feature processing submodule, which converts the pixel block features of the input module into pixel-level features by using a convolution operation; S652: A neighborhood aggregation submodule converts the pixel block features of the input module into pixel-level features using a neighborhood aggregation operation.

8. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The segmentation data feature processing module S66 in step S6 is specifically composed of feature similarity weight aggregation operation. The specific formula of similarity weight aggregation operation is: X sgc represents the final pixel-level features of the segmented data, P is the number of pixels in the segmented data, and X f (a) is a pixel in the segmented data, X f (t) is the target pixel.

9. The graph convolutional fusion network for hyperspectral image classification according to claim 1, characterized in that: The feature fusion module S67 in step S6 specifically learns the relationship between features through a dilated convolution and performs a fusion operation and a final classification. The specific formula of the operation is: Represents the classification result, X h Represents one of the input features to be fused, W dcov represents the parameters of the dilated convolution, W lin represents the linear layer parameters, and σ(·) represents the softmax activation function.

10. The graph convolutional fusion network for hyperspectral image classification according to claim 7, characterized in that: The specific steps of the convolution feature processing submodule S651 in step S65 are: S6511: Deep convolution layer, which uses large convolution kernels to convert the pixel block features of the input module into pixel-level features; S6512: Batch normalization layer, which normalizes the converted pixel-level features; S6513: Mish activation function, which performs nonlinear mapping on features. The specific formula of the S651 convolution feature processing submodule is: Where X Cov Represents the output data of the convolutional feature processing submodule, W Cov,j is the parameter matrix of the jth convolution kernel (the convolution kernel calculated on the jth channel), X out It is the input data of the convolutional feature processing submodule.

11. The graph convolutional fusion network for hyperspectral image classification according to claim 7, characterized in that: The neighborhood aggregation submodule S652 in step S65 mainly performs neighborhood aggregation operation on the features according to the Laplacian matrix, and the specific formula is: Where X Agg Represents the output data of the neighborhood aggregation submodule, It is the input data of the neighborhood aggregation submodule.

Images