Method for hyperspectral image classification based on multi-scale GCN and DCN graph reconstruction

Abstract

The application relates to the technical field of remote sensing image processing and computer vision, and discloses a multiscale GCN and DCN hyperspectral image classification method based on graph reconstruction, which adaptively captures irregular spatial features by combining a dynamic spectral feature extraction module (DSFEM) with a deformable convolution (DCN), extracts global features under different segmentation scales by using a multiscale spatial-spectral graph convolution module (MSSGCM), and dynamically adjusts a graph structure by using a graph structure reconstruction module (GSRM-GL) to accurately model the relationship between superpixel nodes. Meanwhile, a spatial-spectral joint loss function (SSJL) fuses a cross-entropy loss, a spatial total variation loss and a spectral total variation loss to optimize classification accuracy and feature continuity. The multiscale GCN and DCN hyperspectral image classification method based on graph reconstruction provided by the application significantly improves classification accuracy on Indian Pines, University of Pavia and other data sets, still maintains high robustness under a small sample condition, and solves the problems of limited feature extraction and fixed graph structure in the prior art.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and computer vision technology, and in particular to a multi-scale GCN and DCN hyperspectral image classification method based on graph reconstruction. Background Technology

[0002] Hyperspectral remote sensing technology enables the comprehensive observation of both spatial and spectral information. Therefore, hyperspectral images (HSI) obtained through hyperspectral remote sensing are three-dimensional image data containing both spectral and spatial information. Through comprehensive analysis of spatial and spectral information, land cover categories can be accurately identified, which is the primary task of hyperspectral image classification.

[0003] In the field of hyperspectral image (HSI) classification, convolutional neural networks (CNNs) and graph convolutional networks (GCNs) are two widely used mainstream techniques. CNNs, with their fixed convolutional kernel structure, can effectively extract pixel-level features and local spatial dependencies. Graph convolutions, on the other hand, generally operate on superpixel structures to extract irregular superpixel-level features and model long-range relationships.

[0004] While combining CNNs and GCNs can significantly improve classification performance, CNNs struggle to capture irregular spatial overlay relationships in hyperspectral images, limiting feature extraction. Secondly, although GCNs can handle irregular graph structures, they typically perform convolution operations based on fixed graph structures. Such graph structures often rely on an initial graph for construction, which frequently fails to accurately reflect the geometric and spectral similarities between superpixel nodes, causing central nodes to be unable to perceive important features within the graph structure. Therefore, traditional methods are limited in classification accuracy and stability, especially prone to misclassification when dealing with scenarios with complex ground features and irregular spatial information.

[0005] Therefore, existing hyperspectral image classification techniques have significant shortcomings in extracting features of irregular land cover and accurately modeling map structures:

[0006] 1. Although convolutional neural networks (CNNs) can effectively extract local features, due to their fixed convolutional kernel structure, they are difficult to capture irregular spatial coverage relationships in hyperspectral images, and are prone to overfitting when the sample size is limited.

[0007] 2. Although Graph Convolutional Networks (GCNs) have achieved certain advantages in superpixel structures, traditional methods often rely on pre-defined fixed graph structures, which make it difficult to fully express the complex relationships between nodes. This makes it difficult for central nodes to perceive key features in complex neighborhoods, thus affecting the accuracy and robustness of classification.

[0008] Therefore, it is necessary to provide a multi-scale GCN and DCN hyperspectral image classification method based on graph reconstruction, which can make full use of multi-scale feature information in irregular spatial distributions and dynamically adjust the graph structure to more accurately reflect the relationship between nodes, thereby improving the accuracy and robustness of further hyperspectral image classification. Summary of the Invention

[0009] This invention provides a multi-scale GCN and DCN hyperspectral image classification method based on graph reconstruction, aiming to effectively solve the problems of existing technologies struggling to capture irregular spatial features and accurately modeling superpixel node relationships using initial graphs. Figure 1 As shown, this method extracts local features by adjusting the shape of the convolution kernel and captures global features at different segmentation scales by combining multi-scale graph convolution. By constructing a graph reconstruction module, the central node can effectively perceive key features in the graph structure, more accurately reflecting the geometric and spectral similarity between nodes, thereby achieving significant improvements in classification accuracy, continuity of spatial-spectral information, and robustness under small sample conditions.

[0010] The technical solution of the present invention is as follows:

[0011] The multi-scale GCN and DCN hyperspectral image classification method based on graph reconstruction provided by this invention includes the following steps:

[0012] I. Preprocessing of Hyperspectral Images

[0013] The input hyperspectral image is represented as X∈R H×W×B Where H represents the image height, W represents the width, and B represents the spectral dimension. Principal component analysis (PCA) is used to reduce the dimensionality of the spectral data, thereby reducing information redundancy and obtaining the dimensionality-reduced band data.

[0014] II. Dynamic Spectral Feature Extraction Module (DSFEM)

[0015] 1. First, the input hyperspectral image data A 1x1 convolutional layer is used to extract the depth spectral features of the image while maintaining the spatial dimension and reducing computational cost.

[0016]

[0017] Where X(i,j,n) represents the nth spectral channel value at position (i,j) in the input feature map X. The weights of the 1x1 convolution kernel, F 1×1 ∈R H×W×CIt is the output feature map after passing through a 1x1 convolutional layer, where C represents the number of output channels and * represents the convolution operation.

[0018] 2. Perform deformable DCN convolution operation

[0019] ① Generate the initial set of sampling point coordinates

[0020] like Figure 2 As shown, in feature map F 1×1 Generate an initial set of N sample point coordinates P. n ={(p n,x ,p n,y Let p be the coordinates of each sampling point in the convolution kernel, where p = 1, 2, ..., N. n,x and p n,y These represent the coordinates of the nth sampling point in the x and y directions, respectively.

[0021] ② Learning and calculating the offset of the new sampling point coordinates

[0022] To adapt to the different shapes and regions of features in hyperspectral images, the network generates an offset ΔP for each sampling point through a trainable 2D convolutional layer, allowing for dynamic adjustment of the sampling point's position. The offset is generated as follows: ΔP = W 3×3 *F 1×1 W 3×3 This is a 3x3 convolutional kernel used to generate offsets, with 2N output channels, where N represents the number of sampling points. Therefore, the output contains the displacement of each sampling point in the x and y directions. The offset set ΔP contains the displacement pairs for each sampling point: ΔP∈{(Δp n,x ,Δp n,y )|n=1,2,...,N}, where Δp n,x and Δp n,y These represent the displacements of the nth sampling point in the x and y directions, respectively.

[0023] ③ The generated offset ΔP is added to the initial sampling point coordinates P0 and the relative position P of the convolution kernel. n In the process, the coordinates of the new sampling point P′ are calculated using the formula: P′=P0+P n +ΔP, where: P0 represents the reference coordinate grid, defining the fixed position of all pixels in the image; P n This represents the offset of each sampling point in the convolution kernel relative to the reference coordinates. ΔP is the offset learned through convolution.

[0024] ④ Calculate eigenvalues ​​using bilinear interpolation

[0025] At the new sampling point coordinates P′, bilinear interpolation is applied to calculate the eigenvalues ​​to maintain the continuity and spatial consistency of spectral information and reduce information loss during resampling. The calculation formula is as follows:

[0026] x offset =g lt X(q lt )+g rb X(q rb )+g lb X(q lb )+g rt X(q rt )

[0027] g lt =(1+(q) lt,x -p x ′))·(1+(q lt,y -p′ y ))

[0028] g rb =(1-(q) rb,x -p x ′))·(1-(q rb,y -p′ y ))

[0029] g lb =(1+(q) lb,x -p x ′))·(1-(q lb,y -p′ y ))

[0030] g rt =(1-(q) rt,x -p x ′))·(1+(q rt,y -p′ y ))

[0031] Where: q lt ,q rb ,q lb ,q rt X(q) represents the coordinates of four integer sampling points: top left, bottom right, bottom left, and top right; lt ),X(q rb ),X(q lb ),X(q rt ) represent the feature values ​​of these four sampling points in the input feature map.

[0032] g lt ,g rb ,g lb ,g rt It is the weight of the interpolation.

[0033] ⑤ Rearrangement and column convolution operations

[0034] Resampled feature map x offset ∈R C×H×W×N Rearranged as x offset ∈R C×(H×W)×N We use a two-dimensional convolution kernel of size Nx1 to perform column convolution to extract spectral variation features in the vertical direction. The column convolution is calculated as follows:

[0035]

[0036] Where: Y i，j It is the value of the output feature map at position (i,j);

[0037] W n These are the weights of the column convolution kernel, representing the weights at different positions along the column direction;

[0038] x offset (i,j,n) represents the resampled feature value at position (i,j).

[0039] ⑥ Batch normalization and activation function

[0040] The output Y of the column convolution is processed by batch normalization (Norm) and a non-linear activation function (SiLU) to generate intermediate feature maps F. int :

[0041] F int (i,j)=SiLU(Norm(Y i,j ))

[0042] 3. Feature fusion from the second 1x1 convolutional layer

[0043] After completing the DCN spatial feature extraction, a second 1x1 convolutional layer is used to fuse inter-band information and integrate information from different spectral regions to extract discriminative spectral features, resulting in the final pixel-level output feature H. pf :

[0044]

[0045] in represents the weights of the second 1x1 convolution kernel, and C represents the final number of output channels.

[0046] III. Multiscale Spectral-Spatial Graph Convolution Module (MSSGCM)

[0047] 1. The Boruvka algorithm is used to progressively merge the dimensionality-reduced hyperspectral image into five superpixel structures with different segmentation scales: Let m represent the set of superpixels in the l-th layer. l This represents the number of superpixels in the l-th layer. The number of superpixels decreases as the scale increases (m). l >m l+1 m 0 =H*W), where H and W represent the height and width of the input image, respectively.

[0048] 2. Construct the superpixel relation matrix. The relationship between superpixels in layer l and layer l+1 can be modeled, as follows:

[0049]

[0050] Q l+1 The i-th row and j-th column represents the superpixel region of the l-th layer. Did it participate in merging into the superpixel region of layer l+1?

[0051] 3. Graph convolution input feature construction: For multi-scale GCN modules, matrix input features are first used. Input features for constructing the first layer of graph convolution Among them, B * The spectral dimension of the input image, each line and S 1 Each of them Correspondingly:

[0052]

[0053] in Represents the column pair of matrix Q 1 Perform normalization (e.g.) Reshape() means mapping the input from a 3D tensor to a 2D matrix (H×W×B). * →HW×B * ).

[0054] 4. Construct the adjacency matrix. Based on the principle of spatial similarity and the selection criteria for neighbor nodes, define the graph structure as follows:

[0055]

[0056] Where A 1 The i-th row and j-th column in the diagram represents the superpixel region of the l-th layer. and Are they adjacent in spatial dimensions (i.e., do they share a common edge)?

[0057] 5. Multi-scale graph convolution calculation, for A 1 Averaging by row yields (in ),use Perform graph convolution to extract superpixel features at the first segmentation scale.

[0058]

[0059] in The convolution parameters are used to fully explore the impact of land cover features with different topological structures on the classification results, and the output features are... The feature matrix is ​​further divided into three sub-feature matrices based on the channel. Then, use Constructing input features for graph convolution at different scales

[0060]

[0061] Next, we use the adjacency matrix: Correspondingly, parallel graph convolutions are performed on graph structures of different scales to extract superpixel-level features:

[0062]

[0063]

[0064] W2, W3, and W4 represent the convolution parameters of different layers.

[0065] 6. Multi-scale feature integration, which integrates multi-scale features The multi-scale superpixel output features H are obtained through integration. ms :

[0066]

[0067] IV. Hierarchical Feature Fusion Module (HFFM)

[0068] 1. Input pixel-level feature H pf and multi-scale superpixel level features H ms .

[0069] 2. Pixel-level features H pf and superpixel-level features H ms Connect along the channel dimension to generate a unified feature representation H i :

[0070] Hi =concat(H pf H ms )

[0071] Among them, set

[0072] 3. Use depthwise separable convolution (SSConv) to process the fused features H i Perform convolution operations to generate further fused features H fuse =SSConv(H i The SSConv operation can effectively reduce the computational complexity and number of parameters of the model while fusing spatial and spectral information, thereby improving the efficiency of the model.

[0073] V. Graph Structure Reconstruction Based on Geometric and Label Information Module (GSRM-GL)

[0074] 1. Initial construction of superpixel regions: The Boruvka minimum spanning tree algorithm is used to divide the hyperspectral image into superpixel regions of multiple scales. Each superpixel region can be regarded as the result of unsupervised clustering.

[0075] Each superpixel region is treated as a set of similar pixels for subsequent graph structure reconstruction.

[0076] 2. Calculate label information:

[0077] 2.1 As Figure 3 As shown, the feature representation H is used from the model output. fuse and feature matrices at different levels Generate multiple cluster centers

[0078] 2.2 For each superpixel region, calculate its cluster centers in the spatial and spectral dimensions to form feature representations at different levels.

[0079] 2.3 Feature H fuse Divided into tagged collections and unlabeled sets Two parts: Calculating the center of the labeled set, used for subsequent geometric similarity calculations:

[0080]

[0081] Where C n y represents the center of the label node of the nth class. vrepresents the actual label of the labeled node, J represents the total number of categories, and M(·) represents the mean function.

[0082] 3. Calculate the geometric distance

[0083] 3.1 Cosine distance is used to measure the directional difference between superpixel centers. The formula is as follows:

[0084]

[0085] in, and These are the feature vectors of the superpixel region and the label center, respectively.

[0086] 3.2 Standardized Euclidean distance is used to measure the spatial difference of superpixel centers. First, the features are standardized, and then the standardized Euclidean distance is calculated:

[0087]

[0088] 3.3 Calculate the fusion similarity by combining cosine distance and standardized Euclidean distance:

[0089]

[0090] Here, α1 and α2 are weighting coefficients, satisfying α1 + α2 = 1.

[0091] 4. Pseudo-label allocation

[0092] 4.1 For each superpixel region, select the label center with the minimum distance based on the fusion similarity and assign a pseudo-label:

[0093]

[0094] Represents the i-th superpixel region of layer l. With the center of the nth class Geometric similarity metric finds the feature that best matches the class center with the current superpixel region by selecting the minimum distance, and assigns a pseudo-label to each superpixel region in the l-th layer.

[0095] 5. Redefinition of Graph Structure

[0096] The adjacency relationships of the graph structure are redefined by combining geometric similarity and label information. The new adjacency matrix is ​​defined as follows:

[0097]

[0098] Spatial similarity and label information together determine the connectivity between nodes, ensuring that the graph structure retains only nodes with high similarity.

[0099] The newly constructed graph structure is fed into the original model, and the entire model is retrained:

[0100]

[0101] in

[0102] VI. Spatial-Spectral Joint Loss Function (SSJL)

[0103] 1. Spatial-spectral joint loss function L total The overall expression is:

[0104] L total =L CE +λ spatial ·L TV,spatial +λ spectral ·L TV,spectral

[0105] Where: L CE Cross-entropy loss is used to measure the difference between the model's predicted class and the true class; L TV,spatial It is the total spatial variation loss, used to maintain the smoothness of the image space; L TV,spectral To account for the total spectral variation loss and ensure the smoothness of the image in the spectral dimension; λ spatial and λ spectral It is a weighting parameter used to adjust the contribution of different loss terms to the total loss, in order to balance the consistency between classification accuracy and spatial spectral features.

[0106] 2. Calculation of cross-entropy loss

[0107] Cross-entropy loss L CE It is used to calculate the difference between the predicted class and the true class, and is defined as follows:

[0108]

[0109] Where T is the number of training samples, C is the number of classes, and y n,c One-hot encoding for the nth sample, This represents the model's predicted probability for the c-th class of the n-th sample. Cross-entropy loss is a commonly used loss function in classification tasks, used to guide the model to improve classification accuracy.

[0110] 3. Calculation of total spatial variation loss

[0111] L TV,spatial To maintain the smoothness of an image in its spatial dimensions, it is defined as:

[0112]

[0113] Where H and W are the height and width of the image, respectively, and x n,i,j This represents the pixel value of the nth sample at position (i,j). The purpose of this term is to maintain the spatial smoothness of the image and prevent aberrant changes in image space by minimizing the differences between adjacent pixels.

[0114] 4. Calculation of total spectral variation loss

[0115] L TV,spectral To maintain the smoothness of an image in the spectral dimension, it is defined as:

[0116]

[0117] Where, x n,k This represents the pixel value of the nth sample at spectral channel k. This term maintains the smoothness of spectral information by minimizing the differences between adjacent spectral channels, ensuring a more stable feature representation of the spectral image.

[0118] VII. Classification Prediction

[0119] The reconstructed image features are input into a multilayer perceptron (MLP) layer for classification. The softmax function is used to calculate the confidence level of each category, and the classification probability P of each pixel is obtained.

[0120] P = soft max(FCL(H) fuse ))

[0121] in, J represents the total number of categories.

[0122] Predict the category of each pixel and calculate the overall accuracy (OA), average accuracy (AA), and Kappa coefficient between the true and predicted categories to evaluate the effectiveness of the classification method.

[0123] Overall accuracy (OA): Overall accuracy is the ratio of the number of samples correctly classified across all categories to the total number of samples.

[0124]

[0125] Average precision (AA): Used to measure the performance of a classification algorithm in multi-class classification tasks.

[0126] This metric calculates the classification accuracy for each category and then averages these accuracies.

[0127]

[0128] in,

[0129] Kappa coefficient: The Kappa coefficient takes into account the randomness in classification. Its value is between -1 and 1, where 1 indicates that the classifier's prediction is completely consistent with the actual situation, 0 indicates that the classifier's prediction is comparable to random prediction, and a negative value indicates that the consistency between the classifier's prediction and the actual situation is not as good as random prediction.

[0130]

[0131] Assume the number of true samples in each class is a1, ..., ac, and the number of predicted samples in each class is b1, b2, ..., bc, with a total number of samples of b1, b2, ..., bc and a total number of samples of ρ. Then we have

[0132] Compared with related technologies, the multi-scale GCN and DCN hyperspectral image classification method based on graph reconstruction provided by this invention significantly improves classification accuracy and model robustness by dynamically updating the graph structure and extracting multi-scale features. Specifically, this invention has the following technical effects:

[0133] 1. Significantly Improved Classification Accuracy: Through the Dynamic Spectral Feature Extraction (DSFEM) module and the Parallel Multi-Scale Spatial-Spectral Map Convolution (MSSGCM) module, this invention extracts features on superpixel structures at different segmentation scales, effectively capturing both local and global features in ground cover information. Experimental results based on four standard datasets (Indian Pines, University of Pavia, Salinas, and Kennedy Space Center) demonstrate that this invention exhibits higher accuracy and generalization ability. Under the same sample conditions, compared with seven neural network algorithms, the algorithm proposed in this invention improves OA by 1.86%-33.5% on the 2% training set of Indian Pines, 1.16%-38.14% on the 0.2% training set of University of Pavia, 3.84%-28.11% on the 5-sample training set of Salinas, and 1.42%-42.88% on the 2% training set of KSC.

[0134] 2. Enhanced Model Robustness: The graph structure reconstructing module (GSRM-GL) of this invention dynamically adjusts the graph structure by combining geometric and label information, retaining only highly similar nodes, thus improving the adaptability of the graph structure to complex terrain information. For example... Figure 4As shown, even with a small sample size, the method still maintains excellent classification performance on the Indian Pines dataset with only 1%-2% of the training samples, demonstrating high accuracy and stability in both overall and subclass classification. Attached Figure Description

[0135] Figure 1 A schematic diagram of a parallel multi-scale GCN-DCN hyperspectral image classification network for graph reconstruction based on geometric and label information;

[0136] Figure 2 In the DCN dynamic sampling process, taking 5 sampling points as an example, deformable convolution generates dynamic offsets, obtains integer feature values ​​through bilinear interpolation, and extracts spatial features using a convolution kernel of size (N,1) after rearrangement.

[0137] Figure 3 To illustrate the reconstruction process, we use 6 superpixel nodes as an example to represent J different categories and the pseudo-labels generated by the 6 superpixel nodes. In the superpixel structure diagram, we use different colors to represent different pseudo-labels.

[0138] Figure 4 Line graphs showing overall classification accuracy (OA) for different samples: (A) OA accuracy of the Indian Pines dataset with (0.5%, 1%, 2%, 3%, 4%) training samples; (B) OA accuracy of the University of Pavia dataset with (0.1%, 0.2%, 0.3%, 0.4%, 0.5%) training samples; (C) OA accuracy of the Salinas dataset with (3, 5, 7, 9, 11) samples; (D) OA accuracy of the Kennedy Space Center dataset with (0.5%, 1%, 2%, 3%, 4%) training samples.

[0139] Figure 5 The following are comparison results of different algorithms for Salinas on 5 training samples: (a) ground map, (b) false color image, (c) CNN (OA = 69.66%), (d) SSRN (OA = 86.92%), (e) HybridSN (OA = 90.73%), (f) CEGCN (OA = 92.63%), (g) WFCG (OA = 92.58%), (h) MSSGU (OA = 93.93%), (i) AMGCF (OA = 92.88%), (j) Proposed (OA = 97.77%). Detailed Implementation

[0140] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0141] The multi-scale GCN and DCN hyperspectral image classification method based on graph reconstruction provided by this invention includes the following steps:

[0142] Step S1, Data Preprocessing: Preprocess the collected hyperspectral image data and public datasets to prepare for subsequent model training.

[0143] The specific process of step S1 is as follows:

[0144] Step S11: Dataset partitioning;

[0145] The Salinas dataset consists of hyperspectral images acquired by the U.S. Department of Agriculture (USDA) using aerial photography (AVIRIS). The images are 512 × 217 pixels in size, with a spatial resolution of 3.7 meters. The Salinas dataset contains 224 consecutive spectral bands covering a wide range from visible to near-infrared light. The images encompass 16 different land cover categories, totaling 53,987 samples, including farmland, vegetation, and water bodies. Five samples were randomly selected from each category as training samples, 0.1% as validation samples, and the remainder as test samples. The table below shows the data splitting settings for the Salinas dataset's training, validation, and test sets.

[0146] Step S12: Data Processing

[0147] For Salinas hyperspectral data X∈R H×W×B PCA dimensionality reduction was performed to obtain the dimensionality-reduced band data. The obtained data has a height H = 512, a width W = 217, and a number of spectral channels B. * =64, the dimensionality-reduced data is fed into the two branches in parallel.

[0148] Step S2: Model building: Establish a graph reconstruction-based hyperspectral image classification network that integrates multi-scale GCN and DCN.

[0149] Specifically, step S2 includes the following steps:

[0150] Step S21: Construct a dynamic spectral feature extraction module (DSFEM) to capture local pixels and enhance adaptability to spectral changes.

[0151] 1. Perform 1x1 convolution on the dimensionality-reduced Salinas data to extract deep spectral features from the image.

[0152]

[0153] Where: X(i,j,n) represents the nth spectral channel value at position (i,j) in the input feature map X;

[0154] The weights of the 1x1 convolution kernel, F 1×1 ∈R H×W×C It is the output feature map after passing through a 1x1 convolutional layer, where C represents the number of output channels and * represents the convolution operation.

[0155] 2. Using DCN for pixel-level feature extraction in local irregular regions:

[0156]

[0157] Where: Y i，j It is the value of the output feature map at position (i,j);

[0158] W n These are the weights of the column convolution kernel, representing the weights at different positions along the column direction;

[0159] x offset (i,j,n) represents the resampled feature value at position (i,j);

[0160] * indicates a convolution operation.

[0161] After column convolution, batch normalization (Norm) and the SiLU activation function are used to generate intermediate feature maps F. int :

[0162] F int (i,j)=SiLU(Norm(Y i,j ))

[0163] 3. After completing the DCN spatial feature extraction, to further fuse information between bands, we pass these spatial features into a 1x1 convolutional layer. A 1x1 convolution can integrate information from different spectral regions, extracting more discriminative spectral features. The final output pixel-level feature is H. pf It contains diverse spectral and spatial information.

[0164]

[0165] Step S22: Construct a multi-scale spatial-spectral map convolution module (MSSGCM) to extract global features at different scales in parallel.

[0166] 1. The Boruvka algorithm is used to progressively merge the dimensionality-reduced Salinas image into five superpixel structures with different segmentation scales: Let m represent the set of superpixels in the l-th layer. lThis represents the number of superpixels in the l-th layer. The number of superpixels decreases as the scale increases (m). l >m l+1 m 0 =H*W), where H and W represent the height and width of the input image, respectively.

[0167] 2. Construct the Salinas superpixel relation matrix. The relationship between superpixels in layer l and layer l+1 can be modeled, as follows:

[0168]

[0169] Q l+1 The i-th row and j-th column represents the superpixel region of the l-th layer. Did it participate in merging into the superpixel region of layer l+1?

[0170] 3. Graph convolution input feature construction: For multi-scale GCN modules, matrix input features are first used. Input features for constructing the first layer of graph convolution Among them, B * The spectral dimension of the input image, each line and S 1 Each of them Correspondingly:

[0171]

[0172] in Represents the column pair of matrix Q 1 Perform normalization (e.g.) Reshape() means mapping the input from a 3D tensor to a 2D matrix (H×W×B). * →HW×B * ).

[0173] 4. Construct the adjacency matrix. Based on the definition of spatial similarity and the selection of neighbor nodes, we define the graph structure as follows:

[0174]

[0175] Where A 1 The i-th row and j-th column in the diagram represents the superpixel region of the l-th layer. and Are they adjacent in spatial dimensions (i.e., do they share a common edge)?

[0176] 5. Multi-scale graph convolution computation. For A 1 Averaging by row yields (in ),use Perform graph convolution to extract superpixel features at the first segmentation scale.

[0177]

[0178] in The convolution parameters are used to fully explore the impact of land cover features with different topological structures on the classification results, and the output features are... The feature matrix is ​​further divided into three sub-feature matrices based on the channel. Then, use Constructing input features for graph convolution at different scales

[0179]

[0180]

[0181] Next, we use the adjacency matrix: Correspondingly, parallel graph convolutions are performed on graph structures of different scales to extract superpixel-level features:

[0182]

[0183] W2, W3, and W4 represent the convolution parameters of different layers.

[0184] 6. Multi-scale feature integration, which integrates multi-scale features The multi-scale superpixel output features H are obtained through integration. ms :

[0185]

[0186] 7. Input Salinas pixel-level features H pf Salinas multi-scale superpixel-level features H ms .

[0187] 8. Pixel-level feature H pf and superpixel-level features H ms Connect along the channel dimension to generate a unified feature representation H i :

[0188] H i =concat(H pf H ms )

[0189] Among them, set

[0190] 9. Use depthwise separable convolution (SSConv) to process the fused features Hi Perform convolution operations to generate further fused features H fuse =SSConv(H i The SSConv operation can effectively reduce the computational complexity and number of parameters of the model while fusing spatial and spectral information, thereby improving the efficiency of the model.

[0191] Step S23: Construct a graph reconstruction module (GSRM-GL) based on geometric and label information to make up for the shortcomings of the initial graph in measuring the relationship between superpixel nodes and further explore the deep spatial-spectral information hidden in the hyperspectral image.

[0192] 1. Initial construction of superpixel regions: The Boruvka minimum spanning tree algorithm is used to divide the hyperspectral image into superpixel regions of multiple scales. Each superpixel region can be regarded as the result of unsupervised clustering.

[0193] Each superpixel region is treated as a set of similar pixels for subsequent graph structure reconstruction.

[0194] 2. Calculate label information:

[0195] 2.1 Using the feature representation H output by the model fuse and feature matrices at different levels Multiple cluster centers are generated.

[0196] 2.2 For each superpixel region, calculate its cluster centers in the spatial and spectral dimensions to form feature representations at different levels.

[0197] 2.3 Feature H fuse Divided into tagged collections and unlabeled sets Two parts: Calculating the center of the labeled set, used for subsequent geometric similarity calculations:

[0198]

[0199] Where C n y represents the center of the label node of the nth class. v represents the actual label of the labeled node, J represents the total number of categories, and M(·) represents the mean function.

[0200] 3. Calculate the geometric distance

[0201] 3.1 Cosine distance is used to measure the directional difference between superpixel centers. The formula is as follows:

[0202]

[0203] in, and These are the feature vectors of the superpixel region and the label center, respectively.

[0204] 3.2 Standardized Euclidean distance is used to measure the spatial difference of superpixel centers. First, the features are standardized, and then the standardized Euclidean distance is calculated:

[0205]

[0206] 3.3 Calculate the fusion similarity by combining cosine distance and standardized Euclidean distance:

[0207]

[0208] Here, α1 and α2 are weighting coefficients, satisfying α1 + α2 = 1.

[0209] Step S24: Pseudo-label assignment

[0210] 4.1 For each superpixel region, select the label center with the minimum distance based on the fusion similarity and assign a pseudo-label:

[0211]

[0212] Represents the i-th superpixel region of layer l. With the center of the nth class Geometric similarity metric finds the feature that best matches the class center with the current superpixel region by selecting the minimum distance, and assigns a pseudo-label to each superpixel region in the l-th layer.

[0213] 5. Redefinition of Graph Structure

[0214] The adjacency relationships of the graph structure are redefined by combining geometric similarity and label information. The new adjacency matrix is ​​defined as follows:

[0215]

[0216] Spatial similarity and label information together determine the connectivity between nodes, ensuring that the graph structure retains only nodes with high similarity.

[0217] The newly constructed graph structure is used to retrain the classification model, further improving classification accuracy.

[0218]

[0219] in

[0220] Step S3, Model Training: Train the model constructed in step S2 until the entire model converges and the optimal model is preserved.

[0221] 1. Experimental Environment

[0222] To more scientifically evaluate the effectiveness of the proposed method, several representative algorithms were selected for comparison. These include three convolutional neural network-based methods: Deep Convolutional Neural Network (CNN), Spatial Spectral Residual Network (SSRN), and 2D-3D Hybrid Spectral Network (HybridSN). Four methods combining CNN and graph convolutional networks were also included: CEGCN, WFCG, MSSGU, and AMGCF. The hyperparameters obtained by the compared methods are consistent with those in the original literature. Each value in the table uses the average precision and corresponding standard deviation of ten experiments to obtain stable and reliable results on each HSI dataset. The experiments were run on a computer with an NVIDIA GeForce RTX 4060Ti (16GB GPU memory) and an i5-12600KF CPU. Python 3.9 was used as the programming language, and PyTorch 2.4.0 and CUDA version 12.1 were selected as the programming environment.

[0223] 2. Define hyperparameters:

[0224] The learning rate is set to 5e-4. Training is terminated when the number of training iterations reaches the training iteration count Epoch (300 times), resulting in a trained hyperspectral image.

[0225] 3. The spatial spectral joint loss function is forward propagated, and then backpropagated using the Adam optimizer to optimize all parameters of the hyperspectral image classification neural network.

[0226] Using the spatial spectrum joint loss function L total The overall expression is:

[0227] L total =L CE +λ spatial ·L TV,spatial +λ spectral ·L TV,spectral

[0228] Where: L CE Cross-entropy loss is used to measure the difference between the model's predicted class and the true class; L TV,spatial It is the total spatial variation loss, used to maintain the smoothness of the image space; L TV,spectral To account for the total spectral variation loss and ensure the smoothness of the image in the spectral dimension; λ spatial and λ spectral It is a weighting parameter used to adjust the contribution of different loss terms to the total loss, in order to balance the consistency between classification accuracy and spatial spectral features.

[0229] Step S4, Model Validation Inference: Input the test set data into the trained optimal model, infer the pixel category prediction values ​​of the test image, and evaluate the prediction metrics.

[0230] The specific process of step S4 is as follows: After completing the model training, fix the model parameters, input the hyperspectral images and labels in the test set into the optimal model, predict the category to which each pixel belongs, and calculate the overall accuracy OA, average accuracy AA and Kappa coefficient between the true category and the predicted category to evaluate the effectiveness of the classification method.

[0231] Overall accuracy OA:

[0232]

[0233] Average accuracy AA:

[0234]

[0235] Kappa coefficient:

[0236]

[0237] Assume the true number of samples in each class is a1, ..., ac, and the predicted number of samples in each class is b1, b2, ..., bc, with a total number of samples of b1, b2, ..., bc and a total number of samples of ρ. Then we have

[0238] like Figure 5 As shown, given the input of the original Salinas hyperspectral image and its ground truth, the visual classification results of the Salinas dataset are: (a) ground truth map; (b) false-color image; (c) CNN (OA = 69.66%); (d) SSRN (OA = 86.92%); (e) HybridSN (OA = 90.73%); (f) CEGCN (OA = 92.63%); (g) WFCG (OA = 92.58%); (h) MSSGU (OA = 93.93%); (i) AMGCF (OA = 92.88%); (j) Proposed (FMGDR) (OA = 97.77%). Figure 5 It can be seen that the model recognition accuracy of the present invention is higher. This completes the model testing and inference operation.

[0239] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A multi-scale GCN and DCN hyperspectral image classification method based on graph reconstruction, characterized in that, Includes the following steps: S1. Acquire the original hyperspectral image data and preprocess it. Reduce redundant information by using spectral dimensionality reduction method to obtain the dimensionality-reduced hyperspectral data. S2. Construct a multi-scale GCN and DCN hyperspectral image classification neural network based on graph reconstruction. This network includes a dynamic spectral feature extraction module DSFEM and a multi-scale spatial-spectral graph convolution module MSSGCM. In the dynamic spectral feature extraction module DSFEM, 1x1 convolution and deformable convolution DCN are used to extract the spectral-spatial features of local irregular regions. The deformable convolution DCN is implemented through the following steps: a. Generate an initial set of sampling point coordinates and learn the offset through trainable convolutional layers to adjust the sampling position; b. Calculate the feature values ​​of new sampling points using bilinear interpolation and extract the vertical spectral features through column convolution. c. Output features are processed using batch normalization and the nonlinear activation function SiLU; The Multi-Scale Spatial-Spectral Graph Convolutional Module (MSSGCM) extracts global features at different scales in parallel, specifically including the following steps: a. Using the Boruvka algorithm, the dimensionality-reduced Salinas image is divided into five layers of superpixel structures at different segmentation scales through progressive merging, with the number of superpixels decreasing as the scale increases; b. A Salinas superpixel relation matrix is ​​constructed, and convolution input features are constructed for the multi-scale GCN module graph; c. An adjacency matrix is ​​constructed, and the graph structure is defined according to the definition of spatial similarity and the selection of neighbor nodes; d. Multi-scale graph convolution calculation and multi-scale feature integration are performed; e. Salinas pixel-level features and Salinas multi-scale superpixel-level features are input, and the two are connected along the channel dimension to generate a unified feature representation. f. Use depthwise separable convolution SSConv to perform convolution operations on the fused features to generate further fused features; S3. The graph reconstruction module GSRM-GL, based on geometric and label information, constructs a new graph structure to enhance the model's understanding of spatial and spectral information. The graph reconstruction module GSRM-GL is implemented through the following steps: a. Calculate the geometric center of the label category and measure the similarity between the superpixel and the label center based on cosine distance and normalized Euclidean distance; b. Assign pseudo-labels to the superpixel and reconstruct the adjacency matrix to preserve the connection of highly similar nodes. c. Retrain the model using the reconstructed graph structure to improve classification accuracy; S4. Train the network using training samples of known categories to obtain a trained classification model; S5. The classification model is further optimized using a spatial-spectral joint loss function, including cross-entropy loss, spatial total variation loss, and spectral total variation loss, to ensure classification accuracy, spatial information smoothness, and spectral information continuity; the spatial-spectral joint loss function... The expression is: ， in: Cross-entropy loss is used to measure the difference between the model's predicted class and the true class. It is the total spatial variation loss, used to maintain the smoothness of the image space; To compensate for the total spectral variation loss, the smoothness of the image in the spectral dimension is ensured; and It is a weighting parameter used to adjust the contribution of different loss terms to the total loss, in order to balance the consistency between classification accuracy and spatial spectral features; S6. Input the hyperspectral image data to be classified into the trained model, use the classification neural network to classify it at the pixel level, and finally output the classification result of each pixel.

2. The multi-scale GCN and DCN hyperspectral image classification method based on graph reconstruction according to claim 1, characterized in that, The evaluation metrics for the classification results include: a. Overall accuracy (OA): The proportion of correctly classified samples out of the total samples; b. Average Precision (AA): The average classification precision of each category; c. Kappa coefficient: measures the consistency between classification results and random predictions.

3. The multi-scale GCN and DCN hyperspectral image classification method based on graph reconstruction according to claim 1, characterized in that, The formula for calculating bilinear interpolation is as follows: ， ， in, These represent the coordinates of four integer sampling points: top left, bottom right, bottom left, and top right, respectively. These represent the feature values ​​of the four sampling points in the input feature map, respectively; It is the weight of the interpolation.

Images