Feature weighted fusion hyperspectral image classification method based on GAT and CNN

Abstract

The invention discloses a feature weighted fusion hyperspectral image classification method based on GAT and CNN. The method comprises the following steps: firstly, dividing a hyperspectral data set into a training set, a test set and a verification set; then, a double-branch feature weighted fusion network model combining CNN and GAT is constructed; inputting the divided training set into a classification network model, setting parameters, and training to obtain training indexes; and finally, evaluating indexes, data and image information of a classification result. According to the method, the Euclidean and non-Euclidean feature information of the hyperspectral image is fully extracted by utilizing the capture capability of the depth separable convolution and the GAT on the pixel information and the super-pixel data, and the accuracy and generalization capability of classified data are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral image classification, and in particular relates to a hyperspectral image classification method based on feature weighted fusion of GAT and CNN. Background Art

[0002] Hyperspectral images have hundreds or even thousands of continuous bands in the visible and infrared bands, and can capture the characteristic spectral information of objects and surface materials in different bands. Compared with traditional color or single-band images that can only provide limited color or band information, hyperspectral images can provide richer and more detailed spectral characteristics of objects, making it possible to analyze and identify objects more accurately. Therefore, hyperspectral images have been widely used in agricultural crop monitoring, environmental pollution detection, urban planning, mineral exploration, weather forecasting, and military reconnaissance. With its rich spectral information and diverse application capabilities, hyperspectral images have become an indispensable and important part of modern remote sensing technology. The premise of these applications is to accurately classify each pixel in the hyperspectral image (HSI).

[0003] In the past few decades, various machine learning-based classifiers have been developed for hyperspectral image classification. Early classification methods usually map high-dimensional hyperspectral information to low-dimensional information and process low-dimensional data. Therefore, how to establish mapping functions and find separable hyperplanes has become a research goal. For example, logistic regression and extreme learning machines have been used to develop pixel-level HSI classifiers. However, pixel-wise methods usually produce considerable errors or outliers in the final classification map. To alleviate this problem, kernel techniques such as kernel support vector machines and multi-kernel learning are used to improve linear separability. But these methods usually focus on the design of classifiers and ignore the representation and learning of features. In order to make full use of spectral information, typical representation-based methods such as sparse representation, low-order representation, and collaborative representation have been developed. Through representation learning, the intrinsic data structure of the spectrum can be revealed and the dependence on labeled samples can be reduced. In addition, the spatial structure of HSI has been explored, such as graph construction, superpixel segmentation, morphological segmentation, etc., to promote spectral-spatial feature learning. By explicitly modeling the spatial structure of HSI, its spatial information can be better utilized. However, limited by the artificial features and empirical parameters, the above methods cannot learn robust deep feature representations from HSI.

[0004] Compared with traditional machine learning classification methods, deep learning methods can automatically learn adaptive and robust deep features from training data. At present, many classic deep learning methods have been applied to HSI classification and achieved good results. For example, from one-dimensional convolutional neural networks (CNNs) to variants of 3D CNNs, from single CNNs to hybrid CNNs, the advantages of deep learning methods in HSI image classification are that they can better utilize big data and powerful computing power, thereby improving classification accuracy, reducing the need for artificial feature engineering, and being able to handle more complex and high-dimensional data features. Due to the high computational complexity, this hybrid CNN requires high computing power and long training time. Previous deep learning models were designed for Euclidean data, and they often ignored the intrinsic correlation between adjacent land covers. In recent years, graph neural networks (GNNs) have received increasing attention due to their ability to perform convolution operations on graphs of arbitrary structures. By encoding HSI into graphs, the correlation between adjacent land covers can be explicitly utilized, and GNNs can better simulate the spatial context structure of HSI. GNNs can not only perform description learning on non-Euclidean data, but also perform end-to-end representation learning on node feature information and structural information at the same time. HSI data can be converted into graph data through superpixel-based methods, and then the spectral-spatial context information can be effectively modeled using the GNN method. In this way, the number of labels is implicitly expanded, which alleviates the problem of small samples to a certain extent. Superpixel-based GNN can simulate various spatial structures of land cover on the graph, but it cannot generate subtle individual features for each pixel. In contrast, CNN can learn local spectral-spatial features at the pixel level, but its acceptance range is usually limited to a small square window, so the large-scale context structure of HSI may be difficult to capture. How to integrate the advantages of superpixel-based GNN and pixel-based CNN, and data interoperability has gradually become a key issue in the field of HSI classification. Summary of the invention

[0005] The present invention aims to provide an innovative hyperspectral image classification method, which can combine the advantages of graph attention network (GAT) and convolutional neural network (CNN) to achieve feature fusion by extracting features of images of different scales. GAT is an advanced deep learning architecture suitable for processing graph structured data. In the application of hyperspectral image classification, GAT can effectively learn the relative importance between nodes, thereby accurately capturing the non-Euclidean feature information in the graph. At the same time, this method utilizes the powerful pixel information capture capability of deep separable convolution and the processing capability of GAT for superpixel data to comprehensively extract the Euclidean and non-Euclidean feature information of hyperspectral images. Such feature fusion ensures high accuracy and good generalization performance of the classification results. The technical solution adopted by the present invention is a hyperspectral image classification method based on feature weighted fusion of GAT and CNN, characterized in that it is implemented specifically in accordance with the following steps:

[0006] Step 1: Divide the hyperspectral dataset into training set, test set and validation set;

[0007] Step 2: Construct a dual-branch feature weighted fusion classification network model combining CNN and GAT;

[0008] Step 3: input the training set divided in step 1 into the classification network model constructed in step 2, set parameters, and perform training to obtain training indicators;

[0009] Step 4: Evaluate the indicators, data, and image information of the classification results.

[0010] The present invention is also characterized in that:

[0011] Step 1 is implemented as follows:

[0012] Step 1.1, use the classic hyperspectral dataset Indian Pines as the model input for the experiment, preprocess the dataset in GAT and CNN branches respectively, and divide it into training set, test set and validation set. The training set and validation set are randomly selected from 1% of the total data, and the rest of the data is used as the test set.

[0013] Step 2 is implemented as follows:

[0014] Step 2.1, the dual-branch weighted fusion model combining depthwise separable convolution and GAT contains two branches in total, each branch has three parts, namely data preprocessing, multi-scale division and weighted fusion. For the GAT branch, the original pixel image is preprocessed by superpixel segmentation to obtain superpixel information of different segmentation scales, and the obtained information is input into three GAT networks respectively and the result data is spliced ​​as the output of the GAT main branch. For the CNN branch, the spectral convolution layer is used to perform dimensionality reduction and denoising on the input original data, the processed data is downsampled and input into the CNN branch, and the output result is upsampled and spliced ​​as the output of the CNN main branch;

[0015] Step 2.2: For the input data of the CNN branch, spectral convolution is used for preprocessing. First, the input image data is compressed and converted, and then the converted data is standardized. Then it is passed through the spectral convolution layer with a convolution kernel size of 1×1, and finally the LeakyReLU function is used to increase the generalization ability. The processed data is used as the input of the CNN main branch. For the input data of the GAT branch, superpixel segmentation is used for preprocessing. The specific method is to segment the original data into superpixels of different sizes to obtain segmentation information of different scales, and then the processed data is used as the input of the GAT main branch;

[0016] Step 2.3: For the multi-scale division of different branches, the input data obtained by the CNN branch through the spectral convolution layer is pooled twice with a kernel size of 2 and a maximum pooling step of 2 to obtain pixel-level information of three different scales, and the three types of data are input into the three branches of the CNN network. Each CNN branch is divided into three layers, and each layer passes through the position attention module PAM, the channel attention module CAM and the depth-separable convolution layer in turn. The position attention module PAM is used to capture the spatial position relationship in the image, and strengthens the model's understanding and processing ability of spatial information by weighting the features of each position. The position attention module PAM calculates the similarity of different positions in the feature map to generate an attention map, thereby emphasizing the key positions. The channel attention module CAM analyzes the importance of different channels and adjusts the response strength of each channel, so that the model can pay more attention to the feature channels that are more important to the final task. The depth-separable convolution is divided into two steps: first, use point-by-point depth convolution to process each channel of the input separately, and then use channel-by-channel point-by-point convolution to merge the outputs of the depth convolution. The output channels of each part of the first two layers of the first branch are 128, and the output channels of each part of the last layer are 128, 128, and 64 respectively. The output channels of the last two branches are halved in turn with reference to the pooled data. Finally, the output results of the last two branches are spliced ​​with the output results of the first branch after passing through the DYS upsampling module. The DYS upsampling module uses point sampling to select specific sample points from a given sample set, and performs spatial rearrangement operations on the original image. Finally, the spatial structure of the image data is organized to generate a sampled image, and the spliced ​​result is used as the output of the CNN total branch; for the GAT branch, the processed image data is segmented at superpixel scales of 100, 50, and 10 and input into the three branches of GAT respectively. Each GAT branch consists of two layers of GAT networks, and the adjacency matrix A and superpixel position matrix Q obtained by superpixel segmentation are input. The output channels are 128 and 64 respectively. Finally, the outputs of the three branches of GAT are spliced ​​as the output of the overall GAT total branch;

[0017] Step 2.4: For the fusion of superpixel data and pixel data, multiply the output data of each GAT branch with the corresponding scale superpixel position matrix Q obtained by its segmentation to convert it into output data of the same size and channel. Reshape the data of the CNN main branch into data of exactly the same size as the output of the GAT main branch for weighted fusion test, and use the fusion ratio with the best effect as the final network fusion parameter.

[0018] Step 3 is implemented as follows:

[0019] The BatchSize is set to 256, the learning rate is set to 0.001, the Epoch is set to 300, the optimizer is Adam, and 1% of the labeled pixels of each type of data in the Indian Pines dataset are randomly selected as the training set and input into the network model obtained in step 2, and finally a trained classification network model is obtained.

[0020] Step 4 is implemented according to the following steps:

[0021] 1% of the labeled pixels of each category of the Indian Pines dataset except the training set are randomly selected as the validation set, and are input into the classification network model obtained after training with the training set in the previous article, and the trained weight coefficients are brought in, and finally the hyperspectral classification result map and various indicator data are output.

[0022] The beneficial effect of the present invention is that, based on the feature weighted fusion hyperspectral image classification method of GAT and CNN, a local feature extraction branch with GAT as the core is first constructed, and superpixel segmentation is used to convert traditional image data into graph structure data, and the local features of the image are fully extracted; at the same time, a pixel-level feature extraction branch based on CNN is constructed, which can capture the pixel-level features of the image and effectively extract the overall information of the image. The superpixel-level graph structure data is combined with the pixel-level image data to construct a double-branch weighted fusion network, which realizes feature extraction and information fusion from pixel points to graph structure, and not only retains the spatial structure information, but also retains the complex relationship and local structure information between graph nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the overall flow chart of the method of the present invention;

[0024] Figure 2 This is a network framework diagram used in the present invention;

[0025] Figure 3 It is a framework diagram of the DYS module in the network model of the present invention;

[0026] Figure 4 It is the framework diagram of PAM and CAM modules of the present invention;

[0027] Figure 5 This is a comparison chart of the classification results of this method and other methods. DETAILED DESCRIPTION

[0028] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] The present invention is based on the feature weighted fusion method of GAT and CNN for hyperspectral image classification, the flow chart is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0030] Step 1: Divide the hyperspectral dataset into training set, test set and validation set;

[0031] Step 1 is implemented as follows:

[0032] Step 1.1: In order to comprehensively evaluate the performance of the model, the classic hyperspectral dataset Indian Pines was used as the model input for experiments. The dataset was preprocessed in the GAT and CNN branches respectively and divided into training set, test set and validation set. The training set was used to train the model parameters of the subsequent steps, the test set was used to test the effectiveness of the model, and the validation set was used to evaluate the generalization ability of the model during the training process and optimize the model hyperparameters. 1% of the total data was randomly selected for the training set and validation set, and the rest of the data was used as the test set.

[0033] Step 2: Construct a dual-branch feature weighted fusion network model combining CNN and GAT;

[0034] Step 2 is implemented as follows:

[0035] Step 2.1: The dual-branch weighted fusion model combining deep separable convolution and GAT contains two branches in total. Each branch has three parts, namely data preprocessing, multi-scale division and weighted fusion. The overall network framework is as follows: Figure 2 As shown in the figure. For the GAT branch, the original pixel image is preprocessed by superpixel segmentation to obtain superpixel information of different segmentation scales, and the obtained information is input into three GAT networks respectively and the result data is spliced ​​as the output of the GAT main branch. For the CNN branch, the spectral convolution layer is used to reduce the dimension and denoise the input original data, and the processed data is downsampled and input into the CNN branch and the output results are spliced ​​as the output of the CNN main branch.

[0036] Step 2.2: For the input data of the CNN branch, spectral convolution is used for preprocessing. First, the input image data is subjected to channel compression conversion, and then the converted data is standardized, and then passed through the spectral convolution layer with a convolution kernel size of 1×1. Finally, the LeakyReLU function is used to increase the generalization ability, and the processed data is used as the input of the CNN main branch. For the input data of the GAT branch, superpixel segmentation is used for preprocessing. Specifically, the original data is segmented into superpixels of different sizes to obtain segmentation information of different scales, and then the processed data is used as the input of the GAT main branch.

[0037] Step 2.3: For the multi-scale division of different branches, the CNN branch obtains data through the spectral convolution layer and performs two pooling operations with a kernel size of 2 and a maximum pooling step of 2 to obtain pixel-level information of three different scales. The three types of data are input into the three branches of the CNN network. Each CNN branch is divided into three layers, and each layer passes through the position attention module PAM, the channel attention module CAM and the depth-separable convolution layer in turn. The position attention module PAM is an attention mechanism used to capture the spatial position relationship in the image, and strengthens the model's understanding and processing capabilities of spatial information by weighting the features of each position. The position attention module PAM generates an attention map by calculating the similarity of different positions in the feature map, thereby emphasizing the key positions. The channel attention module CAM is an attention mechanism that focuses on the channel dimension of the feature map. By analyzing the importance of different channels and adjusting the response strength of each channel, the model can pay more attention to the feature channels that are more important to the final task. Depthwise separable convolution is an efficient convolution operation, which is divided into two steps: first, use point-by-point depthwise convolution to process each input channel separately, and then use channel-by-channel pointwise convolution to merge the output of depthwise convolution. This design greatly reduces the number of parameters and computational complexity of the model while maintaining or improving performance. The specific model structures of PAM and CAM attention modules are as follows: Figure 4 As shown. The output channels of each part of the first two layers of the first branch are 128, and the output channels of each part of the last layer are 128, 128, and 64 respectively. The output channels of the last two branches are halved in turn with reference to the pooled data. Finally, the output results of the last two branches are upsampled by the DYS module and spliced ​​with the output results of the first branch as the output of the CNN main branch; for the GAT branch, the processed image data is segmented at superpixel scales of 100, 50, and 10 and input into the three branches of GAT respectively. Each GAT branch consists of two layers of GAT networks, and the adjacency matrix A and superpixel position matrix Q obtained by superpixel segmentation are input. The output channels are 128 and 64 respectively. Finally, the outputs of the three branches of GAT are spliced ​​as the output of the overall GAT main branch;

[0038] Step 2.4: For the fusion of superpixel data and pixel data, multiply the output data of each GAT branch with the superpixel position matrix Q of its corresponding scale to convert it into output data of the same size and channel. Reshape the data of the CNN main branch into data of exactly the same size as the output of the GAT main branch for weighted fusion test, and use the fusion ratio with the best effect as the final network fusion parameter.

[0039] Step 3: input the training set divided in step 1 into the classification network model constructed in step 2, set parameters, and perform training to obtain training indicators;

[0040] Step 3 is implemented as follows:

[0041] The BatchSize is set to 256, the learning rate is set to 0.001, the Epoch is set to 300, the optimizer is Adam, and 1% of the labeled pixels of each type of data in the Indian Pines dataset are randomly selected as the training set and input into the network model obtained in step 2, and finally a trained classification network model is obtained.

[0042] Step 4: Evaluate the indicators, data, and image information of the classification results.

[0043] Step 4 is implemented according to the following steps:

[0044] 1% of the labeled pixels of each category of the Indian Pines dataset except the training set are randomly selected as the validation set, and are input into the classification network model obtained after training with the training set in the previous article, and the trained weight coefficients are brought in, and finally the hyperspectral classification result map and various indicator data are output.

[0045] Example 1

[0046] The present invention is based on the feature weighted fusion method of GAT and CNN for hyperspectral image classification, the flow chart is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0047] Step 1: Divide the hyperspectral data set into training set, test set and validation set;

[0048] Step 2: Construct a dual-branch feature weighted fusion network model combining CNN and GAT;

[0049] Step 3: input the training set divided in step 1 into the classification network model constructed in step 2, set parameters, and perform training to obtain training indicators;

[0050] Step 4: Evaluate the indicators, data, and image information of the classification results.

[0051] Example 2

[0052] The present invention is based on the feature weighted fusion method of GAT and CNN for hyperspectral image classification, the flow chart is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0053] Step 1: Divide the hyperspectral data set into training set, test set and validation set;

[0054] Step 1 is implemented as follows:

[0055] Step 1.1: In order to comprehensively evaluate the performance of the model, the classic hyperspectral dataset Indian Pines was used as the model input for experiments. The dataset was preprocessed in the GAT and CNN branches respectively and divided into training set, test set and validation set. The training set was used to train the model parameters of the subsequent steps, the test set was used to test the effectiveness of the model, and the validation set was used to evaluate the generalization ability of the model during the training process and optimize the model hyperparameters. 1% of the total data was randomly selected for the training set and validation set, and the rest of the data was used as the test set.

[0056] Step 2: Construct a dual-branch feature weighted fusion network model combining CNN and GAT;

[0057] Step 2 is implemented as follows:

[0058] Step 2.1: The dual-branch weighted fusion model combining deep separable convolution and GAT contains two branches in total. Each branch has three parts, namely data preprocessing, multi-scale division and weighted fusion. The overall network framework is as follows: Figure 2 As shown in the figure. For the GAT branch, the original pixel image is preprocessed by superpixel segmentation to obtain superpixel information of different segmentation scales, and the obtained information is input into three GAT networks respectively and the result data is spliced ​​as the output of the GAT main branch. For the CNN branch, the spectral convolution layer is used to reduce the dimension and denoise the input original data, and the processed data is downsampled and input into the CNN branch and the output results are spliced ​​as the output of the CNN main branch.

[0059] Step 2.2: For the input data of the CNN branch, spectral convolution is used for preprocessing. First, the input image data is subjected to channel compression conversion, and then the converted data is standardized, and then passed through the spectral convolution layer with a convolution kernel size of 1×1. Finally, the LeakyReLU function is used to increase the generalization ability, and the processed data is used as the input of the CNN main branch. For the input data of the GAT branch, superpixel segmentation is used for preprocessing. Specifically, the original data is segmented into superpixels of different sizes to obtain segmentation information of different scales, and then the processed data is used as the input of the GAT main branch.

[0060] Step 2.3: For the multi-scale division of different branches, the input data obtained by the CNN branch through the spectral convolution layer is pooled twice with a kernel size of 2 and a maximum pooling step of 2 to obtain pixel-level information of three different scales, and the three types of data are input into the three branches of the CNN network. Each CNN branch is divided into three layers, and each layer passes through the position attention module PAM, the channel attention module CAM and the depth-separable convolution layer in turn. The position attention module PAM is used to capture the spatial position relationship in the image, and strengthens the model's understanding and processing ability of spatial information by weighting the features of each position. The position attention module PAM calculates the similarity of different positions in the feature map to generate an attention map, thereby emphasizing the key positions. The channel attention module CAM analyzes the importance of different channels and adjusts the response strength of each channel, so that the model can pay more attention to the feature channels that are more important to the final task. The depth-separable convolution is divided into two steps: first, use point-by-point depth convolution to process each channel of the input separately, and then use channel-by-channel point-by-point convolution to merge the outputs of the depth convolution. The output channels of each part of the first two layers of the first branch are 128, and the output channels of each part of the last layer are 128, 128, and 64 respectively. The output channels of the last two branches are halved in turn with reference to the pooled data. Finally, the output results of the last two branches are spliced ​​with the output results of the first branch after passing through the DYS upsampling module. The DYS upsampling module uses point sampling to select specific sample points from a given sample set, and performs spatial rearrangement operations on the original image. Finally, the spatial structure of the image data is organized to generate a sampled image, and the spliced ​​result is used as the output of the CNN total branch; for the GAT branch, the processed image data is segmented at superpixel scales of 100, 50, and 10 and input into the three branches of GAT respectively. Each GAT branch consists of two layers of GAT networks, and the adjacency matrix A and superpixel position matrix Q obtained by superpixel segmentation are input. The output channels are 128 and 64 respectively. Finally, the outputs of the three branches of GAT are spliced ​​as the output of the overall GAT total branch;

[0061] Step 2.4: For the fusion of superpixel data and pixel data, multiply the output data of each GAT branch with the superpixel position matrix Q of its corresponding scale to convert it into output data of the same size and channel. Reshape the data of the CNN main branch into data of exactly the same size as the output of the GAT main branch for weighted fusion test, and use the fusion ratio with the best effect as the final network fusion parameter.

[0062] Step 3: input the training set divided in step 1 into the classification network model constructed in step 2, set parameters, and perform training to obtain training indicators;

[0063] Step 3 is implemented as follows:

[0064] The BatchSize is set to 256, the learning rate is set to 0.001, the Epoch is set to 300, the optimizer is Adam, and 1% of the labeled pixels of each type of data in the Indian Pines dataset are randomly selected as the training set and input into the network model obtained in step 2, and finally a trained classification network model is obtained.

[0065] Step 4: Evaluate the indicators, data, and image information of the classification results.

[0066] Step 4 is implemented according to the following steps:

[0067] The Indian Pines dataset is randomly selected from each category of data except the training set, and 1% of the labeled pixels are used as the validation set. The validation set is input into the classification network model obtained after training with the training set in the previous text, and the trained weight coefficient is brought in. Finally, the hyperspectral classification result map and various indicator data are output. The overall classification accuracy OA, average classification accuracy AA, and Kappa coefficient are recorded, and compared with the current classification methods as shown in Table 1 for analysis.

[0068] Example 3

[0069] The effect of the present invention is further described below in conjunction with simulation experiments:

[0070] This experiment was conducted on a computer equipped with Windows 10 (64-bit) system, Intel (R) Core (TM) i7-12700F, NVIDIA GeForce RTX 3060Ti,GPU and 16GB RAM.

[0071] Execute step 1. The data uses the Indian_pines dataset. Different types of data in this dataset have relatively similar spectral curves, and the sample categories are extremely unbalanced. The image spatial resolution used is 20m×20m, covering 145×145 pixels. The wavelength range is 0.4-2.5μm, including 220 continuous bands. After removing 20 water absorption bands and noise bands, 200 bands are retained. About half of the data (10,249 out of a total of 21,025 pixels) are labeled as 16 different categories;

[0072] Execute step 2 to build a dual-branch feature weighted fusion model of CNN and GAT;

[0073] Execute step 3 to set the training parameters for training. During the entire training process, set the BatchSize to 256, the learning rate to 0.001, the Epoch to 300, and the optimizer to Adam.

[0074] Execute step 4 and use the trained network model to classify the image data of the test set, and evaluate the indicators, data, and image information of the classification results.

[0075] In order to verify the effectiveness of the method proposed in this invention, the fusion results of the method of this invention are compared with three methods, namely: CEGCN, He3DCNN, and FDGC. The comparison chart of the classification results is shown, as shown in Figure 2. Figure 5 Table 1 shows the objective evaluation index results compared with the other three methods, including accuracy per category, OA (overall accuracy), AA (average accuracy), and Kappa (Kappa coefficient).

[0076] Table 1 Index values ​​of the fusion results of four classification methods on the Indian_pines dataset

[0077] CEGCN He3DCNN FDGC Ours Class1 9.59 35.50 26.61 97.30 Class2 8138 6840 8602 86.85 Class3 41.37 50.00 62.00 70.67 Class4 14.63 60.30 45.22 96.12 Class5 60.33 87.00 88.89 97.91 Class6 98.90 90.10 99.82 98.06 Class7 29.34 80.80 77.14 100 Class8 88.61 95.40 100 100 Class9 21.28 53.80 47.37 81.82 Class10 67.21 69.10 80.85 80.29 Class11 88.17 73.30 81.99 87.86 Class12 33.59 51.90 68.97 75.71 Class13 99.59 91.90 83.19 99.48 Class14 99.85 94.40 100 99.91 Class15 50.18 51.00 55.63 73.89 Class16 25.03 70.20 67.92 95.79 OA (%) 56.82 50.30 82.46 88.19 AA(%) 75.12 73.40 81.06 90.10 Kappa×100 71.09 69.50 78 87.97

[0078] After experimental verification, the proposed algorithm achieved excellent performance indicators in a simulated environment.

Claims

1. A hyperspectral image classification method based on feature weighted fusion of GAT and CNN, characterized in that: Follow the steps below to implement it: Step 1: Divide the hyperspectral data set into training set, test set and validation set; Step 2: Construct a dual-branch feature weighted fusion network model combining CNN and GAT; Step 3: input the training set divided in step 1 into the classification network model constructed in step 2, set parameters, and perform training to obtain training indicators; Step 4: Evaluate the indicators, data, and image information of the classification results.

2. The hyperspectral image classification method based on feature weighted fusion of GAT and CNN according to claim 1 is characterized in that: The step 1 is specifically implemented according to the following steps: Step 1.1, use the classic hyperspectral dataset Indian Pines as the model input for the experiment, preprocess the dataset in GAT and CNN branches respectively, and divide it into training set, test set and validation set. The training set and validation set are randomly selected from 1% of the total data, and the rest of the data is used as the test set.

3. The hyperspectral image classification method based on feature weighted fusion of GAT and CNN according to claim 2 is characterized in that: The step 2 is specifically implemented according to the following steps: Step 2.1, the dual-branch weighted fusion model combining deep separable convolution and GAT contains two branches in total, each branch has three parts, namely data preprocessing, multi-scale division and weighted fusion. For the GAT branch, the original pixel image is preprocessed by superpixel segmentation to obtain superpixel information of different segmentation scales, and the obtained information is input into three GAT networks respectively and the result data is spliced ​​as the output of the GAT main branch. For the CNN branch, the spectral convolution layer is used to perform dimensionality reduction and denoising on the input original data, and the processed data is downsampled and input into the CNN branch and the output results are spliced ​​as the output of the CNN main branch. Step 2.2: For the input data of the CNN branch, spectral convolution is used for preprocessing. First, the input image data is subjected to channel compression conversion, and then the converted data is standardized, and then passed through the spectral convolution layer with a convolution kernel size of 1×1. Finally, the LeakyReLU function is used to increase the generalization ability, and the processed data is used as the input of the CNN main branch. For the input data of the GAT branch, superpixel segmentation is used for preprocessing. Specifically, the original data is segmented into superpixels of different sizes to obtain segmentation information of different scales, and then the processed data is used as the input of the GAT main branch. Step 2.3: For the multi-scale division of different branches, the input data obtained by the CNN branch through the spectral convolution layer is pooled twice with a kernel size of 2 and a maximum pooling step of 2 to obtain pixel-level information of three different scales, and the three types of data are input into the three branches of the CNN network. Each CNN branch is divided into three layers, and each layer passes through the position attention module PAM, the channel attention module CAM and the depthwise separable convolution layer in turn. The position attention module PAM is used to capture the spatial position relationship in the image, and strengthens the model's understanding and processing ability of spatial information by weighting the features of each position. The position attention module PAM calculates the similarity of different positions in the feature map to generate an attention map, thereby emphasizing the key positions. The channel attention module CAM analyzes the importance of different channels and adjusts the response strength of each channel, so that the model can pay more attention to the feature channels that are more important to the final task. The depthwise separable convolution is divided into two steps: first, use point-by-point depthwise convolution to process each channel of the input separately, and then use point-by-point depthwise convolution to process each channel of the input separately. Channel point-by-point convolution merges the output of depth convolution. The output channels of each part of the first two layers of the first branch are 128, and the output channels of each part of the last layer are 128, 128, and 64 respectively. The output channels of the last two branches are halved in accordance with the pooled data. Finally, the output results of the last two branches are spliced ​​with the output results of the first branch after passing through the DYS upsampling module. The DYS upsampling module uses point sampling to select specific sample points from a given sample set, and performs spatial rearrangement operations on the original image. Finally, the spatial structure of the image data is organized to generate a sampled image, and the spliced ​​result is used as the output of the CNN total branch. For the GAT branch, the processed image data is segmented at superpixel scales of 100, 50, and 10 and input into the three branches of GAT respectively. Each GAT branch consists of two layers of GAT networks, and the adjacency matrix A and superpixel position matrix Q obtained by superpixel segmentation are input. The output channels are 128 and 64 respectively. Finally, the outputs of the three branches of GAT are spliced ​​as the output of the overall GAT total branch. Step 2.4: For the fusion of superpixel data and pixel data, multiply the output data of each GAT branch with the superpixel position matrix Q of its corresponding scale to convert it into output data of the same size and channel. Reshape the data of the CNN main branch into data of exactly the same size as the output of the GAT main branch for weighted fusion test, and use the fusion ratio with the best effect as the final network fusion parameter.

4. The hyperspectral image classification method based on feature weighted fusion of GAT and CNN according to claim 3 is characterized in that: The step 3 is specifically implemented according to the following steps: The BatchSize is set to 256, the learning rate is set to 0.001, the Epoch is set to 300, the optimizer is Adam, and 1% of the labeled pixels of each type of data in the Indian Pines dataset are randomly selected as the training set and input into the network model obtained in step 2, and finally a trained classification network model is obtained.

5. The hyperspectral image classification method based on feature weighted fusion of GAT and CNN according to claim 4 is characterized in that: The step 4 is specifically implemented according to the following steps: 1% of the labeled pixels of each category of the Indian Pines dataset except the training set are randomly selected as the validation set, and are input into the classification network model obtained after training with the training set in the previous article, and the trained weight coefficients are brought in, and finally the hyperspectral classification result map and various indicator data are output.

Images