A hyperspectral image classification method and device

Abstract

The application discloses a hyperspectral image classification method and device, belongs to the technical field of image processing classification, combines the graph neural network technology suitable for small sample conditions and the convolutional neural network technology suitable for sample sufficient conditions, proposes a center weighted convolution network model based on graph neural network assistance, trains and cross-verify the model, can effectively fuse the local space spectrum information extracted by the convolutional neural network and the remote context information extracted by the graph neural network on the basis that the model realizes fine classification of the hyperspectral image under limited samples. Compared with other prior art, the classification precision under a small amount of training samples can be obviously improved, the demand for initial labeled samples is not large, the time cost caused by manual labeling of class information can be greatly saved, and the robustness can be guaranteed, and a stable classification model with considerable training effect can be obtained through a small amount of samples. The application is suitable for small sample conditions of hyperspectral image classification.

Description

Technical Field

[0001] This application relates to a hyperspectral image classification method and apparatus, belonging to the field of image processing and classification technology. Background Technology

[0002] Hyperspectral image data is a combination of spectral band data and spatial location data, containing rich and complex information on land cover categories. The purpose of hyperspectral image classification research is to fully extract land cover information using relevant classification theories and techniques, improve hyperspectral image classification accuracy, and provide a solid and reliable foundation for subsequent hyperspectral image applications. Currently, in the field of image processing classification, convolutional neural network frameworks based on attention mechanisms are widely used in hyperspectral image classification due to their superior performance. Although their classification results are good, some problems still exist in practical hyperspectral classification applications. Since convolutional neural networks require a large number of training samples, and the number of labeled samples available for use in hyperspectral datasets is relatively small, manually acquiring additional labeled sample information is usually necessary to solve this problem, consuming significant human and material resources. This is a major challenge that most supervised classification methods currently face when applied to hyperspectral image data classification. Therefore, how to improve classification accuracy with a small number of labeled samples and reduce the cost of manually labeling surface information is a key area of ​​focus and a hot topic. In existing research, hyperspectral classification methods for small sample cases are usually based on improving the neural network model on the basis of the convolutional neural network framework, or using small samples to train a classifier that can extract the vacuous features of hyperspectral images. Although these methods can classify hyperspectral images with small sample cases, the accuracy is low and the classification effect is not good. Summary of the Invention

[0003] The purpose of this application is to provide a hyperspectral image classification method and apparatus. In order to solve the problem of classification accuracy of hyperspectral image datasets with a small number of training samples, a center-weighted convolutional network model based on graph neural network assistance is designed to achieve fine classification of hyperspectral images with limited samples by extracting local ground feature information and remote context information and effectively fusing them.

[0004] To achieve the above objectives, the first aspect of this application provides a hyperspectral image classification method, comprising:

[0005] Obtain the dataset and divide it into a training set, a validation set, and a test set according to a preset partitioning method. The dataset includes hyperspectral data.

[0006] Design a center-weighted convolutional network model based on graph neural network assistance and adjust the model parameters to obtain several center-weighted convolutional network models with different model parameters. The above center-weighted convolutional network models are used to extract local spatial spectral information and mid-to-long-range context information from the above dataset, and to perform feature fusion on the above local land cover information and long-range context information.

[0007] Based on the above training set, several of the above-mentioned center-weighted convolutional network models were trained respectively;

[0008] The above validation set is used to cross-validate several of the above-mentioned center-weighted convolutional network models during the training process to obtain validation results, which include the classification performance of several of the above-mentioned center-weighted convolutional network models.

[0009] Based on the above verification results, the center-weighted convolutional network model with the best classification performance was selected as the target classification model, and the hyperspectral images were classified using the target classification model.

[0010] In one implementation, after determining the center-weighted convolutional network model with the best classification performance as the target classification model, the method further includes:

[0011] The classification performance of the target classification model was tested using the test set described above.

[0012] In one implementation, the above-mentioned acquisition of the dataset includes:

[0013] Obtain the raw hyperspectral image data and label the category information of the above data;

[0014] Principal component analysis was used to reduce the dimensionality of the labeled data, and the resulting data was used as the hyperspectral data mentioned above.

[0015] In one implementation, the aforementioned datasets include the Indian Pines dataset, the Salinas Valley dataset, and the WHU-Hi-HongHu dataset;

[0016] The training of several center-weighted convolutional network models based on the training set described above includes:

[0017] Several of the aforementioned center-weighted convolutional network models were trained using training sets obtained from the Indian Pines dataset, Salinas Valley dataset, and WHU-Hi-HongHu dataset according to a preset partitioning method.

[0018] In one implementation, the above-described center-weighted convolutional network model includes: a graph neural network branch, a center-weighted convolutional branch, and a Softmax classifier;

[0019] The cross-validation of several of the aforementioned center-weighted convolutional network models during training using the aforementioned validation set includes:

[0020] The following steps are performed during the training of several of the above-mentioned center-weighted convolutional network models:

[0021] The mid-to-long-range context information of the above validation set is obtained through the above graph neural network branch;

[0022] The local spatial spectrum information of the above validation set is obtained through the above center-weighted convolution branch;

[0023] The aforementioned mid-to-long-range context information and the aforementioned local spatial spectrum information are fused to obtain the fused feature information.

[0024] Based on the fused feature information, the classification performance of the center-weighted convolutional network model on the validation set is obtained through the Softmax classifier.

[0025] In one embodiment, the above graph neural network branch includes: a graph encoder, a graph neural network, and a graph decoder;

[0026] The aforementioned acquisition of mid-to-long-range context information for the validation set through the aforementioned graph neural network branch includes:

[0027] Superpixels are obtained by performing superpixel segmentation on the hyperspectral data in the above validation set using SLIC;

[0028] The above graph encoder is used to construct the relationship between the superpixels and pixels;

[0029] The superpixels are input as graph nodes into the graph neural network to obtain the mid-to-long-range context information of the superpixels.

[0030] The superpixels are restored to pixel-level hyperspectral data using the aforementioned image decoder, thus obtaining the mid-to-long-range context information of the hyperspectral data in the aforementioned validation set.

[0031] In one implementation, the aforementioned center-weighted convolutional branch includes a spatial spectral feature extractor and a global average pooling layer;

[0032] The local spatial spectral information of the validation set obtained through the aforementioned center-weighted convolution branch includes:

[0033] Obtain local three-dimensional data blocks from the hyperspectral data of the above validation set;

[0034] The above three-dimensional data block is input into the above spatial spectral feature extractor. In the above spatial spectral feature extractor, spatial spectral information is extracted by center-weighted convolution, spectral information is extracted by one-dimensional convolution, and the weight of the center key pixel is increased by the center attention module.

[0035] The three-dimensional data block after passing through the above spatial spectral feature extractor is input into the above global average pooling layer to obtain a one-dimensional feature vector with local spatial spectral information. The above one-dimensional feature vector is used as the local spatial spectral information of the above validation set.

[0036] A second aspect of this application provides a hyperspectral image classification apparatus, comprising:

[0037] The acquisition module is used to acquire the dataset and divide the dataset into training set, validation set and test set according to a preset partitioning method. The dataset includes hyperspectral data.

[0038] The design module is used to design a center-weighted convolutional network model based on graph neural network assistance and adjust the model parameters to obtain several center-weighted convolutional network models with different model parameters. The center-weighted convolutional network models are used to extract local spatial spectral information and mid-to-long-range context information from the dataset and to perform feature fusion on the local land cover information and long-range context information.

[0039] The training module is used to train several of the above-mentioned center-weighted convolutional network models based on the above training set.

[0040] The validation module is used to perform cross-validation on several of the above-mentioned center-weighted convolutional network models during the training process using the above-mentioned validation set, and obtain the validation results, wherein the above-mentioned validation results include the classification performance of several of the above-mentioned center-weighted convolutional network models;

[0041] The classification module is used to determine the center-weighted convolutional network model with the best classification effect as the target classification model based on the above verification results, and to classify the hyperspectral image using the target classification model.

[0042] A third aspect of this application provides a hyperspectral image classification apparatus, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the first aspect or any embodiment of the first aspect.

[0043] The fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the first aspect or any embodiment of the first aspect.

[0044] As can be seen from the above, this application provides a hyperspectral image classification method and apparatus, combining graph neural network technology suitable for small sample situations and convolutional neural network technology suitable for sufficient sample situations. It proposes a center-weighted convolutional network model based on graph neural network assistance. Training and cross-validating this model allows for the effective fusion of local spatial spectral information extracted by the convolutional neural network and long-range contextual information extracted by the graph neural network, achieving fine classification of hyperspectral images with limited samples. This application designs a novel deep learning model based on image processing and deep learning technologies. Compared with other existing technologies, the proposed model significantly improves classification accuracy with a small number of training samples, requires a small number of initial labeled samples, and greatly saves the time cost of manually labeling category information, thus having practical significance. Furthermore, the robustness of the model is guaranteed; a stable and effective classification model can be trained with a small number of samples, and the time cost is within an acceptable range. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating a hyperspectral image classification method provided in an embodiment of this application;

[0047] Figure 2 This is a schematic diagram of the structure of a center-weighted convolutional network model based on graph neural network assistance, provided in an embodiment of this application. Detailed Implementation

[0048] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0049] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0050] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0052] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0053] Example 1

[0054] This application provides a hyperspectral image classification method, such as... Figure 1 As shown, the method includes:

[0055] Step 11: Obtain the dataset and divide it into a training set, a validation set, and a test set according to a preset partitioning method. The dataset includes hyperspectral data.

[0056] Optionally, obtaining the dataset mentioned above includes:

[0057] The raw hyperspectral image data is acquired, and the category information of the above data is labeled as key prior information for the subsequent classification model to extract category information.

[0058] Principal component analysis (PCA) is used to reduce the dimensionality of the labeled data, and the resulting data is used as the hyperspectral data mentioned above. Specifically, since the original hyperspectral data is large in volume, it is generally impossible to load it directly for training. Therefore, PCA is used for dimensionality reduction, which significantly reduces the amount of data to be processed by retaining the most important non-overlapping information, without negatively impacting the final classification accuracy.

[0059] Step 12: Design a center-weighted convolutional network model based on graph neural network assistance and adjust the model parameters to obtain several center-weighted convolutional network models with different model parameters. The center-weighted convolutional network model is used to extract local spatial spectral information and mid-to-long-range context information from the dataset and to perform feature fusion on the local land cover information and long-range context information.

[0060] Optional, such as Figure 2 As shown, the aforementioned center-weighted convolutional network model includes two branches and a classifier. Branch one, called the graph neural network branch, is used to extract long-range contextual information of the hyperspectral data. It includes a graph encoder, a graph neural network (also known as graph sampling and aggregation, GraphSAGE), and a graph decoder. Branch two, called the center-weighted convolutional branch, is used to extract local spatial spectral information of the hyperspectral data. It includes a spatial spectral feature extractor and a global average pooling layer (GAP). In practical applications, when training the model, the input hyperspectral data can first be dimensionality-reduced using PCA to remove redundant spectral information. A graph neural network is then constructed using the graph encoder in branch one. Local data blocks are extracted from branch two and fed into the network model for training. Finally, the information extracted from branch one and branch two is fused and input into a Softmax classifier for classification.

[0061] Specifically, for the graph neural network branch, the input is the entire hyperspectral data after PCA dimensionality reduction. Considering that directly using pixels as graph nodes would result in an unbearable computational burden, the hyperspectral data is first segmented into superpixels using SLIC. Superpixels obtained according to an appropriate segmentation ratio can be used as nodes in the graph neural network branch. A graph encoder is used to build the connections between pixels and superpixels. Then, GraphSAGE in the graph neural network branch is selected to extract mid-to-long-range contextual information. GraphSAGE has advantages over traditional graph neural networks on large-scale graph data. Optionally, the graph neural network branch consists of two stacked GraphSAGE layers. After feature information is extracted by GraphSAGE, a graph decoder is used to restore the superpixels to pixel-level hyperspectral data, thus obtaining the mid-to-long-range contextual information of the training samples. By setting the graph neural network, accurate prediction of nodes can be achieved. Since each superpixel obtained by SLIC segmentation is composed of similar pixels, accurate prediction of nodes improves the accuracy of pixel prediction in the entire superpixel region, which is also key to improving classification accuracy when samples are insufficient.

[0062] For the center-weighted convolution branch, local data (or local 3D data blocks) is first extracted from the hyperspectral data after PCA dimensionality reduction, and then fed into the spatial spectral feature extractor. In the spatial spectral feature extractor, spatial spectral information is extracted through center-weighted convolution (CW-Conv), spectral information is extracted through one-dimensional convolution, and then the weight of the center key pixel is further increased through the center attention module (CAM). Since hyperspectral image classification predicts its pixels, increasing the weight of the center pixel can improve the network's performance and stability. After passing through the spatial spectral feature extraction module, a one-dimensional feature vector containing local spatial spectral and spectral information is obtained through GAP. Finally, the information obtained from the two branches is effectively fused, and the final classification prediction result is obtained using a Softmax classifier. Optionally, in practical applications, spatial spectral and spectral information can be directly extracted through the above spatial spectral feature extractor, or spatial and spectral feature extractors can be set to extract spatial and spectral information separately; this is not limited here.

[0063] Since graph neural networks (GNNs) perform well in small sample sizes, while convolutional neural networks (CNNs) perform well when there are sufficient labeled samples, this embodiment combines the characteristics of both networks to establish a small-sample classifier, namely the aforementioned center-weighted convolutional network model. This model achieves fine classification of hyperspectral images with a limited number of training samples, based on capturing local spatial-spectral information and mid-to-long-range contextual information. To further improve the model's stability and classification accuracy, this embodiment uses CW-Conv instead of ordinary convolution in the center-weighted convolution branch and designs a CAM module to enhance the status of the center pixel, allowing the network to learn the relationship between the center key pixel and surrounding pixels, thus reflecting the importance of the center key pixel. Feature fusion is performed between the local spatial-spectral information extracted by the center-weighted convolution branch and the mid-to-long-range contextual information extracted by the graph neural network branch, combining the advantages of both convolutional neural networks and graph neural networks to achieve high-precision classification results regardless of whether there are sufficient or insufficient samples.

[0064] Step 13: Train several of the above-mentioned center-weighted convolutional network models based on the training set;

[0065] Optionally, the aforementioned datasets include the Indian Pines dataset, the Salinas Valley dataset, and the WHU-Hi-HongHu dataset;

[0066] The training of several center-weighted convolutional network models based on the training set described above includes:

[0067] Several of the aforementioned center-weighted convolutional network models were trained using training sets obtained from the Indian Pines (IP) dataset, Salinas Valley (SV) dataset, and WHU-Hi-HongHu (WHU-Hi) dataset according to a preset partitioning method.

[0068] In one application scenario, to extract the joint spatial-spectral information of hyperspectral images, the hyperspectral data can be divided into three-dimensional data blocks before dividing the dataset into training, validation, and test sets. These three-dimensional data blocks are then proportionally divided into training, validation, and test sets. Specifically, a 9×9×S block can be selected as the input to the center-weighted convolutional branch, where S is the spectral dimension after dimensionality reduction using principal component analysis. The superpixel segmentation ratio is set to 100, meaning each superpixel consists of 100 pixels. The resulting superpixels are then used as graph nodes input to the graph neural network. For the training set, different proportions can be used for different datasets, selecting the lowest possible proportion. For example, only 5% of the data samples in the IP dataset can be used as the training set, only 1% in the SV dataset, and only 0.5% in the WHU-Hi-HongHu dataset. After determining the structure, corresponding model parameters, and dataset partitioning method of the center-weighted convolutional network model using the above method, data from different training datasets can be fed into several center-weighted convolutional network models for training.

[0069] Step 14: Cross-validate several of the above-mentioned center-weighted convolutional network models during the training process using the above-mentioned validation set to obtain validation results, wherein the above-mentioned validation results include the classification performance of several of the above-mentioned center-weighted convolutional network models;

[0070] Optionally, when the aforementioned center-weighted convolutional network model includes: a graph neural network branch, a center-weighted convolutional branch, and a Softmax classifier, the aforementioned cross-validation of several of the aforementioned center-weighted convolutional network models during training using the aforementioned validation set includes:

[0071] The following steps are performed during the training of several of the above-mentioned center-weighted convolutional network models:

[0072] The mid-to-long-range context information of the above validation set is obtained through the above graph neural network branch;

[0073] The local spatial spectrum information of the above validation set is obtained through the above center-weighted convolution branch;

[0074] The aforementioned mid-to-long-range context information and the aforementioned local spatial spectrum information are fused to obtain the fused feature information.

[0075] Based on the fused feature information, the classification performance of the center-weighted convolutional network model on the validation set is obtained using the Softmax classifier. Cross-validating the center-weighted convolutional network model on the validation set during training ensures that the final trained model achieves the most accurate classification results.

[0076] Optionally, when the above graph neural network branch includes: a graph encoder, a graph neural network, and a graph decoder, the acquisition of the mid-to-long-range context information of the above validation set through the above graph neural network branch includes:

[0077] Superpixels are obtained by performing superpixel segmentation on the hyperspectral data in the above validation set using SLIC;

[0078] The above graph encoder is used to construct the relationship between the superpixels and pixels;

[0079] The superpixels are input as graph nodes into the graph neural network to obtain the mid-to-long-range context information of the superpixels.

[0080] The superpixels are restored to pixel-level hyperspectral data using the aforementioned image decoder, thus obtaining the mid-to-long-range context information of the hyperspectral data in the aforementioned validation set.

[0081] Optionally, when the aforementioned center-weighted convolutional branch includes a spatial spectral feature extractor and a global average pooling layer, the acquisition of the local spatial spectral information of the validation set through the aforementioned center-weighted convolutional branch includes:

[0082] Obtain local three-dimensional data blocks from the hyperspectral data of the above validation set;

[0083] The above three-dimensional data block is input into the above spatial spectral feature extractor. Spatial spectral information and spectral information are obtained by center-weighted convolution and one-dimensional convolution in the spatial spectral feature extractor, respectively. The weight of the center key pixel is increased by the center attention module.

[0084] The three-dimensional data block after passing through the above spatial spectral feature extractor is input into the above global average pooling layer to obtain a one-dimensional feature vector with local spatial spectral information and spectral information. The above one-dimensional feature vector is used as the local spatial spectral information of the above validation set.

[0085] Step 15: Based on the above verification results, the center-weighted convolutional network model with the best classification effect is determined as the target classification model, and the hyperspectral image is classified using the target classification model.

[0086] In one application scenario, center-weighted convolutional network models with different model parameters are trained on the training set and then cross-validated on the validation set to obtain the validation results and select the model with the best classification performance. Specifically, the model parameters (weight data) with the best performance can be determined based on the minimum loss value of different models on the validation set during training and saved. The center-weighted convolutional network model with these model parameters is then determined as the target classification model to achieve accurate classification of hyperspectral images in the case of small samples.

[0087] Optionally, after determining the center-weighted convolutional network model with the best classification performance as the target classification model, the process further includes: testing the classification performance of the target classification model using the test set. Specifically, the target classification model is used on the test set to measure metrics such as accuracy, stability, and model classification time, and a complete classification map is plotted to verify the classification performance of the target classification model and further ensure the model's accuracy and robustness.

[0088] As can be seen from the above, this application provides a hyperspectral image classification method that combines graph neural network technology, which is suitable for small sample situations, with convolutional neural network technology, which is suitable for sufficient sample situations. It proposes a center-weighted convolutional network model based on graph neural network assistance. Training and cross-validating this model allows for the effective fusion of local spatial spectral information extracted by the convolutional neural network and long-range contextual information extracted by the graph neural network, achieving fine classification of hyperspectral images with limited samples. This application also designs a novel deep learning model based on image processing and deep learning technologies. Compared to other existing technologies, the model proposed in this application significantly improves classification accuracy with a small number of training samples, requires a small number of initial labeled samples, and greatly saves the time cost of manually labeling category information, thus having practical significance. Furthermore, the robustness of the model is guaranteed; a stable and effective classification model can be trained with a small number of samples, and the time cost is within an acceptable range.

[0089] Example 2

[0090] To verify the superiority and effectiveness of the hyperspectral image classification method provided in this application, comparative experiments were conducted on three common hyperspectral datasets: Indian Pines, Salinas Valley, and WHU-Hi-HongHu. The batch size was set to 16, each training iteration lasted 200 epochs, and the learning rate was set to 0.005. The classification performance of each method was measured using the three most widely used quantitative metrics: overall accuracy (OA), average accuracy (AA), and Kappa coefficient (K). Ten independent experiments were conducted, and the average value was taken. For the Indian Pines dataset, only 5% of the samples were used as the training set, 1% as the test set, and the remaining samples were used for testing. For the Salinas Valley dataset, 1% of the samples were used as both the training and validation sets, and the remaining samples were used as the test set. For the WHU-Hi-HongHu dataset, due to its large sample size, 0.5% of the samples were selected as the training set, 0.1% as the validation set, and the remaining samples as the test set. Experimental results on the three datasets are shown in Table 1.

[0091] Table 1 Comparison of Classification Accuracy Results

[0092]

[0093] Data shows that the method in this application embodiment has significant advantages when the training samples are small, and exhibits higher stability than other methods. In summary, the hyperspectral image classification method described in this application embodiment is an advanced image classification system with a small number of samples.

[0094] Example 3

[0095] This application provides a hyperspectral image classification device, which includes an acquisition module, a design module, a training module, a verification module, and a classification module.

[0096] The acquisition module is used to acquire the dataset and divide the dataset into a training set, a validation set, and a test set according to a preset partitioning method. The dataset includes hyperspectral data.

[0097] The design module is used to design a center-weighted convolutional network model based on graph neural network assistance and adjust the model parameters to obtain several center-weighted convolutional network models with different model parameters. The center-weighted convolutional network model is used to extract local spatial spectral information and mid-to-long-range context information of the dataset, and to perform feature fusion on the local land cover information and long-range context information.

[0098] The training module is used to train several of the above-mentioned center-weighted convolutional network models based on the training set.

[0099] The validation module is used to perform cross-validation on several of the above-mentioned center-weighted convolutional network models during the training process using the above-mentioned validation set, and obtain the validation results, wherein the above-mentioned validation results include the classification performance of several of the above-mentioned center-weighted convolutional network models;

[0100] The classification module is used to determine the center-weighted convolutional network model with the best classification effect as the target classification model based on the above verification results, and to classify the hyperspectral image using the target classification model.

[0101] Optionally, the hyperspectral image classification device further includes a testing module for testing the classification performance of the target classification model using the test set.

[0102] As can be seen from the above, this application provides a hyperspectral image classification device that combines graph neural network technology, which is suitable for small sample situations, with convolutional neural network technology, which is suitable for sufficient sample situations. It proposes a center-weighted convolutional network model based on graph neural network assistance. Training and cross-validating this model allows for the effective fusion of local spatial spectral information extracted by the convolutional neural network and long-range contextual information extracted by the graph neural network, achieving fine classification of hyperspectral images with limited samples. This application also designs a novel deep learning model based on image processing and deep learning technologies. Compared to other existing technologies, the model proposed in this application significantly improves classification accuracy with a small number of training samples, requires a small number of initial labeled samples, and greatly saves the time cost of manually labeling category information, thus having practical significance. Furthermore, the robustness of the model is guaranteed; a stable and effective classification model can be trained with a small number of samples, and the time cost is within an acceptable range.

[0103] Example 4

[0104] This application provides a hyperspectral image classification device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The memory stores software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory and processor are connected via a bus. Specifically, the processor implements any of the steps in Embodiment 1 by running the computer program stored in the memory.

[0105] It should be understood that, in the embodiments of this application, the processor may be a Central Processing Unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0106] Memory may include read-only memory, flash memory, and random access memory, and provides instructions and data to the processor. Some or all of the memory may also include non-volatile random access memory.

[0107] As can be seen from the above, this application provides a hyperspectral image classification device that combines graph neural network technology, which is suitable for small sample situations, with convolutional neural network technology, which is suitable for sufficient sample situations. It proposes a center-weighted convolutional network model based on graph neural network assistance. Training and cross-validating this model allows for the effective fusion of local spatial spectral information extracted by the convolutional neural network and long-range contextual information extracted by the graph neural network, achieving fine classification of hyperspectral images with limited samples. This application also designs a novel deep learning model based on image processing and deep learning technologies. Compared to other existing technologies, the model proposed in this application significantly improves classification accuracy with a small number of training samples, requires a small number of initial labeled samples, and greatly saves the time cost of manually labeling category information, thus having practical significance. Furthermore, the robustness of the model is guaranteed; a stable and effective classification model can be trained with a small number of samples, and the time cost is within an acceptable range.

[0108] It should be understood that if the integrated modules / units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0109] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0110] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0111] It should be noted that the methods and detailed examples provided in the above embodiments can be incorporated into the apparatus and devices provided in the embodiments, and can be referred to each other, without further elaboration.

[0112] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0113] In the embodiments provided in this application, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of the modules or units described above is merely a logical functional division, and in actual implementation, it can be divided in other ways. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0114] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method of hyperspectral image classification, characterized in that, The method comprises the following steps: acquiring a data set and dividing the data set into a training set, a validation set and a test set according to a preset division mode, wherein the data set comprises hyperspectral data; designing a center-weighted convolution network model based on a graph neural network and adjusting model parameters to obtain a plurality of center-weighted convolution network models with different model parameters, wherein the center-weighted convolution network model is used to extract local spatial-spectral information and long-range context information of the data set, and to perform feature fusion on the local spatial-spectral information and long-range context information; training the plurality of center-weighted convolution network models based on the training set; cross-validating the plurality of center-weighted convolution network models during the training process through the validation set to obtain a validation result, wherein the validation result comprises the classification effect of the plurality of center-weighted convolution network models; determining the center-weighted convolution network model with the best classification effect as a target classification model based on the validation result, and classifying a hyperspectral image through the target classification model; the step of acquiring the data set comprises: acquiring data of an original hyperspectral image and labeling the category information of the data; performing dimension reduction on the labeled data based on a principal component analysis method, and taking the data obtained after dimension reduction as the hyperspectral data; the center-weighted convolution network model comprises a graph neural network branch, a center-weighted convolution branch and a Softmax classifier; the step of cross-validating the plurality of center-weighted convolution network models during the training process through the validation set comprises: performing the following steps during the training process of the plurality of center-weighted convolution network models: acquiring long-range context information of the validation set through the graph neural network branch; acquiring local spatial-spectral information of the validation set through the center-weighted convolution branch; performing feature fusion on the long-range context information and the local spatial-spectral information to obtain fused feature information; obtaining the classification effect of the center-weighted convolution network model on the validation set based on the fused feature information through the Softmax classifier.

2. The hyperspectral image classification method of claim 1, wherein, after determining the center-weighted convolution network model with the best classification effect as the target classification model, the method further comprises: testing the classification effect of the target classification model through the test set.

3. The hyperspectral image classification method of claim 1, wherein, the data set comprises an IndianPines data set, a Salinas Valley data set and a WHU-Hi-HongHu data set; the step of training the plurality of center-weighted convolution network models based on the training set comprises: training the plurality of center-weighted convolution network models based on the corresponding training sets obtained from the Indian Pines data set, the Salinas Valley data set and the WHU-Hi-HongHu data set according to the preset division mode.

4. The hyperspectral image classification method of claim 1, wherein, the graph neural network branch comprises a graph encoder, a graph neural network and a graph decoder; the step of acquiring long-range context information of the validation set through the graph neural network branch comprises: performing superpixel segmentation on the hyperspectral data in the verification set by SLIC to obtain superpixels; building a connection between the superpixels and pixels by the graph encoder; inputting the superpixels into the graph neural network as graph nodes to obtain the mid-long range context information of the superpixels; restoring the superpixels into pixel-level hyperspectral data by the graph decoder to obtain the mid-long range context information of the hyperspectral data in the verification set.

5. The method of hyperspectral image classification of claim 1, wherein, The center weighted convolution branch includes a spatial-spectral feature extractor and a global average pooling layer. Then, the local spatial-spectral information of the verification set is obtained by the center weighted convolution branch, including: obtaining a local three-dimensional data block in the hyperspectral data of the verification set; inputting the three-dimensional data block into the spatial-spectral feature extractor, in which the spatial-spectral information is extracted by center weighted convolution, the spectral information is extracted by one-dimensional convolution, and the weight of the center key pixel is improved by a center attention module; inputting the three-dimensional data block after the spatial-spectral feature extractor into the global average pooling layer to obtain a one-dimensional feature vector with local spatial-spectral information, and the one-dimensional feature vector is taken as the local spatial-spectral information of the verification set.

6. A hyperspectral image classification apparatus, characterized by including: an acquisition module configured to acquire a data set and divide the data set into a training set, a verification set and a test set according to a preset division manner, wherein the data set includes hyperspectral data; a design module configured to design a center weighted convolution network model based on a graph neural network and adjust model parameters to obtain a plurality of center weighted convolution network models with different model parameters, wherein the center weighted convolution network model is used to extract local spatial-spectral information and mid-long range context information of the data set, and to perform feature fusion on the local spatial-spectral information and the long-range context information; a training module configured to train a plurality of center weighted convolution network models based on the training set; a verification module configured to cross-validate a plurality of center weighted convolution network models during the training process by using the verification set to obtain a verification result, wherein the verification result includes the classification effect of a plurality of center weighted convolution network models; a classification module configured to determine the center weighted convolution network model with the best classification effect as a target classification model according to the verification result, and to classify hyperspectral images by using the target classification model; the acquisition of the data set includes: acquiring data of an original hyperspectral image and labeling the category information of the data; performing dimensionality reduction on the labeled data based on a principal component analysis method, and taking the data obtained after dimensionality reduction as the hyperspectral data; the center weighted convolution network model includes a graph neural network branch, a center weighted convolution branch and a Softmax classifier; the cross-validation of a plurality of center weighted convolution network models during the training process by using the verification set includes: the following steps are performed during the training process of a plurality of center weighted convolution network models: obtaining the mid-long range context information of the verification set by using the graph neural network branch; obtaining local spatial-spectral information of the verification set through the center weighted convolution branch; performing feature fusion on the mid-long range context information and the local spatial-spectral information to obtain fused feature information; obtaining a classification effect of the center weighted convolution network model on the verification set through the Softmax classifier based on the fused feature information.

7. A hyperspectral image classification apparatus comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 5.

Images