Hyperspectral image classification method and device, electronic equipment and medium

Abstract

The invention provides a hyperspectral image classification method, which comprises the following steps of: inputting a true value image containing a hyperspectral image and a ground feature corresponding to the hyperspectral image, and dividing the hyperspectral image into a training set and a test set; the method comprises the steps of clustering pixels in a hyperspectral image, segmenting each clustering region to generate a plurality of segmented regions, generating a segmented region and pixel mapping relation matrix according to pixel distribution of each segmented region, and constructing a pixel node adjacency matrix and a superpixel node adjacency matrix; constructing a double-branch graph convolutional network; the double-branch graph convolutional network is trained; and predicting a label of the test set data through the trained double-branch graph convolutional network, and completing classification of the hyperspectral image. According to the method, the global and local features of the image can be accurately extracted, the fineness and accuracy of feature extraction are further improved through the arrangement of pixel-level feature extraction branches, and a solid foundation is provided for high-precision classification.

Description

Technical Field

[0001] The present invention relates to the field of image or video recognition or understanding, and in particular to a hyperspectral image classification method, device, electronic equipment and medium. Background Art

[0002] As a data cube structure, hyperspectral images integrate the spatial distribution information of objects and the spectral reflectance and radiation characteristics in hundreds of approximately continuous bands. This data structure provides a rich source of information for many fields such as environmental monitoring, geological exploration, and medical testing. By deeply analyzing the spectral and spatial information in hyperspectral images and utilizing the spectral similarities of similar objects and the spectral differences of heterogeneous objects, high-precision object target recognition and classification can be achieved, providing a solid theoretical foundation and technical support for research in related fields.

[0003] In the early hyperspectral image classification technology, it mainly relied on pixel-level spectral information and adopted machine learning algorithms such as Support Vector Machines (SVM) and Random Forest (RF). SVM achieves category division by constructing the optimal classification hyperplane in the high-dimensional feature space, while RF improves the accuracy and stability of classification by integrating multiple decision trees. However, these traditional methods are difficult to effectively capture deep feature representations when processing hyperspectral images due to the limitations of the model structure, and are prone to dimensionality disasters in high-dimensional data environments, thus limiting the improvement of classification performance.

[0004] With the rapid development of deep learning technology, Convolutional Neural Network (CNN) and Graph Convolutional Network (GCN) have gradually emerged in the field of hyperspectral image classification. With its local connection and weight sharing characteristics, CNN can efficiently extract the spectral features of multiple pixels in the local space, and improve the model's understanding of image features by constructing local spatial filters of different sizes. In addition, the introduction of residual network structure can further alleviate the problems of gradient disappearance and performance degradation, and improve the training efficiency and classification performance of the network. However, when processing hyperspectral images, the fixed size and fixed weight of CNN's convolution kernel limits its ability to process global spatial features and adapt to the distribution of features in different regions.

[0005] In contrast, GCN treats pixels in hyperspectral images as nodes and aggregates similar spectral features in the global space through the adjacency relationship between nodes, thereby improving classification accuracy. In order to reduce the computational burden, GCN usually operates at the superpixel level, but this may lead to the loss of subtle pixel features and feature smoothing problems, which in turn affects classification accuracy. To compensate for this deficiency, the ability of CNN to extract multiple pixel features in local regular regions can be combined, but the incompatibility of CNN and GCN in data representation structure poses challenges to the effective propagation and fusion of features. Although this problem can be alleviated by constructing a specific conversion network, this not only increases the complexity of the network, but may also lead to the loss of complementary information between pixels and superpixels due to the limited size of the training set.

[0006] Therefore, how to achieve deep fusion and efficient utilization of spectral and spatial information in hyperspectral images has become a technical problem that needs to be urgently solved in current hyperspectral image classification technology. Summary of the invention

[0007] In view of this, the embodiments of the present disclosure provide a hyperspectral image classification method, device, electronic device and medium, which at least partially solve the problems existing in the prior art.

[0008] In a first aspect, the present disclosure provides a hyperspectral image classification method, which includes: Input includes a hyperspectral image and a ground object truth map corresponding to the hyperspectral image, and divides the hyperspectral image into a training set and a test set using labels provided by the ground object truth map; Clustering the pixels in the hyperspectral image according to the similarity of spectral features to obtain a plurality of clustering regions, segmenting each of the clustering regions to generate a plurality of segmented regions, generating a segmented region and pixel mapping relationship matrix according to the pixel distribution of each segmented region, and constructing a pixel node adjacency matrix and a superpixel node adjacency matrix; Construct a dual-branch graph convolutional network; Inputting the hyperspectral image, the ground object truth map, the multiple segmented regions, the segmented region and pixel mapping relationship matrix, the pixel node adjacency matrix and the superpixel node adjacency matrix into the dual-branch graph convolutional network, and training the dual-branch graph convolutional network; The labels of the test set are predicted by the trained dual-branch graph convolutional network to complete the classification of the hyperspectral image.

[0009] According to a specific implementation of the embodiment of the present disclosure, the method of dividing the hyperspectral image into a training set and a test set by using the labels provided by the ground object truth map includes: Removing invalid pixels in the hyperspectral image according to the ground object truth map, and classifying the remaining valid pixels in the hyperspectral image according to the labels; For the valid pixels corresponding to each category set, the valid pixels are selected according to a preset ratio to obtain the training set and the test set.

[0010] According to a specific implementation method of the embodiment of the present disclosure, each of the clustered regions is segmented to generate multiple segmented regions, a segmented region and pixel mapping relationship matrix is ​​generated according to the pixel distribution of each segmented region, and a pixel node adjacency matrix and a superpixel node adjacency matrix are constructed, including: Clustering pixels of the hyperspectral image based on spectral feature similarity; Performing eight-connected region segmentation on pixels corresponding to each cluster category to obtain a plurality of segmented regions after segmentation; According to the pixel distribution of the plurality of segmented areas, constructing a mapping relationship matrix between the segmented areas and pixels; Constructing the pixel node adjacency matrix in each of the segmented areas to obtain the pixel node adjacency matrix of each of the segmented areas; Using the segmented area and pixel mapping relationship matrix, obtaining super pixel features of each segmented area; The superpixel node adjacency matrix is ​​constructed by calculating the similarity between each of the superpixel features.

[0011] According to a specific implementation of an embodiment of the present disclosure, the step of constructing a dual-branch graph convolutional network includes: Building a pixel-level graph convolution network branch including two layers of graph convolution networks, which is used to extract local features inside each segmented area and splice the extracted local features; Building a super-pixel-level graph convolutional network branch consisting of two layers of graph convolutional networks, for capturing the global features of the hyperspectral image and decoding the global features into local features; Fusion of the local features with the local features obtained by decoding the global features; The pixel-level graph convolutional network branch is connected in parallel with the superpixel-level graph convolutional network branch, and then connected in series with a fusion module to form a dual-branch graph convolutional network.

[0012] According to a specific implementation of the embodiment of the present disclosure, the training of the dual-branch graph convolutional network includes: A cross entropy loss function is used to calculate the predicted value of the dual-branch graph convolutional network and the loss value between the labels corresponding to the training set, and the dual-branch graph convolutional network is trained.

[0013] According to a specific implementation of the embodiment of the present disclosure, clustering pixels in the hyperspectral image according to spectral feature similarity to obtain multiple clustering regions, segmenting each of the clustering regions to generate multiple segmented regions, including: Divide the pixels corresponding to each cluster category into eight connected regions; For each pixel in each cluster category, determining whether the pixel has a connectivity relationship with the neighboring pixels of the pixel according to whether the pixel and the neighboring pixels of the pixel have the same cluster label; For each of the pixels in each of the cluster categories, the pixel and each of the neighboring pixels having a connectivity relationship with the pixel are divided into the same segmentation area.

[0014] According to a specific implementation of the embodiment of the present disclosure, the trained dual-branch graph convolutional network predicts the label of the test set data to complete the classification of the hyperspectral image, including: Inputting the final fusion features output by the dual-branch graph convolutional network into the classifier; Traversing each pixel of the hyperspectral image to obtain a predicted classification label of each pixel; The predicted classification label is searched in the index information of the test set, and the corresponding sample label is located according to the index information.

[0015] In a second aspect, the present disclosure provides a hyperspectral image classification device, comprising: A data loading and diversity module, for inputting a hyperspectral image and a ground object truth map corresponding to the hyperspectral image, and dividing the hyperspectral image into a training set and a test set using labels provided by the ground object truth map; A preprocessing and segmentation module is used to cluster the pixels in the hyperspectral image according to the similarity of spectral features to obtain multiple clustering regions, segment each clustering region to generate multiple segmented regions, generate a segmented region and pixel mapping relationship matrix according to the pixel distribution of each segmented region, and construct a pixel node adjacency matrix and a superpixel node adjacency matrix; Dual-branch graph convolutional network building module, used to build a dual-branch graph convolutional network; A network training module, used for inputting the hyperspectral image, the ground object truth map, the multiple segmented regions, the segmented region and pixel mapping relationship matrix, the pixel node adjacency matrix and the superpixel node adjacency matrix into the dual-branch graph convolutional network to train the dual-branch graph convolutional network; The classification prediction module is used to predict the labels of the test set data through the trained dual-branch graph convolutional network to complete the classification of the hyperspectral image.

[0016] According to a specific implementation of the embodiment of the present disclosure, the data loading and diversity module divides the hyperspectral image into a training set and a test set using the labels provided by the ground object truth map, including: Removing invalid pixels in the hyperspectral image according to the ground object truth map, and classifying the remaining valid pixels in the hyperspectral image according to the labels; For the valid pixels corresponding to each category set, the valid pixels are processed according to a preset ratio to obtain the training set and the test set.

[0017] According to a specific implementation of the embodiment of the present disclosure, the preprocessing and division module segments each of the clustering regions to generate a plurality of segmented regions, generates a segmented region and pixel mapping relationship matrix according to the pixel distribution of each segmented region, and constructs a pixel node adjacency matrix and a superpixel node adjacency matrix, including: clustering the pixels of the hyperspectral image based on spectral feature similarity; Performing eight-connected region segmentation on pixels corresponding to each cluster category to obtain a plurality of segmented regions after segmentation; According to the pixel distribution of the plurality of segmented areas, constructing a mapping relationship matrix between the segmented areas and pixels; Constructing the pixel node adjacency matrix in each of the segmented areas to obtain the pixel node adjacency matrix of each of the segmented areas; Using the segmented area and pixel mapping relationship matrix, obtaining super pixel features of each segmented area; The superpixel node adjacency matrix is ​​constructed by calculating the similarity between each of the superpixel features.

[0018] According to a specific implementation of the embodiment of the present disclosure, the dual-branch graph convolutional network construction module constructs a dual-branch graph convolutional network, including: building a pixel-level graph convolutional network branch including two layers of graph convolutional networks, for extracting local features inside each segmented area, and splicing the extracted local features; Building a super-pixel-level graph convolutional network branch consisting of two layers of graph convolutional networks, for capturing the global features of the hyperspectral image and decoding the global features into local features; Fusion of the local features with the local features obtained by decoding the global features; The pixel-level graph convolutional network branch is connected in parallel with the superpixel-level graph convolutional network branch, and then connected in series with a fusion module to form a dual-branch graph convolutional network.

[0019] According to a specific implementation method of an embodiment of the present disclosure, the network training module uses a cross entropy loss function to calculate the prediction value of the dual-branch graph convolutional network and the loss value between the corresponding labels of the training set to train the dual-branch graph convolutional network.

[0020] According to a specific implementation of the embodiment of the present disclosure, the preprocessing and segmentation module clusters the pixels in the hyperspectral image according to the similarity of spectral features to obtain multiple clustering regions, and segments each of the clustering regions to generate multiple segmented regions, including: Divide the pixels corresponding to each cluster category into eight connected regions; For each pixel in each cluster category, determining whether the pixel has a connectivity relationship with the neighboring pixels of the pixel according to whether the pixel and the neighboring pixels of the pixel have the same cluster label; For each of the pixels in each of the cluster categories, the pixel and each of the neighboring pixels having a connectivity relationship with the pixel are divided into the same segmentation area.

[0021] According to a specific implementation of the embodiment of the present disclosure, the classification prediction module uses the trained dual-branch graph convolutional network to predict the label of the test set data and complete the classification of the hyperspectral image, including: Inputting the final fusion features output by the dual-branch graph convolutional network into the classifier; Traversing each pixel of the hyperspectral image to obtain a predicted classification label of each pixel; The predicted classification label is searched in the index information of the test set, and the corresponding sample label is located according to the index information.

[0022] In a third aspect, an embodiment of the present disclosure provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned hyperspectral image classification method.

[0023] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium, which stores computer instructions for causing the computer to execute the above-mentioned hyperspectral image classification method.

[0024] In summary, compared with the prior art, this embodiment has the following advantages: 1. A dual-branch graph convolutional network was constructed, which showed significant advantages in global and local region feature extraction. It can accurately capture the dependencies between segmented regions in hyperspectral images, and deeply learn the local spatial correlation between pixels within each region, thereby achieving more accurate and rich feature expression and extraction, greatly improving classification performance.

[0025] 2. The advanced eight-connected segmentation algorithm is used to segment the hyperspectral image, and the generated segmentation results have clear boundaries and excellent connectivity. It not only effectively reduces the interference of noise on feature extraction, but also enables the constructed superpixel features to more accurately reflect the global context information of the hyperspectral image, significantly enhancing the feature expression ability.

[0026] 3. A pixel-level feature extraction branch is set up to construct a separate graph for each pixel in the segmented area, and the graph convolution operation is used again for feature extraction. This makes up for the deficiency that some pixel features may be too smooth when relying solely on the superpixel network, while avoiding the problem of incompatible data structures. It can capture the subtle differences of individual pixels in the local area, further improving the precision and accuracy of feature extraction.

[0027] In summary, this embodiment achieves accurate extraction of global and local features of hyperspectral images through a dual-branch graph convolutional network and an eight-connected segmentation algorithm, significantly enhancing the feature expression capability. At the same time, the setting of the pixel-level feature extraction branch further improves the precision and accuracy of feature extraction, providing a solid foundation for high-precision classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific implementation methods and drawings. Here, the exemplary implementation methods of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0029] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "an example embodiment" and "an embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0030] Figure 1 A schematic diagram of the process of a hyperspectral image classification method provided by the first embodiment of the present invention; Figure 2 Schematic diagram of the segmentation map obtained by using the segmentation algorithm; Figure 3 A schematic diagram of the structure of a hyperspectral image classification device provided by a second embodiment of the present invention; Figure 4 An exemplary structural diagram of a device capable of implementing the method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0031] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0032] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0033] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0034] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0035] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0036] See also Figure 1The present application embodiment provides a hyperspectral image classification method, which can be executed by an electronic device, in particular, by one or more processors in the electronic device to implement the following steps: S101, inputting a hyperspectral image and a ground object truth map corresponding to the hyperspectral image, and dividing the hyperspectral image into a training set and a test set using labels provided by the ground object truth map.

[0037] Among them, a hyperspectral image is an image containing multiple spectral bands, each of which represents the spectral information of the corresponding wavelength. These spectral bands can cover multiple spectral regions such as visible light, near-infrared light, and short-wave infrared light.

[0038] Moreover, hyperspectral images can provide rich spectral information of surface objects and can be used to distinguish different types of objects (such as vegetation, water bodies, soil, buildings, etc.). For each pixel in a hyperspectral image, the pixel can include a spectral vector, which can include parameters such as reflectance or radiation intensity values ​​of the pixel in a large number of narrow bands in the wavelength range of 400 to 2500 nanometers (nm).

[0039] In addition, the ground object truth map is an image spatially aligned with the hyperspectral image, and each pixel in the ground object truth map corresponds to at least one label. Among them, at least one label may include a non-zero ground object category label, and the ground object category label is used to indicate the ground object type corresponding to the pixel.

[0040] For example, for the label of the feature category corresponding to any pixel in the feature truth map, if the parameter value of the label is 0, it means that the pixel is an unlabeled background area; if the parameter value of the label is 1, it means that the pixel is a vegetation area such as a forest or grassland; if the parameter value of the label is 2, it means that the pixel is a water area such as a lake; if the parameter value of the label is 3, it means that the pixel is a building such as a house.

[0041] Furthermore, each pixel in the ground object true value map and each pixel in the corresponding hyperspectral image can correspond one-to-one based on the position of each pixel in the ground object true value map and the corresponding hyperspectral image.

[0042] It should be noted that the label corresponding to each pixel is obtained through field surveys, remote sensing interpretation or other reliable methods, and is therefore considered to be true or reliable. Accordingly, the ground object truth map can be used to train classification algorithms and supervised classification algorithms, so that the training classification algorithms and supervised classification algorithms can accurately identify ground objects of different ground object types from hyperspectral images.

[0043] In addition, the size of the hyperspectral image can be marked as H×W, and the number of spectral bands is C. The ground object truth map corresponding to the hyperspectral image also has a spatial dimension of H×W, and each pixel of the ground object truth map corresponds to an integer label, each integer label is a parameter value corresponding to each pixel, and each parameter value is a discrete parameter value. Furthermore, each integer label of the ground object truth map corresponds to the label of the ground object category of each pixel in the hyperspectral image, and each integer label can be used to represent the ground object category of the pixel at the corresponding position of the hyperspectral image.

[0044] It should also be noted that, in practical applications, the electronic device can obtain multiple hyperspectral images and multiple ground object true value maps according to the operation triggered by the user, and the multiple hyperspectral images and multiple ground object true value maps can be Figure 1 For the sake of simplicity, the present embodiment of the application only takes one hyperspectral image and one corresponding ground object true value map as an example for description, and the present embodiment of the application does not specifically limit the number of hyperspectral images and ground object true value maps.

[0045] For example, the electronic device can obtain a hyperspectral image through a pre-set storage space or according to an input operation triggered by a user. and ground feature true value map , complete the input of hyperspectral image and ground object ground truth map. Where H and W represent the height and width of the hyperspectral image space respectively, and C is the number of spectral bands.

[0046] Correspondingly, the electronic device can process the hyperspectral image based on the information of the ground object truth map, and obtain the training set and test set corresponding to the hyperspectral image in combination with the labels provided by the ground object truth map, so that in subsequent steps, the hyperspectral image classification model can be trained and verified by the training set and test set.

[0047] Specifically, the electronic device can first obtain a hyperspectral image and a ground object truth map, and read the hyperspectral image and the ground object truth map, and then combine the ground object category label corresponding to each pixel in the ground object truth map, traverse each pixel in the hyperspectral image, and delete invalid pixels in the hyperspectral image according to the parameter value of the ground object category label in the ground object truth map corresponding to each pixel.

[0048] For example, the electronic device may first determine each pixel whose parameter value of the label of the ground object category in the ground object truth map is 0, and then search for the corresponding pixel in the hyperspectral image according to the position of each determined pixel, and then remove each searched pixel. Accordingly, after deleting the invalid pixels in the hyperspectral image (the pixels whose parameter value of the label of the corresponding ground object category is 0), the electronic device screens out the valid pixels in the hyperspectral image, so that each valid pixel can be combined to obtain the valid pixel set N.

[0049] For the effective pixel set N, the electronic device can group the pixels according to the labels of the object categories corresponding to each effective pixel, that is, the electronic device can group the pixels according to the object categories indicated by the labels of the object categories corresponding to each effective pixel in the object truth map. Accordingly, for the effective pixels of each object category, the electronic device can randomly select the corresponding effective pixels as the training set according to a preset ratio, and use the remaining effective pixels as the test set. For example, the electronic device can select 5% of the effective pixels to generate the training set, and use the remaining 95% of the effective pixels to generate the test set.

[0050] S102, clustering pixels in the hyperspectral image according to spectral feature similarity to obtain multiple clustering regions, segmenting each clustering region to generate multiple segmented regions, and then generating a segmented region and pixel mapping relationship matrix according to the pixel distribution of each segmented region, and constructing a pixel node adjacency matrix and a superpixel node adjacency matrix.

[0051] Among them, the electronic device can use the K-means algorithm and the eight-connected domain segmentation algorithm to divide the hyperspectral image into regions, and construct a pixel node adjacency matrix and a superpixel (a larger pixel block composed of multiple adjacent pixels) node adjacency matrix. Of course, in practical applications, the electronic device can also use other algorithms to divide the hyperspectral image into regions, and the embodiment of the present application does not specifically limit the region division method of the hyperspectral image.

[0052] First, the electronic device can cluster the pixels of the hyperspectral image using the K-means algorithm according to the number of clusters K, and divide all pixels into K cluster categories according to the similarity of the spectral characteristics of each pixel to obtain a clustering result graph.

[0053] The number of clusters K may be pre-set or acquired according to an operation triggered by a user, and the present application embodiment does not specifically limit the method for acquiring the number of clusters K. Moreover, each cluster category has a corresponding cluster label for identifying and distinguishing symbols or names of different clusters, for example, the cluster label may be a number, letter, or string, etc., which is not limited in the present application embodiment.

[0054] In addition, spectral feature similarity refers to the degree of proximity or consistency between two or more spectra in terms of features.

[0055] It should be noted that in practical applications, since pixels with similar spectral features in a hyperspectral image may be dispersed in different areas of the hyperspectral image, pixels with similar spectral features in the clustering results may be distributed in multiple non-adjacent areas.

[0056] Afterwards, the electronic device can segment the multiple cluster areas by applying the eight-connected domain segmentation algorithm for each cluster category according to the ranges where the multiple cluster areas are located, and obtain at least one segmented area. Specifically, the electronic device can compare the cluster label of any pixel in each cluster area with the cluster labels of eight neighboring pixels (upper, lower, left, right and four diagonal directions) adjacent to the pixel, and determine whether the pixel has a connected relationship with each neighboring pixel. If the cluster labels of the two are the same, it is determined that the two have a connected relationship; if the cluster labels of the two are different, it is determined that the two do not have a connected relationship.

[0057] Accordingly, for each pixel in each clustering area, pixels having the same clustering label and having a connectivity relationship with other pixels in the clustering result map according to the eight-connectivity rule can be segmented into the same segmentation area.

[0058] However, for each pixel in the hyperspectral image, pixels with similar spectral characteristics may be far apart in spatial position. Therefore, each pixel can be divided into different segmentation areas in the above method, and this segmentation method can ensure that the segmentation result has a high degree of spatial coherence.

[0059] Furthermore, in practical applications, electronic devices can also use pre-set algorithms to identify pixels that have similar spectral features but are discretely distributed in hyperspectral images and generate pixel groups, so that they can combine the various pixel groups to generate more accurate segmentation areas of hyperspectral images, making up for the deficiency that relying solely on superpixel networks may cause some pixel features to be too smooth, while avoiding the problem of incompatible data structures.

[0060] For example, see Figure 2 , Figure 2 A schematic diagram of a segmentation diagram of a hyperspectral image provided in an embodiment of the present application, wherein the set of segmented regions is: ,in Indicates segmented areas, n is the total number of segmented regions, Indicates segmentation area The jth pixel in the segmentation area, m is the segmentation area The number of pixels in .

[0061] Accordingly, the electronic device can construct a segmentation area and pixel mapping relationship matrix according to the pixel distribution of each segmentation area. , where Z represents the number of valid pixels, n is the total number of segmented regions. iLine j Elements of a column The definition of is shown in formula (1): (1) in, Represents the first i Whether the pixel belongs to j If the segmented area i The pixels belong to j If is 1; if i Pixels do not belong to j If is 0. Represents the first i pixels.

[0062] The pixel node adjacency matrix is ​​constructed for each pixel in the segmented area. Specifically, for the first i The adjacency matrix of the partitioned regions , As shown in formula (2): in, , represents the adjacency matrix Middle j Line k The element of the column, i.e. i pixels in the segmented area and pixels The edge weight value between them can be calculated based on the similarity of spectral features between pixels. Indicated in i The pixels in the segmented area The set of the top c pixels in similarity ranking (where c <m)。

[0063] Afterwards, the electronic device can map the relationship matrix between the segmented area and the pixel , construct a super pixel feature matrix. Specifically, the electronic device can map the relationship matrix between the segmented area and the pixel , extract the superpixel features of each segmented area and form a superpixel feature matrix .

[0064] Among them, Encoder() represents the encoder that converts the pixel features of the hyperspectral image into super-pixel features. Express The result after column-by-column standardization, T represents the matrix transpose operation, and Flatten (X) represents the eigenvector obtained by flattening the hyperspectral data through the spatial dimension.

[0065] Correspondingly, the electronic device can construct an adjacency matrix of superpixel nodes based on the superpixel feature matrix. Specifically, the adjacency matrix of superpixel nodes As shown in formula (3): The elements Representation Matrix Middle i Line j The elements of the column are superpixel blocks and superpixel blocks The edge weight value between them can be calculated based on the spectral feature similarity between superpixel blocks. Representation and superpixel blocks The set of the top c superpixel blocks ranked by similarity (where c <n)。

[0066] Through the above steps, the hyperspectral image was divided into regions using the K-means algorithm and the eight-connected domain segmentation algorithm, and the pixel node adjacency matrix and the superpixel node adjacency matrix were constructed.

[0067] S103, construct a dual-branch graph convolutional network.

[0068] Among them, the dual-branch graph convolutional network may include: a pixel-level branch and a super-pixel-level graph convolutional network branch, and the pixel-level branch and the super-pixel-level graph convolutional network branch are both formed by two layers of graph convolutional networks.

[0069] In this embodiment, first, a pixel-level branch consisting of a two-layer graph convolutional network is constructed in the electronic device, aiming to go deep into each segmented area and extract local features.

[0070] Correspondingly, the pixel-level graph convolutional network branch can capture the fine information of each pixel and its neighboring pixels in the hyperspectral image through graph convolution operations. Finally, the local features extracted from each segmented area are spliced ​​to obtain a complete set of local features, which can provide detailed information for subsequent feature fusion.

[0071] At the same time, this embodiment also constructs another super-pixel-level graph convolution network branch composed of a two-layer graph convolution network in the electronic device, and the super-pixel-level graph convolution network branch focuses on capturing the features (global features) between pixels in the entire range of the hyperspectral image.

[0072] Specifically, the electronic device can extract features from the hyperspectral image through a graph convolution operation at the superpixel level to obtain the global features of the hyperspectral image. Afterwards, the electronic device can decode these global features into local features to ensure that while retaining global information, it can be connected to the local features of the pixel-level graph convolution network branch. Among them, the graph convolution operation at the superpixel level is to perform a graph convolution operation on the superpixel feature matrix constructed in the above embodiment.

[0073] In order to make full use of the features extracted by the pixel-level graph convolutional network branch and the superpixel-level graph convolutional network branch, the embodiment of the present application can also fuse the local features output by the pixel-level graph convolutional network branch with the global features output by the superpixel-level graph convolutional network branch in the electronic device, and combine the local features with the global features through the feature splicing fusion strategy to form a more expressive fusion feature.

[0074] Finally, the pixel-level graph convolution network branch and the super-pixel-level graph convolution network branch are connected in parallel to form a dual-branch network architecture. On this basis, the fusion module is connected in series to fuse the features of the two branches. In other words, the results output by the pixel-level graph convolution network branch and the super-pixel-level graph convolution network branch can be input into the fusion module, and the local features and global features can be fused through the fusion module.

[0075] In order to realize the classification of fused features, a softmax classifier can also be pre-set in the electronic device. The softmax classifier can accurately classify the pixels in the hyperspectral image based on the fused features, thereby completing the construction of the entire dual-branch graph convolutional network.

[0076] Specifically, the feature interaction layer of the pixel-level graph convolutional network branch constructed by the electronic device can be expressed as formula (4): in For the i Region l Layer feature matrix, For the i The first ( l +2) layer feature matrix, Relu is a nonlinear activation function, , for The degree matrix of .

[0077] Then the electronic device can perform feature interaction on the features of the segmented area and input them into the linear layer for feature mapping, as shown in formula (5): in, For thei The output of the region after passing through the feature mapping layer, For the i The output of the region after passing through the feature interaction layer, For the l The weight value of the layer.

[0078] Correspondingly, the specific feature set output by the pixel-level graph convolutional network branch is shown in formula (6): in, is the result of splicing the final output of all segmented regions. It is a sorting operation so that the spliced ​​pixels are arranged according to the original distribution.

[0079] A superpixel-level graph convolutional network branch consisting of two layers of graph convolutional networks is constructed. The initial feature of the network is V. The first network propagation form of this branch is shown in formula (7): in For this branch l Layer feature matrix, For the first ( l +1) layer feature matrix, , for The degree matrix of .

[0080] The final global features of the superpixel-level graph convolutional network branch need to be decoded into local features, as shown in formula (8): in It is the global feature output by the superpixel-level graph convolutional network branch.

[0081] The local features output by the pixel-level graph convolutional network branch are fused with the decoded local features output by the superpixel-level graph convolutional network branch, and the obtained fusion feature F is shown in formula (9): in, Represents a concatenation operation.

[0082] The pixel-level graph convolution network branch is then connected in parallel with the superpixel-level graph convolution network branch, and then connected in series with the fusion module to form a dual-branch graph convolution network. The final fusion features of this network are input into the softmax classifier so that classification can be achieved through the softmax classifier.

[0083] S104, inputting the hyperspectral image, the ground object truth map, multiple segmented regions, the segmented region and pixel mapping relationship matrix, the pixel node adjacency matrix and the superpixel node adjacency matrix into the dual-branch graph convolutional network to train the dual-branch graph convolutional network.

[0084] In this embodiment, the electronic device can input multiple data into the dual-branch graph convolutional network, including: hyperspectral images, ground truth maps, multiple segmented regions, segmented region and pixel mapping relationship matrix, pixel node adjacency matrix, and superpixel node adjacency matrix.

[0085] Among them, hyperspectral images are used to provide spectral information; ground truth maps are used to indicate the true categories of different objects on the ground; multiple segmentation regions S are used to define specific areas of concern in the image; the segmentation region and pixel mapping relationship matrix Q is used to establish the correspondence between the segmentation regions and the pixels in the image; the pixel node adjacency matrix is ​​used to describe the connection relationship between pixels with similar spectral characteristics in the segmentation region; and the superpixel node adjacency matrix is ​​used to represent the connection between superpixels.

[0086] After using the dual-branch graph convolutional network to classify and predict the hyperspectral image, the electronic device can obtain the predicted distribution of the dual-branch graph convolutional network and the actual category distribution, and then calculate the difference between the predicted distribution of the dual-branch graph convolutional network and the actual category distribution through the cross entropy loss function.

[0087] Among them, by minimizing the cross entropy loss value, the parameters of the two-branch graph convolutional network can be adjusted to make the prediction results of the two-branch graph convolutional network closer to the true classification label.

[0088] The cross entropy loss function is shown in formula (10): Where N is the total number of labeled pixels, C is the number of categories, Represents the label vector Middle elements, if pixels The true category is Class, then , otherwise , Represents predicted pixels Belong to The probability of the class.

[0089] S105, through the trained dual-branch graph convolutional network, the labels of the test set data are predicted to complete the classification of the hyperspectral image.

[0090] In this embodiment, the fused feature matrix F is input into a classifier (e.g., a softmax classifier) ​​of a trained dual-branch graph convolutional network. The classifier processes each feature vector and outputs a probability distribution vector, which represents the probability that the feature vector belongs to each category. For each pixel's feature vector, the electronic device can select the probability distribution vector with the largest parameter value output by the classifier, and use the index corresponding to the probability distribution vector as the predicted classification label. By traversing the probability distribution vector of each pixel, a list containing all pixel predicted classification labels can be obtained to complete the classification of the hyperspectral image.

[0091] In order to verify the classification performance of the embodiment of the present application for the hyperspectral image, the index information of the test set can be used to correspond the predicted classification label with the pixel in the test set. The electronic device can locate the corresponding sample label according to the index information of the test set, thereby determining the sample in the test set corresponding to each predicted classification label. The index information of the test set refers to the unique identifier used to identify and locate each test sample in the test data set.

[0092] For example, create a data structure with the index information of the test set as the key (or index) and the predicted classification label as the value. In this way, you can quickly find the corresponding predicted classification label through the index information.

[0093] Through the above steps, a predicted classification label is successfully assigned to each sample in the test set. That is, first predict the labels of all samples, then obtain the predicted labels of the test set according to the index of the test set, and then calculate the predicted labels of the test set and the sample labels of the test set to obtain the performance index of the classification result of the hyperspectral image provided by the embodiment of the present application.

[0094] In this implementation, a simulation is also performed to verify the hyperspectral image classification method provided in this embodiment. The details are as follows: It is implemented in the Python 3.7 programming language and the TensorFlow 1.15.0 deep learning framework. Its hardware configuration is an Intel Core i7-9750H central processor and an NVIDIA GTX 1650 graphics processor, which ensures efficient data processing and computing performance.

[0095] In the simulation experiment, the following three hyperspectral image datasets were used: Indian Pines dataset, this dataset is collected by the Airborne Visible / Infrared Imaging Spectrometer (AVIRIS) sensor in the agricultural area in northwest Indiana, USA. The dataset contains a 145×145 pixel original image with a spatial resolution of 20 meters, covering 220 spectral bands in the wavelength range from 0.4 to 2.5 microns. In order to improve data quality, the bands affected by the water absorption band (104-108, 150-163 and 220) are usually removed in experiments, and finally 200 valid bands are retained. The dataset is annotated with 16 different ground object categories and contains a total of 10,249 labeled samples.

[0096] PaviaU dataset, this dataset was taken by the Reflective Optics System Imaging Spectrometer (ROSIS) sensor and covers the University of Pavia in Italy and its surrounding areas. The spatial resolution of the dataset is 1.3 meters and the image size is 610×340 pixels. After band denoising, 103 valid bands are finally retained, with a band range of 0.43 to 0.86 microns. The dataset is annotated with 9 ground object categories and contains a total of 42,776 labeled samples.

[0097] Salinas dataset, this dataset was collected by AVIRIS sensor over Salinas Valley, California, USA. The image size is 512×217 pixels, the spatial resolution is 3.7 meters, and the spectral band covers the range from 0.4 to 2.5 microns. After removing the bands affected by noise (108-112, 154-167, and 224), 204 valid bands are retained. 16 ground object categories are annotated in the dataset, containing a total of 54,129 labeled samples.

[0098] Under the above technical environment, the performance was compared with seven hyperspectral image classification methods (SVM, SSRN, SpectralFormer, DBDA, FuNet-C, MDGCN and SPGCN-21) in the prior art. The experiment was repeated 10 times on the three data sets of Indian Pines, PaviaU and Salinas, and the average and standard deviation of the three evaluation indicators of overall accuracy (OA), average accuracy (AA) and kappa coefficient were calculated to comprehensively evaluate the classification performance of the embodiment of the present application.

[0099] First, the classification performance of this embodiment and seven different existing methods on the Indian Pines dataset was compared, where the proportion of the training set used was set to 5%. See Table 1 for specific results, which is a comparison of the classification performance of this embodiment and seven existing methods on the Indian Pines dataset.

[0100] Table 1 Then, the classification performance of this embodiment and the seven existing methods on the PaviaU dataset was compared. The comparison was performed under the condition that the proportion of the selected training set was 2%. See Table 2 for details, which shows the classification performance comparison results of this embodiment and the seven existing methods on the PaviaU dataset.

[0101] Table 2 Finally, the classification performance of this embodiment and the seven existing methods on the Salinas dataset was compared, and the proportion of the selected training set was 1%. See Table 3 for details, which shows the classification performance comparison results of this embodiment and the seven existing methods on the PaviaU dataset.

[0102] Table 3 It can be seen from Tables 1 to 3 that this embodiment shows the best classification performance on three commonly used hyperspectral datasets compared with the other seven existing methods.

[0103] In summary, compared with the prior art, this embodiment has the following advantages: In summary, this embodiment achieves accurate extraction of global and local features of hyperspectral images through a dual-branch graph convolutional network and an eight-connected segmentation algorithm, significantly enhancing the feature expression capability. At the same time, the setting of the pixel-level feature extraction branch further improves the precision and accuracy of feature extraction, providing a solid foundation for high-precision classification.

[0104] The second embodiment of the present invention provides a hyperspectral image classification device, see Figure 3 , Figure 3 A schematic diagram of the structure of a hyperspectral image classification device provided in this embodiment includes: The data loading and dividing module 310 is used to input a hyperspectral image and a ground object truth map corresponding to the hyperspectral image, and divide the hyperspectral image into a training set and a test set using labels provided by the ground object truth map; The preprocessing and segmentation module 320 is used to cluster the pixels in the hyperspectral image according to the similarity of spectral features to obtain multiple clustering regions, segment each clustering region to generate multiple segmentation regions, generate a segmentation region and pixel mapping relationship matrix according to the pixel distribution of each segmentation region, and construct a pixel node adjacency matrix and a superpixel node adjacency matrix; A dual-branch graph convolution network construction module 330 is used to construct a graph convolution network including a pixel-level graph convolution network branch and a super-pixel-level graph convolution network branch, and connect the pixel-level graph convolution network branch and the super-pixel-level graph convolution network branch in parallel, and then connect them in series with the fusion module to form a dual-branch graph convolution network, and then use a classifier to classify the final fusion features; The network training module 340 is used to input the hyperspectral image, the ground object truth map, the multiple segmented regions, the segmented region and pixel mapping relationship matrix, the pixel node adjacency matrix and the superpixel node adjacency matrix into the dual-branch graph convolution network to train the dual-branch graph convolution network; The classification prediction module 350 is used to predict the labels of the test set data through the trained dual-branch graph convolutional network to complete the classification of the hyperspectral image.

[0105] A third embodiment of the present invention further provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the hyperspectral image classification method of any of the aforementioned embodiments.

[0106] A fourth embodiment of the present invention further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the hyperspectral image classification method described in any of the aforementioned embodiments.

[0107] The fifth embodiment of the present invention further provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the hyperspectral image classification method of any of the aforementioned embodiments.

[0108] The sixth embodiment of the present invention further provides a computer program, which includes program instructions. When the program instructions are executed by a computer, the computer is enabled to execute the hyperspectral image classification method of any of the above embodiments.

[0109] Figure 4A schematic diagram of a method or device 1000 that can implement an embodiment of the present invention is shown, which may include more or fewer devices than shown in the figure in some embodiments. In some embodiments, it can be implemented using a single or multiple devices. In some embodiments, it can be implemented using cloud or distributed devices.

[0110] like Figure 4 As shown, the device 1000 includes a processor 1001, which can perform various appropriate operations and processes according to the program and / or data stored in the read-only memory (ROM) 1002 or the program and / or data loaded from the storage part 1008 to the random access memory (RAM) 1003. Processor 1001 can be a multi-core processor or include multiple processors. In some embodiments, processor 1001 can include a general main processor and one or more special coprocessors, such as a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In the random access memory 1003, various programs and data required for the operation of the device 1000 are also stored. The processor 1001, the read-only memory 1002 and the random access memory 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0111] The processor and the memory are used together to execute the program stored in the memory. When the program is executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0112] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, a touch screen, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed. Figure 4 Only some components are schematically shown, which does not mean that the device 1000 only includes Figure 4 Components shown.

[0113] The systems, devices, modules or units described in the above embodiments may be implemented by a computer or its associated components. The computer may be, for example, a mobile terminal, a smart phone, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a personal digital assistant, a media player, a navigation device, a game console, a tablet computer, a wearable device, a smart TV, an Internet of Things system, a smart home, an industrial computer, a server or a combination thereof.

[0114] Although not shown, in an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the hyperspectral image classification method described in Example 1 is implemented.

[0115] The storage media in the embodiments of the present invention include permanent and non-permanent, removable and non-removable items that can be implemented by any method or technology to store information. Examples of storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0116] Although not shown, the embodiment of the present invention further provides a computer program product, including: a computer program / instruction, which implements the hyperspectral image classification method described in Embodiment 1 when executed by a processor.

[0117] The methods, programs, systems, devices, etc. of the embodiments of the present invention may be executed or implemented in a single or multiple networked computers, or may be practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks may be performed by remote processing devices connected via a communication network.

[0118] Those skilled in the art should understand that the embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, those skilled in the art can imagine that the implementation of the functional modules / units or controllers and related method steps described in the above embodiments can be implemented in software, hardware or a combination of software / hardware.

[0119] Unless explicitly stated, the actions or steps of the methods, programs, and embodiments of the present invention do not have to be performed in a specific order and can still achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0120] In this article, multiple embodiments of the present invention are described, but for the sake of simplicity, the description of each embodiment is not exhaustive, and the same or similar features or parts between the embodiments may be omitted. In this article, "one embodiment", "some embodiments", "example", "specific example", or "some examples" are intended to be applicable to at least one embodiment or example of the present invention, but not all embodiments. The above terms do not necessarily mean to refer to the same embodiment or example. In the absence of contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0121] The exemplary systems and methods of the present invention have been specifically shown and described with reference to the above embodiments, which are merely examples of the best modes for implementing the present systems and methods. It will be appreciated by those skilled in the art that various changes may be made to the embodiments of the systems and methods described herein when implementing the present systems and / or methods without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A hyperspectral image classification method, characterized in that: include: Input includes a hyperspectral image and a ground object truth map corresponding to the hyperspectral image, and divides the hyperspectral image into a training set and a test set using labels provided by the ground object truth map; Clustering the pixels in the hyperspectral image according to the similarity of spectral features to obtain a plurality of clustering regions, segmenting each of the clustering regions to generate a plurality of segmented regions, generating a segmented region and pixel mapping relationship matrix according to the pixel distribution of each segmented region, and constructing a pixel node adjacency matrix and a superpixel node adjacency matrix; Construct a dual-branch graph convolutional network; Inputting the hyperspectral image, the ground object truth map, the multiple segmented regions, the segmented region and pixel mapping relationship matrix, the pixel node adjacency matrix and the superpixel node adjacency matrix into the dual-branch graph convolutional network, and training the dual-branch graph convolutional network; The labels of the test set are predicted by the trained dual-branch graph convolutional network to complete the classification of the hyperspectral image.

2. The hyperspectral image classification method according to claim 1, characterized in that: The method of dividing the hyperspectral image into a training set and a test set by using the labels provided by the ground object truth map includes: Removing invalid pixels in the hyperspectral image according to the ground object truth map, and classifying the remaining valid pixels in the hyperspectral image according to the labels; For the valid pixels corresponding to each category set, the valid pixels are selected according to a preset ratio to obtain the training set and the test set.

3. The hyperspectral image classification method according to claim 1, characterized in that: The step of segmenting each of the cluster regions to generate a plurality of segmented regions, generating a segmented region and pixel mapping relationship matrix according to the pixel distribution of each segmented region, and constructing a pixel node adjacency matrix and a superpixel node adjacency matrix includes: Clustering pixels of the hyperspectral image based on spectral feature similarity; Performing eight-connected region segmentation on pixels corresponding to each cluster category to obtain a plurality of segmented regions after segmentation; According to the pixel distribution of the plurality of segmented areas, constructing a mapping relationship matrix between the segmented areas and pixels; Constructing the pixel node adjacency matrix in each of the segmented areas to obtain the pixel node adjacency matrix of each of the segmented areas; Using the segmented area and pixel mapping relationship matrix, obtaining super pixel features of each segmented area; The superpixel node adjacency matrix is ​​constructed by calculating the similarity between each of the superpixel features.

4. The hyperspectral image classification method according to claim 1, characterized in that: The construction of a dual-branch graph convolutional network includes: Building a pixel-level graph convolution network branch including two layers of graph convolution networks, which is used to extract local features inside each segmented area and splice the extracted local features; Building a super-pixel-level graph convolutional network branch consisting of two layers of graph convolutional networks, for capturing the global features of the hyperspectral image and decoding the global features into local features; Fusion of the local features with the local features obtained by decoding the global features; The pixel-level graph convolutional network branch is connected in parallel with the superpixel-level graph convolutional network branch, and then connected in series with a fusion module to form a dual-branch graph convolutional network.

5. The hyperspectral image classification method according to claim 1, characterized in that: The training of the dual-branch graph convolutional network includes: A cross entropy loss function is used to calculate the predicted value of the dual-branch graph convolutional network and the loss value between the labels corresponding to the training set, and the dual-branch graph convolutional network is trained.

6. The hyperspectral image classification method according to claim 1, characterized in that: The pixels in the hyperspectral image are clustered according to the similarity of spectral features to obtain a plurality of cluster regions, and each of the cluster regions is segmented to generate a plurality of segmented regions, including: Divide the pixels corresponding to each cluster category into eight connected regions; For each pixel in each cluster category, determining whether the pixel has a connectivity relationship with the neighboring pixels of the pixel according to whether the pixel and the neighboring pixels of the pixel have the same cluster label; For each of the pixels in each of the cluster categories, the pixel and each of the neighboring pixels having a connectivity relationship with the pixel are divided into the same segmentation area.

7. The hyperspectral image classification method according to claim 1, characterized in that: The trained dual-branch graph convolutional network predicts the labels of the test set data to complete the classification of the hyperspectral image, including: Inputting the final fusion features output by the dual-branch graph convolutional network into the classifier; Traversing each pixel of the hyperspectral image to obtain a predicted classification label of each pixel; The predicted classification label is searched in the index information of the test set, and the corresponding sample label is located according to the index information.

8. A hyperspectral image classification device, characterized in that: include: A data loading and diversity module, for inputting a hyperspectral image and a ground object truth map corresponding to the hyperspectral image, and dividing the hyperspectral image into a training set and a test set using labels provided by the ground object truth map; A preprocessing and segmentation module is used to cluster the pixels in the hyperspectral image according to the similarity of spectral features to obtain multiple clustering regions, segment each clustering region to generate multiple segmented regions, generate a segmented region and pixel mapping relationship matrix according to the pixel distribution of each segmented region, and construct a pixel node adjacency matrix and a superpixel node adjacency matrix; Dual-branch graph convolutional network building module, used to build a dual-branch graph convolutional network; A network training module, used for inputting the hyperspectral image, the ground object truth map, the multiple segmented regions, the segmented region and pixel mapping relationship matrix, the pixel node adjacency matrix and the superpixel node adjacency matrix into the dual-branch graph convolutional network to train the dual-branch graph convolutional network; The classification prediction module is used to predict the labels of the test set data through the trained dual-branch graph convolutional network to complete the classification of the hyperspectral image.

9. An electronic device, characterized in that: The electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the hyperspectral image classification method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the hyperspectral image classification method according to any one of claims 1 to 7.

Images