A hyperspectral image classification method and system based on spectral clustering and negative sample mining

Abstract

This invention belongs to the field of image processing, specifically relating to a hyperspectral image classification method and system based on spectral clustering and negative sample mining. It aims to overcome the shortcomings of existing hyperspectral image classification methods in effectively capturing heterogeneous information in images. The method includes acquiring a hyperspectral image, extracting image patches and constructing a superpixel segmentation network to obtain multiple superpixel nodes, superpixel feature maps of each superpixel node, and mapping relationships between pixels and superpixels; clustering multiple superpixel nodes using a spectral clustering algorithm to construct graph nodes, and selecting a set of negative samples for each superpixel node based on a deterministic point process to form a negative sample set; inputting the negative sample set and the neighbor graph of the superpixel nodes into a Graph-UNet network to obtain superpixel features that fuse heterogeneous information; and using the mapping relationships between pixels and superpixels, mapping the superpixel features that fuse heterogeneous information to the pixel feature space and classifying them to obtain the classification result of the hyperspectral image.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a hyperspectral image classification method and system based on spectral clustering and negative sample mining. Background Technology

[0002] Hyperspectral imaging technology acquires spectral information of surface materials across continuous narrow bands, forming a data cube containing spatial, radiometric, and spectral features. This allows for accurate differentiation of different types of land features using fine spectral characteristics. Hyperspectral remote sensing image classification is a crucial aspect of this technology, aiming to assign a unique category label to each pixel in the image. Currently, methods for hyperspectral image classification mainly fall into two categories: one is based on convolutional neural networks, which treats the image as a regular Euclidean data grid and extracts local spatial-spectral joint features through convolutional structures; the other is based on graph neural networks, which maps the image to a graph structure in non-Euclidean space, using pixels or superpixels as graph nodes and aggregating information using edge connections between nodes, enabling explicit modeling of spatial context relationships of arbitrary shapes.

[0003] Existing hyperspectral image classification methods based on graph neural networks still have the following shortcomings in practical applications: First, most graph convolutional networks are homogeneous graph networks, whose design is based on the assumption that the features of adjacent nodes are smooth and the categories are similar. However, there is a lot of heterogeneous information in hyperspectral images. For example, adjacent pixels in the boundary area of ​​land features may belong to different categories, and there may be spectral abrupt changes within the same type of land feature due to mixed pixels or noise. Homogeneous graph convolutional networks perform indiscriminate smooth aggregation of the features of neighboring nodes, which can easily cause feature contamination and blurring of classification boundaries in heterogeneous areas such as land feature boundaries. Second, the classification performance is highly dependent on the predefined graph structure. Existing methods mostly construct graph structures based on the adjacency matrix of k-nearest neighbors or superpixels. This fixed graph construction method based on heuristic rules is difficult to adapt to the inherent characteristics of the data, and superpixel segmentation errors will be directly transmitted to subsequent classification models, affecting the final classification accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a hyperspectral image classification method and system based on spectral clustering and negative sample mining, aiming to overcome the shortcomings of existing hyperspectral image classification methods, such as large number of parameters, high computation time, and difficulty in effectively capturing heterogeneous information in images.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: This invention provides a hyperspectral image classification method based on spectral clustering and negative sample mining, comprising: S1, acquiring a hyperspectral image, extracting image patches and constructing a superpixel segmentation network to obtain multiple superpixel nodes, superpixel feature maps of each superpixel node, and mapping relationships between pixels and superpixels; S2, using a spectral clustering algorithm to cluster multiple superpixel nodes to construct graph nodes, and selecting a set of negative samples for each superpixel node based on a determinant point process to form a negative sample set; S3, inputting the negative sample set and the neighbor graph of the superpixel nodes into a Graph-UNet network to obtain superpixel features that fuse heterogeneous information; S4, using the mapping relationship between pixels and superpixels, mapping the superpixel features that fuse heterogeneous information to the pixel feature space and classifying them to obtain the classification result of the hyperspectral image.

[0006] In step S1, a superpixel segmentation network is constructed to obtain multiple superpixel nodes, superpixel feature maps of each superpixel node, and mapping relationships between pixels and superpixels. Specifically, this includes: extracting multi-scale spatial-spectral features from hyperspectral image patches to obtain multi-scale fusion feature maps; dividing the multi-scale fusion feature maps according to a fixed-size grid to obtain initial superpixel nodes; and iteratively updating the attribution matrix between pixels and superpixels and the features of superpixel nodes to obtain the final superpixel nodes, superpixel feature maps, and mapping relationships between pixels and superpixels.

[0007] In step S2, the spectral clustering algorithm is used to cluster multiple superpixel nodes. Specifically, this includes: constructing an adjacency matrix based on the neighbor relationships between superpixel nodes, and constructing a normalized Laplacian matrix based on the adjacency matrix; performing eigenvalue decomposition on the normalized Laplacian matrix to obtain eigenvectors, and clustering the eigenvectors to obtain cluster labels for the superpixel nodes.

[0008] In step S2, a set of negative samples is selected for each superpixel node based on the determinant point process to form a negative sample set. Specifically, this includes: calculating the average superpixel feature of each category according to the cluster label, and determining the multiple categories with the largest distance to the average superpixel feature of each superpixel node; selecting a preset number of superpixel nodes with the largest feature distance to the superpixel node as candidate negative samples in each determined category; selecting the set of negative sample nodes with the greatest diversity from the candidate negative samples for each superpixel node using the determinant point process; and constructing a negative adjacency matrix based on the negative sample node sets of all superpixel nodes.

[0009] In step S3, the negative sample set and the neighbor graph of the superpixel node are input into the Graph-UNet network to obtain superpixel features that fuse heterogeneous information. Specifically, this includes: inputting the superpixel feature map of the superpixel node, the adjacency matrix, and the negative adjacency matrix into the graph convolutional layer, respectively, and fusing information through the difference operation to obtain the fused superpixel features; inputting the fused superpixel features into the graph pooling layer, the graph inverse pooling layer, and the graph convolutional layer in sequence to obtain the enhanced superpixel features; and superimposing the enhanced superpixel features and the fused superpixel features through skip connections to obtain superpixel features that fuse heterogeneous information.

[0010] In step S4, the mapping relationship between pixels and superpixels is used to map the superpixel features that fuse heterogeneous information to the pixel feature space and classify them. Specifically, this includes: mapping the superpixel features that fuse heterogeneous information to the feature space corresponding to each pixel through the mapping relationship between pixels and superpixels to obtain pixel-level features; inputting the pixel-level features into a fully connected layer and outputting the category label of each pixel to obtain the classification result of the hyperspectral image.

[0011] Step S1 involves acquiring a hyperspectral image and extracting image patches, specifically including: performing principal component analysis to reduce the dimensionality of the hyperspectral image; extracting pixels within the neighborhood of each pixel to form an image patch; and performing oversampling on pixels of a few categories.

[0012] Multi-scale spatial-spectral features of hyperspectral image patches are extracted to obtain multi-scale fused feature maps. Specifically, this includes: performing convolution, normalization, and activation operations on the hyperspectral image patches sequentially at multiple scales, and downsampling between scales using max pooling layers; upsampling the downsampled feature maps to restore them to their original resolution; and concatenating the feature maps at different scales through channels and fusing them through convolutional layers to obtain multi-scale fused feature maps.

[0013] The standardized Laplacian matrix is ​​subjected to eigenvalue decomposition to obtain eigenvectors, and then the eigenvectors are clustered. Specifically, this includes: performing eigenvalue decomposition on the standardized Laplacian matrix to obtain eigenvalues ​​and their corresponding eigenvectors; calculating the difference between adjacent eigenvalues, determining the position of the eigenvalue with the largest difference, and taking the eigenvector before that position; standardizing the extracted eigenvectors, and then using the k-means algorithm to cluster them to obtain the cluster labels of the superpixel nodes.

[0014] The present invention also provides a hyperspectral image classification system based on spectral clustering and negative sample mining, including a processor and a memory, the memory being connected to the processor, and the processor being configured to execute the hyperspectral image classification method based on spectral clustering and negative sample mining as described above.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention converts hyperspectral images into graph-structured data by constructing a superpixel segmentation network. Superpixels are used as graph nodes to participate in subsequent graph convolution operations. Compared with the graph construction method using pixels as nodes, this significantly reduces the number of graph nodes and lowers the computational complexity of the graph convolutional network. At the same time, the superpixel segmentation network adopts a multi-scale feature extraction and iterative update mechanism, which can provide high-quality graph nodes for subsequent graph processing while preserving effective spatial-spectral information. This effectively avoids the shortcomings of traditional three-dimensional convolutional networks, such as large number of parameters and high computation time, and provides efficient technical support for real-time classification of hyperspectral images.

[0016] 2. This invention introduces a spectral clustering algorithm to cluster superpixel nodes, and mines a diverse set of negative samples for each superpixel node based on a determinant point process. By constructing positive and negative adjacency matrices to represent the homogeneous and heterogeneous information between nodes respectively, the two are input into the Graph-UNet network for difference fusion. This enables the graph convolutional network to simultaneously cluster the similar features of homogeneous neighbor nodes and the contrast features of heterogeneous neighbor nodes, effectively overcoming the problems of feature contamination and classification boundary ambiguity in heterogeneous regions such as ground object boundaries and mixed pixels in existing homogeneous graph convolutional networks, and significantly improving the accuracy of hyperspectral image classification.

[0017] 3. This invention constructs an end-to-end classification framework from pixel space to superpixel space and back to pixel space. It achieves bidirectional feature mapping through the mapping relationship between pixels and superpixels, which not only retains the computational efficiency advantages of superpixel graph structures, but also achieves high-precision classification at the pixel level. At the same time, through the synergistic effect of spectral clustering and negative sample mining, the construction and learning process of graph nodes can adapt to the inherent heterogeneity of data, avoiding the error propagation problem caused by superpixel segmentation errors in traditional fixed graph structures, and ensuring the robustness and generalization ability of the classification model. Attached Figure Description

[0018] Figure 1 This is a flowchart of a hyperspectral image classification method based on spectral clustering and negative sample mining provided in an embodiment of this application; Figure 2 This is a schematic diagram of the test results of a hyperspectral image classification method based on spectral clustering and negative sample mining provided in an embodiment of this application; Figure 3 This is a schematic diagram of a confusion matrix provided in an embodiment of this application; Figure 4 This is a schematic diagram of another test result provided in an embodiment of this application; Figure 5 This is another schematic diagram of a confusion matrix provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] For example, such as Figure 1 As shown, this application provides a hyperspectral image classification method based on spectral clustering and negative sample mining, including: S1. Acquire hyperspectral images, extract image patches and construct a superpixel segmentation network to obtain multiple superpixel nodes, superpixel feature maps of each superpixel node, and mapping relationships between pixels and superpixels.

[0021] For example, in step S1, acquiring a hyperspectral image and extracting image patches specifically includes: performing principal component analysis (PCA) dimensionality reduction on the hyperspectral image; extracting pixels within the neighborhood of each pixel to form an image patch; and performing oversampling processing on pixels of a few categories.

[0022] In some embodiments, PCA is first used to reduce the dimensionality of the input hyperspectral Indian pine remote sensing image, resulting in a spectral dimension of 30. Then, data normalization and image patch division centered on the current pixel are performed, with each patch being [size missing]. The image patch data is then divided into training, validation, and test sets in a 2:2:6 ratio, with each patch labeled as a category corresponding to the current pixel.

[0023] In step S1, a superpixel segmentation network is constructed to obtain multiple superpixel nodes, superpixel feature maps of each superpixel node, and mapping relationships between pixels and superpixels. Specifically, this includes: extracting multi-scale spatial-spectral features from hyperspectral image patches to obtain multi-scale fusion feature maps; dividing the multi-scale fusion feature maps according to a fixed-size grid to obtain initial superpixel nodes; and iteratively updating the attribution matrix between pixels and superpixels and the features of superpixel nodes to obtain the final superpixel nodes, superpixel feature maps, and mapping relationships between pixels and superpixels.

[0024] As one possible approach, multi-scale spatial-spectral features of hyperspectral image patches are extracted to obtain a multi-scale fused feature map. Specifically, this involves: performing convolution, normalization, and activation operations on the hyperspectral image patches sequentially at multiple scales, and downsampling between scales using max pooling layers; upsampling the downsampled feature map to restore it to its original resolution; and concatenating the feature maps at different scales through channels and fusing them through convolutional layers to obtain a multi-scale fused feature map.

[0025] Hyperspectral image patches are input into a superpixel segmentation network to generate superpixel map structure data, i.e., multiple superpixel nodes, superpixel feature maps of each superpixel node, and mapping relationships between pixels and superpixels. As one possible implementation, the superpixel segmentation network includes a multi-scale feature extraction module and a superpixel sampling module. The multi-scale feature extraction module extracts multi-scale spatial-spectral features from the hyperspectral image patches to obtain a multi-scale fused feature map; the superpixel sampling module divides the multi-scale fused feature map into superpixel nodes.

[0026] For example, the multi-scale feature extraction module consists of three scales, each consisting of two convolutional layers, a batch normalization layer, and an activation function, alternately connected. A max-pooling layer is used for downsampling between every two scales. After each max-pooling layer, an upsampling operation is used to upsample the feature map and restore the same resolution as the input hyperspectral image patch, where the kernel size of all convolutional layers is [value missing]. Finally, convolutional layers are used to concatenate the feature maps at the three scales, and then fusion is performed through another convolutional layer to obtain the multi-scale fused feature map F. H .

[0027] For example, the superpixel sampling module divides the multi-scale fused feature map according to a fixed-size grid, taking each grid region as an initial superpixel node S0, and taking the average feature value of all pixels in each grid region as the initial superpixel feature Fs0 of the initial superpixel node S0; by iteratively updating the attribution matrix Q between pixels and superpixels and the features of the superpixel nodes, the final superpixel node S and the superpixel feature map F corresponding to each superpixel node are obtained. S And the mapping relationship between pixels and superpixels. Among them, the attribution matrix between pixels and superpixels is used to represent the attribution relationship between each pixel and each superpixel node.

[0028] S2. Use the spectral clustering algorithm to cluster multiple superpixel nodes to construct graph nodes, and select a set of negative samples for each superpixel node based on the determinant point process to form a negative sample set.

[0029] As one possible implementation, step S2 uses a spectral clustering algorithm to cluster multiple superpixel nodes. Specifically, this includes: constructing an adjacency matrix A based on the neighbor relationships between superpixel nodes S; if two superpixel nodes are spatially adjacent, the element at the corresponding position in the adjacency matrix has a value of 1, otherwise it has a value of 0; and constructing a normalized Laplacian matrix L1 based on the adjacency matrix A; performing eigenvalue decomposition on the normalized Laplacian matrix to obtain eigenvectors φ, and clustering the eigenvectors φ to obtain the cluster labels L of the superpixel nodes.

[0030] For example, eigenvalue decomposition is performed on the standardized Laplacian matrix to obtain eigenvectors, and then clustering is performed on the eigenvectors. Specifically, this includes: performing eigenvalue decomposition on the standardized Laplacian matrix to obtain eigenvalues ​​λ and their corresponding eigenvectors λ; calculating the difference between adjacent eigenvalues, determining the position of the eigenvalue with the largest difference, and taking the eigenvector before that position; standardizing the extracted eigenvectors, and then using the k-means algorithm to perform clustering to obtain the clustering labels of the superpixel nodes.

[0031] As one possible implementation, step S2 selects a set of negative samples for each superpixel node based on the determinant point process, forming a negative sample set. Specifically, this includes: calculating the average superpixel feature of each category according to the cluster label, and determining the multiple categories with the largest distance to the average superpixel feature for each superpixel node. For example, the selection formula is as follows: For example, select m categories, where m is the number of positive samples; where, , Indicates the results after clustering , The average superpixel feature of the class. Within each defined class, a predetermined number of superpixel nodes with the largest feature distance (i.e., the least similar superpixel nodes) are selected as candidate negative samples; for example, the selection formula is: ,in, and Represents a superpixel node , The superpixel features are determined. A determinant-based point process is used to select the most diverse set of negative sample nodes for each superpixel node from the candidate negative samples. A negative adjacency matrix A is constructed based on the negative sample node sets of all superpixel nodes. n .

[0032] S3. Input the negative sample set and the neighbor graph of the superpixel node into the Graph-UNet network to obtain superpixel features that fuse heterogeneous information.

[0033] As one possible implementation, step S3 inputs the negative sample set and the neighbor graph of the superpixel node into the Graph-UNet network to obtain superpixel features that fuse heterogeneous information. Specifically, this includes: inputting the superpixel feature map F of the superpixel node... S With adjacency matrix A and negative adjacency matrix A respectively n The input is fed into two convolutional layers, and information is fused through interpolation to obtain the fused superpixel feature F1(S). The specific formula is: F1(S) = GCN(A,F) S )-GCN(A n ,F S GCN(·) represents graph convolution operation; the fused superpixel features F1(S) are sequentially input into the graph pooling layer, graph inverse pooling layer and graph convolution layer to obtain enhanced superpixel features; the enhanced superpixel features and the fused superpixel features are superimposed through skip connections to obtain superpixel features that fuse heterogeneous information.

[0034] For example, inputting the fused superpixel feature F1(S) into a graph pooling layer yields the graph node feature F2(S), which is more important for classification: F2(S) = Pool(A, F1(S)). Inputting the graph node feature F2(S) into a graph inverse pooling layer and padding with zeros restores the original graph node feature F3(S): F3(S) = Uppool(A, F2(S)). Then, inputting F3(S) into a graph convolutional network layer yields the superpixel feature F4(S): F4(S) = GCN(A, F3(S)). Introducing a skip connection between F4(S) and the fused superpixel feature F1(S) results in a superpixel feature F1(S) that incorporates heterogeneous information. a (S), specifically: F a (S)=F1(S)+F4(S).

[0035] S4. Utilize the mapping relationship between pixels and superpixels to map the superpixel features that fuse heterogeneous information to the pixel feature space and classify them to obtain the classification results of the hyperspectral image.

[0036] As one possible implementation, step S4 utilizes the mapping relationship between pixels and superpixels to map the superpixel features that fuse heterogeneous information to the pixel feature space and perform classification. Specifically, this includes: mapping the superpixel features that fuse heterogeneous information to the feature space corresponding to each pixel through the mapping relationship between pixels and superpixels to obtain pixel-level features; inputting the pixel-level features into a fully connected layer and outputting the category label of each pixel to obtain the classification result of the hyperspectral image.

[0037] This application also provides a hyperspectral image classification system based on spectral clustering and negative sample mining, including a processor and a memory. The memory is connected to the processor, and the processor is configured to execute the hyperspectral image classification method based on spectral clustering and negative sample mining as described above. More specifically, the memory stores a fully trained model. Model training includes setting a preset initial learning rate, a preset batch size, and a preset number of iterations. For example, the preset initial learning rate, preset batch size, and preset number of iterations are: g=0.0001, batch size=64, epoch=10, respectively. The hyperspectral image classification method based on spectral clustering and negative sample mining is trained using a training set, and the model is evaluated in real time using a validation set. During training and evaluation, the Adam optimizer is used to optimize the loss function to control the adjustment direction of the model parameters, thereby obtaining a hyperspectral image classification model. The hyperspectral image patch to be classified is input into the hyperspectral image classification model, which performs the following steps: S1. Acquire the hyperspectral image, extract the image patch, and construct a superpixel segmentation network to obtain multiple superpixel nodes, superpixel feature maps of each superpixel node, and mapping relationships between pixels and superpixels; S2. Cluster the multiple superpixel nodes using a spectral clustering algorithm to construct graph nodes, and select a set of negative samples for each superpixel node based on a deterministic point process to form a negative sample set; S3. Input the negative sample set and the neighbor graph of the superpixel node into the Graph-UNet network to obtain superpixel features that fuse heterogeneous information; S4. Utilize the mapping relationship between pixels and superpixels to map the superpixel features that fuse heterogeneous information to the pixel feature space and perform classification to obtain the classification result of the hyperspectral image.

[0038] For example, refer to Figure 2 , Figure 2 The classification results of Indianpine using a depth map convolutional network based on spectral clustering and negative sample mining are shown below. Figure 2 As can be seen, the true label map and the predicted result map maintain a high degree of consistency, achieving accurate classification of hyperspectral remote sensing images.

[0039] To provide a more intuitive understanding of the performance of the depth map convolutional model based on spectral clustering and negative sample mining, a confusion matrix is ​​used to visualize each class. For example, ... Figure 3 As shown. Figure 3 The horizontal axis represents the model's prediction results, and the vertical axis represents the true labels. Figure 3It can be seen that the prediction accuracy is higher for both cells with fewer and more categories. For example, the number of “Grass-pasture-mowed” cells is 28, and its accuracy is 1.00. The number of “Corn-notill” cells is 1428, of which 1398 cells are accurately predicted, and its accuracy is 0.98.

[0040] This application also provides another solution, which removes the spectral clustering algorithm and negative sample mining strategy, and consists only of a superpixel segmentation network and a depth map convolutional network. For example, the specific steps of this embodiment are as follows: A hyperspectral image patch is input into the superpixel segmentation network to generate superpixel map structure data. The superpixel segmentation network includes a multi-scale feature extraction module and a superpixel sampling module. The multi-scale spatial-spectral features of the hyperspectral image patch are extracted by the multi-scale feature extraction module to obtain a multi-scale fused feature map. The multi-scale fused feature map is divided into superpixel nodes by the superpixel sampling module. Specifically, the multi-scale fused feature map is divided according to a fixed-size grid, with each grid region serving as an initial superpixel node, and the average feature value of all pixels within each grid region serving as the initial superpixel feature of that initial superpixel node. By iteratively updating the attribution matrix between pixels and superpixels and the features of the superpixel nodes, the final superpixel nodes, the superpixel feature maps corresponding to each superpixel node, and the mapping relationship between pixels and superpixels are obtained. An adjacency matrix is ​​constructed based on the neighbor relationships between superpixel nodes. The adjacency matrix and the superpixel feature map are input into the Graph-UNet network to obtain enhanced superpixel features. Specifically, the superpixel feature maps and adjacency matrices of superpixel nodes are input into a graph convolutional layer to obtain fused superpixel features. These fused superpixel features are then sequentially input into a graph pooling layer, a graph inverse pooling layer, and a graph convolutional layer to obtain enhanced superpixel features. The enhanced superpixel features are then superimposed on the fused superpixel features via skip connections to obtain further enhanced superpixel features. Utilizing the mapping relationship between pixels and superpixels, the enhanced superpixel features are mapped to the pixel feature space, and classification is performed through a fully connected layer to obtain the classification results for the hyperspectral image.

[0041] For example, Figure 4 This is a schematic diagram illustrating the classification results of the Indian Pine dataset in this embodiment. Figure 5 This is the corresponding confusion matrix diagram. According to... Figure 4 It can be seen that the classification results of this embodiment maintain a high degree of consistency with the actual label map overall, but there are obvious classification errors in the boundary areas of land features and in a few category areas. Specifically, as shown in the example... Figure 5As shown in the confusion matrix, the prediction accuracy for the "Grass-pasture-mowed" category (with 28 samples) is 0.89, which is lower than 1.00 in Example 1; in the "Corn-notill" category (with 1428 samples), only 1320 pixels were accurately predicted, with an accuracy of 0.92, which is lower than 0.98 in the method provided in the embodiments of this application.

[0042] contrast Figure 2 and Figure 4 , Figure 3 and Figure 5 It can be seen that the introduction of spectral clustering and negative sample mining strategies effectively improves the classification ambiguity problem in the boundary regions of ground features, while also enhancing the ability to identify a few categories. This is because the spectral clustering algorithm can cluster based on the spectral and spatial feature similarity of superpixel nodes, constructing graph nodes that can represent the homogeneity within categories and the heterogeneity between categories. Meanwhile, the negative sample mining strategy based on the determinant-based point process selects the most diverse set of negative samples for each superpixel node. By constructing negative and positive adjacency matrices and inputting them into the Graph-UNet network, the model can simultaneously cluster the similar features of homogeneous neighbor nodes and the contrastive features of heterogeneous neighbor nodes, thereby achieving more accurate classification in heterogeneous regions such as ground feature boundaries and mixed pixels. Therefore, the comparative results show that this application, by introducing spectral clustering and negative sample mining strategies, effectively overcomes the problems of feature contamination and classification boundary ambiguity in heterogeneous regions such as ground feature boundaries in existing homogeneous graph convolutional networks, significantly improving the accuracy of hyperspectral image classification.

[0043] In the description of this specification, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0044] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A hyperspectral image classification method based on spectral clustering and negative sample mining, characterized in that, include: S1. Acquire hyperspectral images, extract image patches and construct a superpixel segmentation network to obtain multiple superpixel nodes, superpixel feature maps of each superpixel node, and mapping relationships between pixels and superpixels. S2. Cluster the multiple superpixel nodes using the spectral clustering algorithm to construct graph nodes, and select a set of negative samples for each superpixel node based on the determinant point process to form a negative sample set. S3. Input the negative sample set and the neighbor graph of the superpixel node into the Graph-UNet network to obtain superpixel features that fuse heterogeneous information; S4. Using the mapping relationship between the pixels and superpixels, the superpixel features that fuse heterogeneous information are mapped to the pixel feature space and classified to obtain the classification result of the hyperspectral image.

2. The hyperspectral image classification method based on spectral clustering and negative sample mining according to claim 1, characterized in that, In step S1, a superpixel segmentation network is constructed to obtain multiple superpixel nodes, superpixel feature maps of each superpixel node, and mapping relationships between pixels and superpixels. Specifically, this includes: extracting multi-scale spatial-spectral features from hyperspectral image patches to obtain multi-scale fusion feature maps; dividing the multi-scale fusion feature maps according to a fixed-size grid to obtain initial superpixel nodes; and iteratively updating the attribution matrix between pixels and superpixels and the features of superpixel nodes to obtain the final superpixel nodes, superpixel feature maps, and mapping relationships between pixels and superpixels.

3. The hyperspectral image classification method based on spectral clustering and negative sample mining according to claim 1, characterized in that, In step S2, the spectral clustering algorithm is used to cluster the multiple superpixel nodes. Specifically, this includes: constructing an adjacency matrix based on the neighbor relationships between superpixel nodes, and constructing a normalized Laplacian matrix based on the adjacency matrix; performing eigenvalue decomposition on the normalized Laplacian matrix to obtain eigenvectors, and clustering the eigenvectors to obtain cluster labels for the superpixel nodes.

4. The hyperspectral image classification method based on spectral clustering and negative sample mining according to claim 3, characterized in that, In step S2, a set of negative samples is selected for each superpixel node based on the determinant point process to form a negative sample set. Specifically, this includes: calculating the average superpixel feature of each category according to the clustering label, and determining multiple categories with the largest distance to the average superpixel feature of each superpixel node; selecting a preset number of superpixel nodes with the largest feature distance to the superpixel node as candidate negative samples in each determined category; selecting the set of negative sample nodes with the greatest diversity from the candidate negative samples for each superpixel node using the determinant point process; and constructing a negative adjacency matrix based on the negative sample node sets of all superpixel nodes.

5. The hyperspectral image classification method based on spectral clustering and negative sample mining according to claim 4, characterized in that, In step S3, the negative sample set and the neighbor graph of the superpixel node are input into the Graph-UNet network to obtain superpixel features that fuse heterogeneous information. Specifically, this includes: inputting the superpixel feature map of the superpixel node, the adjacency matrix, and the negative adjacency matrix into a graph convolutional layer, respectively, and performing information fusion through a difference operation to obtain fused superpixel features; inputting the fused superpixel features sequentially into a graph pooling layer, a graph inverse pooling layer, and a graph convolutional layer to obtain enhanced superpixel features; and superimposing the enhanced superpixel features and the fused superpixel features through skip connections to obtain the superpixel features that fuse heterogeneous information.

6. The hyperspectral image classification method based on spectral clustering and negative sample mining according to claim 1, characterized in that, In step S4, the mapping relationship between pixels and superpixels is used to map the superpixel features that fuse heterogeneous information to the pixel feature space and perform classification. Specifically, this includes: mapping the superpixel features that fuse heterogeneous information to the feature space corresponding to each pixel through the mapping relationship between pixels and superpixels to obtain pixel-level features; inputting the pixel-level features into a fully connected layer and outputting the category label of each pixel to obtain the classification result of the hyperspectral image.

7. The hyperspectral image classification method based on spectral clustering and negative sample mining according to claim 1, characterized in that, Step S1 involves acquiring a hyperspectral image and extracting image patches, specifically including: performing principal component analysis to reduce the dimensionality of the hyperspectral image; extracting pixels within the neighborhood of each pixel to form an image patch; and performing oversampling on pixels of a few categories.

8. The hyperspectral image classification method based on spectral clustering and negative sample mining according to claim 2, characterized in that, The extraction of multi-scale spatial-spectral features from hyperspectral image patches to obtain a multi-scale fused feature map specifically includes: performing convolution, normalization, and activation operations on the hyperspectral image patches sequentially at multiple scales, and downsampling between scales using max pooling layers; upsampling the downsampled feature map to restore it to the original resolution; and concatenating the feature maps at different scales through channels and fusing them through convolutional layers to obtain a multi-scale fused feature map.

9. The hyperspectral image classification method based on spectral clustering and negative sample mining according to claim 3, characterized in that, The step of performing eigenvalue decomposition on the standardized Laplacian matrix to obtain eigenvectors and then clustering the eigenvectors specifically includes: performing eigenvalue decomposition on the standardized Laplacian matrix to obtain eigenvalues ​​and their corresponding eigenvectors; calculating the difference between adjacent eigenvalues, determining the position of the eigenvalue with the largest difference, and taking the eigenvector before that position; standardizing the extracted eigenvectors and using the k-means algorithm to cluster them to obtain the clustering labels of the superpixel nodes.

10. A hyperspectral image classification system based on spectral clustering and negative sample mining, characterized in that, It includes a processor and a memory, the memory being connected to the processor, and the processor being configured to perform a hyperspectral image classification method based on spectral clustering and negative sample mining as described in any one of claims 1-9.

Images