A Hyperspectral Remote Sensing Image Classification Method and Device Based on Graph Neural Networks
By constructing an adjacency matrix and a graph neural network, the problems of noise interference and spatial relationship neglect in hyperspectral remote sensing image classification are solved, achieving efficient image classification results and improving classification accuracy and robustness.
Patent Information
- Application Number
- CN202510177022.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-18
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2045-02-18
AI Technical Summary
Existing hyperspectral remote sensing image classification methods struggle to effectively handle complex scenes using traditional statistical and machine learning algorithms, resulting in insufficient classification accuracy and robustness. CNN models, when processing hyperspectral data, neglect the continuity and spatial correlation of ground cover distribution, leading to abnormal pixels affecting model performance.
A graph neural network-based approach is adopted to capture the spatial relationships between pixels by constructing an adjacency matrix, identify and remove abnormal pixels using a cosine similarity metric function, perform dimensionality reduction by combining principal component analysis, and extract spectral features through a KAN network to construct graph structure data for classification.
It significantly improves the classification efficiency and accuracy of hyperspectral remote sensing images, solves the noise interference and classification performance bottlenecks existing in traditional methods, and achieves higher classification accuracy and stability.
Smart Images
Figure CN119888367B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a hyperspectral remote sensing image classification method and apparatus based on graph neural networks, belonging to the field of hyperspectral remote sensing image processing technology. Background Technology
[0002] Hyperspectral remote sensing images, rich in spectral information, have been widely used in modern agriculture, environmental protection, and military reconnaissance. In agriculture, hyperspectral remote sensing images can be used to monitor crop health and assess yield and quality; in environmental protection, they help monitor vegetation cover, water pollution, and soil quality; and in military reconnaissance, they can be used for target detection and identification. However, during the acquisition and application of hyperspectral data, noise has become a significant challenge due to sensor performance limitations and complex environmental factors. The phenomenon of different spectra for the same object and different objects with the same spectrum makes hyperspectral remote sensing image classification tasks more complex and difficult. In early research, researchers mainly relied on traditional statistical methods for hyperspectral remote sensing image classification, such as spectral angular distance and Euclidean distance algorithms, to achieve basic image classification through spectral feature analysis based on prior knowledge. Although these methods can distinguish different land cover types to some extent, they are difficult to handle classification problems in complex scenarios. With the advancement of stochastic machine learning techniques, algorithms such as k-nearest neighbor (KNN), support vector machine (SVM), and random forest (RF) have been introduced into hyperspectral remote sensing image classification. These methods significantly improve the accuracy and robustness of classification through pattern recognition and feature selection. SVM, in particular, has achieved good results in hyperspectral remote sensing image classification due to its strong generalization ability. The rise of deep learning has brought new opportunities to hyperspectral remote sensing image classification. Deep learning methods, represented by Convolutional Neural Networks (CNNs), have achieved historic breakthroughs in image processing thanks to their powerful feature extraction and abstraction capabilities. CNNs can automatically learn high-level features in images, thus greatly improving the performance of hyperspectral remote sensing image classification. However, traditional CNN models have certain limitations when processing hyperspectral data. Since CNNs can only process structured data, training and testing samples in hyperspectral remote sensing image classification usually need to be processed into neighborhood blocks of the same size. This method ignores the continuity and spatial correlation of ground cover distribution in hyperspectral remote sensing images, especially for nodes at class boundaries, where neighborhood blocks often contain pixels from different classes, resulting in a large number of anomalous pixels. These anomalous pixels negatively impact model training and testing, reducing classification accuracy and reliability. Summary of the Invention
[0003] The purpose of this invention is to provide a hyperspectral remote sensing image classification method and apparatus based on graph neural networks. By improving the ability to extract spectral and spatial features, it achieves effective classification of raw hyperspectral remote sensing images and significantly improves classification efficiency and accuracy.
[0004] To achieve the above objectives / to solve the above technical problems, the present invention is implemented using the following technical solution.
[0005] On one hand, the present invention provides a hyperspectral remote sensing image classification method based on graph neural networks, comprising the following steps:
[0006] Acquire the three-dimensional hyperspectral remote sensing image of the target;
[0007] Construct an adjacency matrix based on each pixel in the three-dimensional hyperspectral remote sensing image to be tested;
[0008] The spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be tested are reconstructed. Based on the reconstructed spectral features of ground objects, the cosine similarity metric function is used to identify abnormal pixels in the adjacency matrix. The abnormal pixels are removed to obtain the optimized adjacency matrix.
[0009] Dimensionality reduction processing is performed on the three-dimensional hyperspectral remote sensing image to be tested;
[0010] The optimized adjacency matrix and the dimensionality-reduced 3D hyperspectral remote sensing image to be tested are input into a pre-trained graph neural network model to obtain the classification result;
[0011] The processing of the trained graph neural network model includes:
[0012] The optimized adjacency matrix is used to construct a graph structure data from the dimensionality-reduced 3D hyperspectral remote sensing image. The neighbor features of each pixel in the graph structure data are aggregated by average aggregation to obtain aggregated features. The features of the center pixel are extracted from the aggregated features by the KAN network to obtain the embedding of the center pixel. The embedding of the center pixel is then processed by Softmax to obtain the classification result.
[0013] Furthermore, the specific method for constructing an adjacency matrix based on each pixel in the three-dimensional hyperspectral remote sensing image to be measured is as follows:
[0014] Each pixel in the hyperspectral remote sensing image is treated as a node and connected to its neighboring nodes to construct an adjacency matrix, expressed as:
[0015] ;
[0016] in: For pixels The adjacency matrix, Let i be the i-th pixel in the hyperspectral remote sensing image. , For the set of real numbers, The number of bands in a hyperspectral remote sensing image. represent The neighboring nodes, For the j-th neighbor node, ;
[0017] Construct an adjacency matrix to form graph-structured data, so as to better capture the spatial relationships between pixels.
[0018] Furthermore, the process of reconstructing the spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be tested, identifying abnormal pixels in the adjacency matrix based on the reconstructed spectral features using a cosine similarity metric function, and removing the abnormal pixels to obtain an optimized adjacency matrix specifically includes:
[0019] The spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be measured are reconstructed using the following expression:
[0020] ;
[0021] ;
[0022] ;
[0023] in, It is the reconstructed first spectral features of each pixel yes The first pixel Values for each band, Spectral characteristics The mean, Spectral characteristics Standard deviation;
[0024] The raw spectral data is standardized to ensure that the data distribution conforms to a normal distribution. A standard normalization function is used to scale the spectral features to eliminate numerical differences between different spectral features and increase the distinguishability of different categories of spectral features, thereby improving the accuracy of subsequent processing steps.
[0025] The optimized adjacency matrix is expressed as:
[0026] ;
[0027] ;
[0028] ;
[0029] in, The spectral features of the j-th neighbor node after reconstruction; yes and dot product, yes The Euclidean norm, yes The Euclidean norm;
[0030] and yes and The Values for each band;
[0031] When the cosine similarity metric function When and The edges between them are from the adjacency matrix After removing the middle part, we get the optimized adjacency matrix. ;
[0032] To avoid interference from abnormal pixels, a cosine similarity metric function is used to remove abnormal pixels in first-order nodes, and a new adjacency matrix is constructed to improve the robustness and accuracy of the adjacency matrix.
[0033] Furthermore, the dimensionality reduction processing of the three-dimensional hyperspectral remote sensing image to be tested specifically includes:
[0034] The mean normalization of each band value for each pixel is expressed as follows:
[0035] ;
[0036] in: A data matrix for each band value of each pixel. This is a data matrix of each pixel's band values after removing the mean;
[0037] Calculate the covariance matrix of the data matrix after removing the mean. The expression is:
[0038] ;
[0039] Where n is the number of samples;
[0040] For covariance matrix Perform eigenvalue decomposition to obtain eigenvalues and the corresponding feature vector :
[0041] ;
[0042] Where: eigenvectors For the first The orientation of each principal component, eigenvalues For the first The magnitude of the variance in the direction of each principal component;
[0043] Before choosing Eigenvectors with large eigenvalues This forms a dimension-reduced transformation matrix, expressed as:
[0044] ;
[0045] in, Is included before A matrix of eigenvectors;
[0046] Projecting hyperspectral remote sensing images onto a low-dimensional space, resulting in dimensionality-reduced data. for:
[0047] ;
[0048] in: It is a data matrix of the band values of each pixel after dimensionality reduction;
[0049] Using principal component analysis (PCA) reduces the number of bands, facilitates the extraction of key spectral features, reduces data dimensionality, and lowers computational complexity.
[0050] Furthermore, the optimized adjacency matrix is used to construct a graph structure data from the dimensionality-reduced 3D hyperspectral remote sensing image. The neighbor features of each pixel in the graph structure data are aggregated using an average aggregation method to obtain aggregated features. A KAN network is then used to extract the features of the center pixel from the aggregated features to obtain the embedding of the center pixel. Softmax processing is then applied to the embedding of the center pixel to obtain the classification result. Specifically, this includes:
[0051] ;
[0052] ;
[0053] ;
[0054] in: For the neural network The mean of the features of the neighboring nodes in the layer. For the first The average value between the center node and its neighboring nodes. It is the optimized adjacency matrix In and pixel The set of adjacent pixels, In the optimized adjacency matrix The neighboring pixels of a pixel, It is the first The pixel number Layer embedding, It is an aggregation The neighboring pixel The embedding obtained after layer features
[0055] It is the first The pixel number Layer embedding, It is the first The pixel number Layer embedding, , yes Middle Features of each pixel after dimensionality reduction;
[0056] The first in the hyperspectral remote sensing image The characteristics of each band are processed by the output of a univariate function. It is a function in a high-dimensional space. It is the first The final embedding is output for each pixel;
[0057] Multiple aggregation methods are used to integrate the first-order spatial features of nodes, and the Kolmogorov-Arnold Network (KAN) model is combined to extract spectral features, thereby enhancing the model's expressive power, classification ability, and accuracy.
[0058] Secondly, the present invention provides a hyperspectral remote sensing image classification device based on a graph neural network, comprising:
[0059] The acquisition module is used to acquire the three-dimensional hyperspectral remote sensing image to be tested;
[0060] The construction module is used to construct an adjacency matrix based on each pixel in the three-dimensional hyperspectral remote sensing image to be tested;
[0061] The optimization module is used to reconstruct the spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be tested. Based on the reconstructed spectral features of ground objects, the cosine similarity metric function is used to identify abnormal pixels in the adjacency matrix, and the abnormal pixels are removed to obtain the optimized adjacency matrix.
[0062] The dimension reduction module is used to perform dimension reduction processing on the three-dimensional hyperspectral remote sensing image to be tested.
[0063] The classification module is used to input the optimized adjacency matrix and the dimensionality-reduced 3D hyperspectral remote sensing image to be tested into a pre-trained graph neural network model to obtain the classification result.
[0064] The processing of the trained graph neural network model includes:
[0065] The optimized adjacency matrix is used to construct a graph structure data from the dimensionality-reduced 3D hyperspectral remote sensing image. The neighbor features of each pixel in the graph structure data are aggregated by average aggregation to obtain aggregated features. The features of the center pixel are extracted from the aggregated features by the KAN network to obtain the embedding of the center pixel. The embedding of the center pixel is then processed by Softmax to obtain the classification result.
[0066] Thirdly, the present invention provides a hyperspectral remote sensing image classification system based on graph neural networks, comprising:
[0067] Memory, used to store computer programs / instructions;
[0068] A processor is used to execute the computer program / instructions to implement the steps of the above-described hyperspectral remote sensing image classification method based on graph neural networks.
[0069] Fourthly, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the above-described hyperspectral remote sensing image classification method based on graph neural networks.
[0070] Fifthly, the present invention provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the above-described hyperspectral remote sensing image classification method based on graph neural networks.
[0071] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: First, this invention treats each pixel of a hyperspectral remote sensing image as a node, constructs an adjacency matrix, and forms graph-structured data to better capture the spatial relationships between pixels. Next, the original spectral data is standardized to ensure that the data distribution conforms to a normal distribution, thereby improving the accuracy of subsequent processing steps. Based on this, a cosine similarity metric function is used to identify and remove abnormal pixels in first-order neighbor nodes, optimizing the adjacency matrix and reducing noise interference during classification. Subsequently, principal component analysis (PCA) is applied to reduce the dimensionality of the data, simplifying the feature space. Based on this, a graph neural network model is constructed, integrating the first-order spatial features of nodes using various aggregation methods, and combining it with a Kolmogorov-Arnold Network (KAN) model to extract spectral features, thereby enhancing the model's expressive power. Finally, the trained advanced classification model is used for remote sensing image detection and classification. This method not only achieves effective classification of original hyperspectral remote sensing images but also significantly improves classification efficiency and accuracy, bringing a breakthrough to the field of hyperspectral remote sensing image processing. Attached Figure Description
[0072] Figure 1 This is a flowchart of the present invention.
[0073] Figure 2 This is a framework diagram of the present invention.
[0074] Figure 3 This is an RGB pseudo-color image of the Indian Pines hyperspectral remote sensing image according to an embodiment of the present invention.
[0075] Figure 4 The image shown is a real label image of the Indian Pines hyperspectral remote sensing image according to an embodiment of the present invention.
[0076] Figure 5 A schematic diagram of the SVM classification method results;
[0077] Figure 6 This is a schematic diagram of the results of the 2D-CNN classification method;
[0078] Figure 7 A schematic diagram of the results of the 3D-CNN classification method;
[0079] Figure 8 A schematic diagram of the SSRN classification results;
[0080] Figure 9 A schematic diagram of the SSFTT classification results;
[0081] Figure 10 A schematic diagram of the results of the GraphTransformer classification method;
[0082] Figure 11 A schematic diagram of the results of the GAT classification method;
[0083] Figure 12 A schematic diagram of the GraphSAGE classification method results;
[0084] Figure 13 A schematic diagram of the GCN classification results;
[0085] Figure 14 This is a schematic diagram of the classification results in this embodiment. Detailed Implementation
[0086] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.
[0087] The term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " generally indicates an "or" relationship between the related objects.
[0088] Example 1
[0089] like Figures 1 to 2 The embodiment shown provides a hyperspectral remote sensing image classification method based on graph neural networks, including the following steps:
[0090] Step S1: Obtain the three-dimensional hyperspectral remote sensing image to be tested, and construct an adjacency matrix based on each pixel in the three-dimensional hyperspectral remote sensing image to be tested, specifically including:
[0091] Acquiring 3D hyperspectral remote sensing images Each pixel Construct an adjacency matrix as a node , each node With 48 neighboring nodes Connected;
[0092] Adjacency Matrix The expression is:
[0093] ;
[0094] in: For pixels The adjacency matrix, Let i be the i-th pixel in the hyperspectral remote sensing image. , The number of bands in a hyperspectral remote sensing image. represent The neighboring nodes, For the j-th neighbor node, , These represent the length and width of the hyperspectral remote sensing image, respectively.
[0095] Step S2: Reconstruct the spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be measured. Based on the reconstructed spectral features, use the cosine similarity metric function to identify abnormal pixels in the adjacency matrix. Remove the abnormal pixels to obtain an optimized adjacency matrix. Specifically, this includes:
[0096] Step S2.1: The reconstruction of the spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be tested specifically includes:
[0097] Spectral information in the original three-dimensional hyperspectral remote sensing image data to be measured Feature scaling is performed by normalizing the spectral features using a standard normalization function, expressed as follows:
[0098] ;
[0099] ;
[0100] ;
[0101] in, It is the reconstructed first spectral features of each pixel Let i be the i-th pixel in the hyperspectral remote sensing image. yes The first pixel Values for each band, Spectral characteristics The mean, Spectral characteristics standard deviation The number of bands in the hyperspectral remote sensing image;
[0102] Step S2.2: The first-order neighbor nodes constructed based on spatial assumptions may contain pixels from other categories. To avoid abnormal pixels... To mitigate interference, a cosine similarity metric function is used to remove abnormal pixels from first-order nodes. The expression for the optimized adjacency matrix is as follows:
[0103] ;
[0104] ;
[0105] ;
[0106] in, The spectral features of the j-th neighbor node after reconstruction; yes and dot product, yes The Euclidean norm, yes The Euclidean norm;
[0107] and yes and The Values for each band;
[0108] When the cosine similarity metric function At that time, and The edges between them are from the adjacency matrix After removing the middle part, we get the optimized adjacency matrix. .
[0109] Step S3: Perform dimensionality reduction processing on the three-dimensional hyperspectral remote sensing image to be measured, specifically including:
[0110] Original 3D hyperspectral remote sensing image Each pixel The spectrum was reduced to 70 bands using principal component analysis (PCA);
[0111] The expression for mean normalization of each band value for each pixel:
[0112] ;
[0113] in: The mean value of the band. A data matrix for each band value of each pixel. This is a data matrix of each pixel's band values after removing the mean;
[0114] Calculate the covariance matrix of the data matrix after removing the mean. The expression is:
[0115] ;
[0116] Where n is the number of samples;
[0117] For covariance matrix Perform eigenvalue decomposition to obtain eigenvalues and the corresponding feature vector :
[0118] ;
[0119] Where: eigenvectors For the first The orientation of each principal component, eigenvalues For the first The magnitude of the variance in the direction of each principal component;
[0120] Before choosing Eigenvectors with large eigenvalues This forms a dimension-reduced transformation matrix, expressed as:
[0121] ;
[0122] in, Is included before A matrix of eigenvectors;
[0123] Projecting hyperspectral remote sensing images onto a low-dimensional space, resulting in dimensionality-reduced data. for:
[0124] ;
[0125] in: It is a data matrix of each pixel's band values after dimensionality reduction.
[0126] Step S4: Input the optimized adjacency matrix and the dimensionality-reduced 3D hyperspectral remote sensing image to be tested into a pre-trained graph neural network model to obtain the classification result; wherein, the processing of the pre-trained graph neural network model includes: constructing graph structure data from the dimensionality-reduced 3D hyperspectral remote sensing image to be tested using the optimized adjacency matrix; aggregating the neighbor features of each pixel in the graph structure data using an average aggregation method to obtain aggregated features; extracting the features of the center pixel from the aggregated features using a KAN network to obtain the embedding of the center pixel; and performing Softmax processing on the embedding of the center pixel to obtain the classification result, specifically including:
[0127] Step S4.1: The graph neural network model aggregates information from neighboring pixels using an aggregation strategy, extracts the spectral features of each pixel through the neural network, and then... After the next iteration of aggregation, each pixel can capture its... Structural information within the skip neighborhood, formally, in the graph neural network's first... The layers are represented as follows:
[0128] ;
[0129] ;
[0130] in: For the neural network The mean of the features of the neighboring nodes in the layer. For the The average value between the center node and its neighboring nodes. It is the optimized adjacency matrix In and pixel The set of adjacent pixels, In the optimized adjacency matrix The neighboring pixels of a pixel, yes Pixel number Layer embedding, It is an aggregation The neighboring pixel The embedding obtained after layer features;
[0131] It is the first The pixel number Layer embedding, It is the first The pixel number Layer embedding, and , yes Middle Features of each pixel after dimensionality reduction;
[0132] To extract spectral features more fully, the KAN classification model is used to extract the features of the center pixel of the aggregated information. The expression is as follows:
[0133] ;
[0134] in, It is a univariate function, representing the first variable in a hyperspectral remote sensing image. The characteristics of each band are processed by the output of a univariate function. It is a function in a high-dimensional space, used to combine multiple... The result, after multiple graph convolutions, outputs The final embedding of the node .
[0135] The training method for the above graph neural network model includes: acquiring three-dimensional hyperspectral remote sensing images and constructing a training set; inputting the training set into the graph neural network model to be trained; using the Adam optimizer to optimize the loss function; adjusting the network model parameters; stopping training when the loss no longer decreases; enabling the model to accurately distinguish different categories of spectral features; and obtaining a trained graph neural network model.
[0136] (1) Initialize the estimates of the first and second moments:
[0137] ;
[0138] ;
[0139] (2) Calculate the gradient in each iteration. :
[0140] (3) Update the first moment estimate (momentum):
[0141] ;
[0142] (4) Update the second-order moment estimate;
[0143] ;
[0144] (5) Calculate the first moment estimate of the deviation correction and second-order moment estimation :
[0145] ;
[0146] ;
[0147] (6) Parameter update formula:
[0148] ;
[0149] like Figure 3 As shown, this embodiment uses the Indian Pines dataset collected in 1992 in a rural area of northwestern Indiana, USA, using an AVIRIS sensor, with a wavelength range of 0.4-2.45 nm. The spatial resolution is 20 meters, containing 145 The dataset initially contained 145 pixels and 224 bands, some of which were water-absorbing or useless. These bands were removed in the experiment, resulting in a final dataset with 200 bands. The dataset contains a total of 21,025 pixels representing 16 types of crops, of which 10,249 pixels are labeled. The true labels for the ground features are as follows: Figure 4 As shown.
[0150] like Figures 5 to 14 The classification diagrams of different methods are shown. For ease of comparison, this embodiment displays false-color images and real labels of Indian Pines data. 1% of the training samples were randomly selected from each class in the Indian Pines dataset. This invention was compared with several advanced graph neural network methods. The experimental results are shown in Table 1. It can be seen that the OA, AA, and K of our proposed method all exceed those of the nine compared methods. In 6 out of 16 land cover categories, the classification accuracy of our method surpasses that of other methods. The OA of SVM is 59.23%. Among the four CNN-based classification models (i.e., 2D-CNN, 3D-CNN, SSRN, and SSFTT), SSRN performed best with an OA of 86.16%. Among the four GCN-based methods (i.e., GraphTransformer, GAT, GraphSAGE, and GCN), GCN performed best with an OA of 88.85%, while our proposed NAGIN achieved 91.07%, a 2.22% improvement over GCN.
[0151] Table 1 shows the accuracy and classification metrics for each class obtained using different methods on the Indian Pines dataset.
[0152]
[0153] As shown in the table above, SVM classification performs the worst, exhibiting a noticeable salt-and-pepper effect. Other methods utilize spatial features, which provides better noise control, but result in large areas of misclassified regions, especially at class boundaries. Relatively speaking, the method presented in this paper has better overall classification performance.
[0154] Example 2
[0155] This embodiment provides a hyperspectral remote sensing image classification device based on graph neural networks, including:
[0156] The acquisition module is used to acquire the three-dimensional hyperspectral remote sensing image to be tested;
[0157] The construction module is used to construct an adjacency matrix based on each pixel in the three-dimensional hyperspectral remote sensing image to be tested;
[0158] The optimization module is used to reconstruct the spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be tested. Based on the reconstructed spectral features of ground objects, the cosine similarity metric function is used to identify abnormal pixels in the adjacency matrix, and the abnormal pixels are removed to obtain the optimized adjacency matrix.
[0159] The dimension reduction module is used to perform dimension reduction processing on the three-dimensional hyperspectral remote sensing image to be tested.
[0160] The classification module is used to input the optimized adjacency matrix and the dimensionality-reduced 3D hyperspectral remote sensing image to be tested into a pre-trained graph neural network model to obtain the classification result.
[0161] The processing of the trained graph neural network model includes:
[0162] The optimized adjacency matrix is used to construct a graph structure data from the dimensionality-reduced 3D hyperspectral remote sensing image. The neighbor features of each pixel in the graph structure data are aggregated by average aggregation to obtain aggregated features. The features of the center pixel are extracted from the aggregated features by the KAN network to obtain the embedding of the center pixel. The embedding of the center pixel is then processed by Softmax to obtain the classification result.
[0163] Example 3
[0164] This embodiment provides a hyperspectral remote sensing image classification system based on graph neural networks, including:
[0165] Memory, used to store computer programs / instructions;
[0166] A processor is used to execute the computer program / instructions to implement the steps of the above-described hyperspectral remote sensing image classification method based on graph neural networks.
[0167] Example 4
[0168] This embodiment provides a computer-readable storage medium storing a computer program / instruction thereon. When the computer program / instruction is executed by a processor, it implements the steps of the above-described hyperspectral remote sensing image classification method based on graph neural networks.
[0169] Example 5
[0170] This embodiment provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-described hyperspectral remote sensing image classification method based on graph neural networks.
[0171] The above embodiments demonstrate the feasibility of the present invention in hyperspectral remote sensing image classification. The present invention solves the classification performance bottleneck caused by fixed spatial structure in remote sensing image classification, improves the quality of training and test samples, and makes the classification performance more stable when training samples are randomly selected, thus effectively improving the accuracy of hyperspectral remote sensing image classification.
[0172] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0173] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0174] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.
[0175] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0176] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all protected by the present invention.
Claims
1. A hyperspectral remote sensing image classification method based on graph neural networks, characterized in that, Includes the following steps: Acquire the three-dimensional hyperspectral remote sensing image of the target; Construct an adjacency matrix based on each pixel in the three-dimensional hyperspectral remote sensing image to be tested; The spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be tested are reconstructed. Based on the reconstructed spectral features of ground objects, the cosine similarity metric function is used to identify abnormal pixels in the adjacency matrix. The abnormal pixels are removed to obtain the optimized adjacency matrix. Dimensionality reduction processing is performed on the three-dimensional hyperspectral remote sensing image to be tested; The optimized adjacency matrix and the dimensionality-reduced 3D hyperspectral remote sensing image to be tested are input into a pre-trained graph neural network model to obtain the classification result; The processing of the trained graph neural network model includes: The optimized adjacency matrix is used to construct a graph structure data from the dimensionality-reduced 3D hyperspectral remote sensing image. The neighbor features of each pixel in the graph structure data are aggregated by average aggregation to obtain aggregated features. The features of the center pixel are extracted from the aggregated features by the KAN network to obtain the embedding of the center pixel. The embedding of the center pixel is processed by Softmax to obtain the classification result. The process involves reconstructing the spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be tested, identifying abnormal pixels in the adjacency matrix based on the reconstructed spectral features using a cosine similarity metric function, and removing these abnormal pixels to obtain an optimized adjacency matrix. Specifically, this includes: The spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be measured are reconstructed using the following expression: ； ； ； in, It is the reconstructed first spectral features of each pixel yes The first pixel Values for each band, Spectral characteristics The mean, Spectral characteristics Standard deviation; The optimized adjacency matrix is expressed as: ； ； ； in, The spectral features of the j-th neighbor node after reconstruction; yes and dot product, yes The Euclidean norm, yes The Euclidean norm; and yes and The The value of each band; when the cosine similarity metric function At that time, and The edges between them are from the adjacency matrix After removing the middle part, we get the optimized adjacency matrix. .
2. The hyperspectral remote sensing image classification method based on graph neural networks according to claim 1, characterized in that, The step of constructing an adjacency matrix based on each pixel in the three-dimensional hyperspectral remote sensing image to be measured specifically includes: Each pixel in the hyperspectral remote sensing image is treated as a node and connected to its neighboring nodes to construct an adjacency matrix, expressed as: ； in: For pixels The adjacency matrix, Let i be the i-th pixel in the hyperspectral remote sensing image. , For the set of real numbers, The number of bands in a hyperspectral remote sensing image. represent The neighboring nodes, For the j-th neighbor node, .
3. The hyperspectral remote sensing image classification method based on graph neural networks according to claim 1, characterized in that, The dimensionality reduction processing of the three-dimensional hyperspectral remote sensing image to be tested specifically includes: The mean normalization of each band value for each pixel is expressed as follows: ； in: A data matrix for each band value of each pixel. This is a data matrix of each pixel's band values after removing the mean; Calculate the covariance matrix of the data matrix after removing the mean. The expression is: ； Where n is the number of samples; For covariance matrix Perform eigenvalue decomposition to obtain eigenvalues and the corresponding feature vector : ； Where: eigenvectors For the first The orientation of each principal component, eigenvalues For the first The magnitude of the variance in the direction of each principal component; Before choosing Eigenvectors with large eigenvalues This forms a dimension-reduced transformation matrix, expressed as: ； in, Is included before A matrix of eigenvectors; Projecting hyperspectral remote sensing images onto a low-dimensional space, resulting in dimensionality-reduced data. for: ； in: It is a data matrix of each pixel's band values after dimensionality reduction.
4. The hyperspectral remote sensing image classification method based on graph neural networks according to claim 3, characterized in that, The process involves constructing a graph-structured data structure from the dimensionality-reduced 3D hyperspectral remote sensing image using an optimized adjacency matrix. Neighbor features of each pixel in the graph-structured data are then aggregated using an average aggregation method to obtain aggregated features. A KAN network is used to extract the features of the center pixel from the aggregated features, resulting in the embedding of the center pixel. Softmax processing is then applied to the embedding of the center pixel to obtain the classification result. Specifically, this includes: ； ； ； in: For the neural network The mean of the features of the neighboring nodes in the layer. For the first The average value between the center node and its neighboring nodes. It is the optimized adjacency matrix In and pixel The set of adjacent pixels, In the optimized adjacency matrix The neighboring pixels of a pixel, It is The pixel number Layer embedding, It is an aggregation The neighboring pixel The embedding obtained after layer features It is The pixel number Layer embedding, It is The pixel number Layer embedding, , yes The Middle Features of each pixel after dimensionality reduction; The first in the hyperspectral remote sensing image The characteristics of each band are processed by the output of a univariate function. It is a function in a high-dimensional space. It is The final embedding is output by each pixel.
5. A hyperspectral remote sensing image classification device based on graph neural networks, characterized in that, include: The acquisition module is used to acquire the three-dimensional hyperspectral remote sensing image to be tested; The construction module is used to construct an adjacency matrix based on each pixel in the three-dimensional hyperspectral remote sensing image to be tested; The optimization module is used to reconstruct the spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be tested. Based on the reconstructed spectral features of ground objects, the cosine similarity metric function is used to identify abnormal pixels in the adjacency matrix, and the abnormal pixels are removed to obtain the optimized adjacency matrix. The dimension reduction module is used to perform dimension reduction processing on the three-dimensional hyperspectral remote sensing image to be tested. The classification module is used to input the optimized adjacency matrix and the dimensionality-reduced 3D hyperspectral remote sensing image to be tested into a pre-trained graph neural network model to obtain the classification result. The processing of the trained graph neural network model includes: The optimized adjacency matrix is used to construct a graph structure data from the dimensionality-reduced 3D hyperspectral remote sensing image. The neighbor features of each pixel in the graph structure data are aggregated by average aggregation to obtain aggregated features. The features of the center pixel are extracted from the aggregated features by the KAN network to obtain the embedding of the center pixel. The embedding of the center pixel is processed by Softmax to obtain the classification result. The process involves reconstructing the spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be tested, identifying abnormal pixels in the adjacency matrix based on the reconstructed spectral features using a cosine similarity metric function, and removing these abnormal pixels to obtain an optimized adjacency matrix. Specifically, this includes: The spectral features of ground objects in the three-dimensional hyperspectral remote sensing image to be measured are reconstructed using the following expression: ； ； ； in, It is the reconstructed first spectral features of each pixel yes The first pixel Values for each band, Spectral characteristics The mean, Spectral characteristics Standard deviation; The optimized adjacency matrix is expressed as: ； ； ； in, The spectral features of the j-th neighbor node after reconstruction; yes and dot product, yes The Euclidean norm, yes The Euclidean norm; and yes and The The value of each band; when the cosine similarity metric function At that time, and The edges between them are from the adjacency matrix After removing the middle part, we get the optimized adjacency matrix. .
6. A hyperspectral remote sensing image classification system based on graph neural networks, characterized in that, include: Memory, used to store computer programs / instructions; A processor for executing the computer program / instructions to implement the steps of the hyperspectral remote sensing image classification method based on graph neural networks as described in any one of claims 1-4.
7. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the hyperspectral remote sensing image classification method based on graph neural networks as described in any one of claims 1-4.
8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the hyperspectral remote sensing image classification method based on graph neural networks as described in any one of claims 1-4.