Hyperspectral remote sensing image classification method based on hybrid high-order feature propagation graph network
By adding random perturbation and consistency regularization to the hyperspectral remote sensing image classification model, the oversmoothing problem is solved, the effective aggregation of high-order features and the improvement of classification accuracy are achieved, and the computational resource requirements are reduced.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN UNIV
- Filing Date
- 2024-06-26
- Publication Date
- 2026-05-26
AI Technical Summary
Existing hyperspectral remote sensing image classification models are prone to oversmoothing when aggregating high-order features, resulting in features becoming too uniform and making it difficult to effectively extract complex ground structure information. Furthermore, the design of deep networks increases the demand for computational resources.
By adding random perturbation elements to the initial graph structure to generate multiple perturbation matrices, multi-order feature mixing and propagation are performed. The model is then trained by combining consistency regularization and a preset loss function, which reduces the dependence of nodes on their neighborhoods, enhances the robustness to noise information, and improves the model's generalization ability.
Without increasing network depth, it effectively aggregates high-order features, alleviates oversmoothing, improves classification accuracy and model performance, and reduces computational resource requirements.
Smart Images
Figure CN118781404B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation map network. Background Technology
[0002] Compared to multispectral remote sensing imagery, hyperspectral remote sensing imagery is rich in spectral band information. By organically combining two-dimensional spatial features with one-dimensional spectral features, hyperspectral remote sensing image classification has attracted widespread attention from scholars and professionals both domestically and internationally for its excellent applicability in downstream applications such as vegetation monitoring, urban planning, and natural disaster assessment. The techniques applied to hyperspectral remote sensing image classification have evolved from traditional classification methods to early convolutional neural network methods. However, convolutional neural networks suffer from drawbacks such as shared kernel parameters and an inability to adapt flexibly, easily losing class boundary information in remote sensing images with complex ground structures. Therefore, graph convolutional networks, which express different connection patterns and features between pixels through graph structures and flexibly aggregate neighbor node information through weight matrices to adapt to different local neighborhood structures, have become an emerging method for hyperspectral remote sensing image classification, enabling more accurate classification decisions at class boundaries.
[0003] Graph Convolutional Networks (GCNs) iteratively aggregate features from nodes in a graph, effectively capturing the relationships between nodes and updating node features. Although the traditional Vanilla GCN, based on a first-order Chebyshev polynomial approximation, can only aggregate first-order information, it can still achieve good classification results through simple two-layer stacking. However, for remote sensing images with complex spatial structures of ground features, higher-order features can reveal more complex local geographic structures in the image, helping the model infer the complex relationships between pixel nodes.
[0004] One of the most popular ways to aggregate high-order features is by stacking network layers. Therefore, these deep GCN models have shown significant advantages in handling large-scale graph data, capturing more complex graph structural features while maintaining model performance. However, deep graph convolutional networks face a series of challenges when dealing with complex graph structural data, especially overcoming the oversmoothing phenomenon unique to graph convolutional networks during training. Oversmoothing refers to the tendency of node features to become homogeneous as the number of network layers increases, making it difficult to distinguish inter-class relationships and leading to a decline in model performance. To address these issues, researchers have proposed various strategies. JK-Net takes a spatial perspective, employing skip connections and adaptive mechanisms between convolutional layers to preserve the locality of node information. Li et al. proposed a model that combines deep and shallow features, successfully mitigating overfitting and oversmoothing problems by introducing concepts such as residuals, dense connections, and dilated convolutions during deep GCN training. These methods all achieve good classification performance by deepening the model to capture high-order features and mitigating the oversmoothing problem at deeper network layers.
[0005] As a message-passing framework, Graph Convolutional Networks (GCNs) offer several methods, besides stacking convolutional layers, to aggregate higher-order features without increasing model depth. However, since the feature propagation step in GCNs is the primary cause of oversmoothing, these methods may still exhibit oversmoothing, requiring corresponding countermeasures in algorithm design. For example, some algorithms decouple the feature propagation and transformation processes of GCNs, improving the feature propagation process. The SGC algorithm extracts higher-order information by applying iteratively obtained powers in a single-layer network, while the APPNP algorithm propagates target node information to more distant nodes by retaining partial root node information during random walks to update the node's state until convergence. Other algorithms improve the aggregation pattern of single-layer convolutional layers to extract higher-order neighborhood information; for instance, the MixHop algorithm incorporates a second-order Delta operator, enabling the model to acquire higher-order neighborhood information within a single layer. Furthermore, Rong et al. proposed the DropEdge method, which alleviates the problem of node features converging to invariant subspaces by randomly deleting edges from the graph. This method first demonstrates that oversmoothing can cause the model to converge to a subspace that is only related to the graph's topology. It then mitigates this problem by increasing the feature dimension of the convergent subspace, specifically by generating different deformed copies of the original graph using DropEdge. This increases the randomness and diversity of the input data, effectively preventing output features from converging. Furthermore, DropEdge, as a regularization technique, weakens message passing by reducing edges in the graph. This helps reduce oversmoothing because sparse graphs mean each node receives less information, forcing the model to focus more on important connections between nodes. This method can also solve the oversmoothing problem and achieve excellent classification results as the network depth increases.
[0006] While significant progress has been made in the research of high-order GCNs in computer vision, their application in hyperspectral image classification remains limited. Recently, Yang et al. proposed a Deep Dynamic Hyperspectral Image Classification Graph Network (DNAGSI) based on residual and identity mappings, representing a preliminary attempt to extract high-order information using deep GCNs in hyperspectral image classification. However, no algorithm has yet been designed to aggregate high-order features without increasing model depth and is suitable for hyperspectral remote sensing images. Compared to deep GCNs, a well-designed network model can address the limited range problem in message-passing models without introducing additional parameters.
[0007] Existing GCN networks suffer from oversmoothing, leading to feature inseparability. This results in most models only being able to aggregate two-order features during convolution, making it difficult to extract information from higher-order neighbor nodes in hyperspectral remote sensing images. Although a few algorithms have attempted to stack GCN layers to build deep networks and achieved good classification results, building deep networks significantly increases model parameters and consumes more computing resources.
[0008] Therefore, there is an urgent need to solve how to achieve high-order feature aggregation while saving computer resources, and to alleviate the problem of feature convergence caused by oversmoothing, thereby improving classification accuracy. Summary of the Invention
[0009] This invention provides a hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network to address the shortcomings of existing technologies.
[0010] In a first aspect, the present invention provides a hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network, comprising:
[0011] Collect a remote sensing image dataset, and generate an initial map structure based on the spatial and spectral features of the remote sensing image dataset;
[0012] Random perturbation elements are added repeatedly to the feature matrix of the initial graph structure to generate multiple perturbation matrices, which are then used as several data augmentations.
[0013] The multiple perturbation matrices are subjected to multi-order feature mixing propagation to obtain several enhanced feature matrices. The several enhanced feature matrices are then input into a two-layer multilayer perceptron for feature transformation to obtain the feature transformation output result.
[0014] Consistency regularization is used to constrain the prediction consistency of unlabeled data in the several data augmentations, and the model parameters of the feature transformation output are trained using a preset loss function to obtain a trained hyperspectral remote sensing image classification model.
[0015] The hyperspectral remote sensing image data to be processed is input into the hyperspectral remote sensing image classification model, and the image classification prediction result is output.
[0016] According to the present invention, a hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network is provided. Random perturbation elements are cyclically added to the feature matrix of the initial graph structure to generate multiple perturbation matrices. These multiple perturbation matrices are then used as several data augmentations, including:
[0017] A random mask is generated from a Bernoulli random variable, and the feature vector of each pixel in the feature matrix is determined to be retained or discarded based on the random mask.
[0018] The discarded feature vectors are re-aggregated based on the random mask, perturbation probability factor, and any row vector in the feature matrix, and together with the retained feature vectors, they form the plurality of perturbation matrices.
[0019] According to the present invention, a hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network is provided, wherein multi-order feature hybrid propagation is performed on the multiple perturbation matrices respectively to obtain several enhanced feature matrices, including:
[0020] A multi-order adjacency matrix is randomly generated from the plurality of perturbation matrices, and the power exponent mean of the adjacency matrix from order 0 to multi-order is obtained based on the adjacency matrix;
[0021] Multiplying the mean of the power exponents by the plurality of perturbation matrices yields the enhanced feature matrix;
[0022] The enhanced feature matrix is randomly perturbed using perturbation matrices with different masks to obtain the plurality of enhanced feature matrices.
[0023] According to the present invention, a hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network is provided, which employs consistency regularization to constrain the prediction consistency of unlabeled data in several data augmentations, including:
[0024] Calculate the enhanced mean of all unlabeled data, and use the enhanced mean as the center of the label distribution;
[0025] The label distribution center is sharpened using a low-entropy prediction method to generate a label prediction probability, which represents the probability that any pixel node is predicted to be any land feature.
[0026] The predicted consistency is converged using a preset consistency loss function.
[0027] According to the present invention, a hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network is provided, which uses a preset consistency loss function to converge the prediction consistency, including:
[0028] The distance between the feature transformation output and the sum of the label prediction probabilities is calculated and used as the preset consistency loss function.
[0029] Wherein, the summation of the label prediction probabilities is a vector set of the prediction probabilities of the label prediction probabilities on any type of land cover.
[0030] According to the present invention, a hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network is provided, which trains the model parameters of the feature transformation output result using a preset loss function to obtain a trained hyperspectral remote sensing image classification model, including:
[0031] The preset loss function is obtained by summing the product of the preset consistency loss function and the weight hyperparameter with the cross-entropy loss function;
[0032] The model parameters of the feature transformation output are updated using the preset loss function. After the model converges, the hyperspectral remote sensing image classification model is output.
[0033] Secondly, the present invention also provides a hyperspectral remote sensing image classification system based on a hybrid high-order feature propagation graph network, comprising:
[0034] The generation module is used to acquire remote sensing image datasets and generate an initial map structure based on the spatial and spectral features of the remote sensing image datasets.
[0035] The perturbation module is used to repeatedly add random perturbation elements to the feature matrix of the initial graph structure to generate multiple perturbation matrices, and to use the multiple perturbation matrices as several data augmentations.
[0036] The propagation module is used to perform multi-order feature mixing propagation on the multiple perturbation matrices to obtain several enhanced feature matrices, and input the several enhanced feature matrices into a two-layer multilayer perceptron for feature transformation to obtain the feature transformation output result.
[0037] The constraint module is used to constrain the prediction consistency of unlabeled data in the several data augmentations by using consistency regularization, and to train the model parameters of the feature transformation output using a preset loss function to obtain a trained hyperspectral remote sensing image classification model.
[0038] The output module is used to input the hyperspectral remote sensing image data to be processed into the hyperspectral remote sensing image classification model and output the image classification prediction results.
[0039] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network as described above.
[0040] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network as described above.
[0041] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network as described above.
[0042] The hyperspectral remote sensing image classification method provided by this invention uses a hybrid high-order feature propagation graph network to aggregate high-order features without increasing the network depth, in order to address the oversmoothing phenomenon in the feature propagation step of the graph convolutional network. At the same time, it reduces the high dependence of nodes on their neighborhoods to enhance robustness to noise information. Through a semi-supervised classification model training strategy suitable for hyperspectral remote sensing images, it improves the model's generalization ability and overall performance on unlabeled data. Attached Figure Description
[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0044] Figure 1 This is a flowchart illustrating the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network provided by the present invention.
[0045] Figure 2 This is a classification framework diagram based on hybrid high-order feature propagation graph network provided by the present invention;
[0046] Figure 3 This is a schematic diagram of adding random perturbation provided by the present invention;
[0047] Figure 4This is a schematic diagram of the structure of the hyperspectral remote sensing image classification system based on a hybrid high-order feature propagation graph network provided by the present invention;
[0048] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0050] Figure 1 This is a flowchart illustrating the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network provided in this embodiment of the invention. Figure 1 As shown, it includes:
[0051] Step 100: Acquire a remote sensing image dataset, and generate an initial map structure based on the spatial and spectral features of the remote sensing image dataset;
[0052] Step 200: Add random perturbation elements to the feature matrix of the initial graph structure multiple times in a loop to generate multiple perturbation matrices, and use the multiple perturbation matrices as several data augmentations;
[0053] Step 300: Perform multi-order feature mixing propagation on the multiple perturbation matrices to obtain several enhanced feature matrices, and input the several enhanced feature matrices into a two-layer multilayer perceptron for feature transformation to obtain the feature transformation output result;
[0054] Step 400: Consistency regularization is used to constrain the prediction consistency of unlabeled data in the several data augmentations, and the model parameters of the feature transformation output are trained using a preset loss function to obtain a trained hyperspectral remote sensing image classification model.
[0055] Step 500: Input the hyperspectral remote sensing image data to be processed into the hyperspectral remote sensing image classification model, and output the image classification prediction result.
[0056] Specifically, this embodiment of the invention first uses the initial graph structure as a template and employs the DropNode method to add random perturbations to the graph structure. This reduces the dependence of each node on specific neighboring nodes and makes the graph structure more sparse, thus slowing down the convergence speed of the feature propagation process to the invariant subspace. Subsequently, mixed-order propagation is performed on the perturbated graph structure. By integrating low-order features into the propagation process of high-order features, the model can pay more attention to local low-order features, thereby further alleviating the oversmoothing problem. In addition, combining the idea of data augmentation, the DropNode method is regarded as an augmentation method to generate multiple augmented copies to expand the data samples. The output features of each augmented graph structure after feature propagation are fed into a simple multilayer perceptron network for feature transformation and inference. Consistency regularization is used as a loss function to guide the gradient backpropagation of the model, which constrains the multiple augmented feature maps, completes the training and prediction of the model, and finally obtains the land cover labels of each pixel.
[0057] This invention provides an algorithm that can aggregate high-order features without increasing network depth to address the oversmoothing phenomenon in the feature propagation step of graph convolutional networks. It also reduces the high dependence of nodes on their neighborhoods to enhance robustness to noise information. Furthermore, by designing a semi-supervised classification model training strategy suitable for hyperspectral remote sensing images, it improves the model's generalization ability and overall performance on unlabeled data.
[0058] As one example, such as Figure 2 As shown, the process of this embodiment of the invention includes the following steps:
[0059] Step 1: First, generate an initial map structure based on the spatial and spectral features of the image.
[0060] Step 2: Add random perturbations (randomly discard elements in X) to the feature matrix X of the graph structure generated in Step 1. Repeat this process multiple times to generate multiple perturbation matrices. And treat it as Q data augmentations.
[0061] The random perturbation operation is obtained by multiplying the mask by the eigenvectors. Specifically, a random mask is first generated, then the eigenvectors of each pixel are multiplied by it, and the perturbation matrix is generated by randomly setting the eigenvectors of some nodes in the feature matrix X to 0. In the DropNode method, Bernoulli random variables possess binary properties (1 represents retention, 0 represents masking) and independence, and are simple and easy to control; therefore, Bernoulli random variables are chosen to introduce perturbations. First, a mask ε is randomly generated. i ~Bernoulli(1-δ), ε i Let v be independent and identically distributed Bernoulli random variables. For each node v i , mask ε iThe value of ε determines whether the node's features are discarded during random perturbations (set to 0). i =1, then node v i The eigenvectors remain unchanged; if ε i =0, then node v i The feature vectors are discarded and will be regenerated later through feature aggregation from neighboring nodes. The formula is expressed as follows: X i Let δ represent the vector in the i-th row of the characteristic matrix X, where δ is the perturbation probability factor, and δ is the vector in the i-th row of the characteristic matrix X. Scaling is performed to ensure that the expected feature matrix remains unchanged before and after the perturbation. It's important to note that the addition of the random perturbation is only performed during training, similar to the DropOut method, during the inference and prediction phase.
[0062] Step 3, for the multiple perturbation matrices in Step 2 Multi-level feature mixing propagation is performed separately.
[0063] Multi-order hybrid propagation takes into account more low-order local information when performing high-order propagation, that is... in Indicates orders from 0 to P. The power-law mean. Due to the perturbation operation, the propagation process will exhibit randomness in generating the adjacency matrix depending on the perturbation. Then use replace Perform multi-stage hybrid propagation.
[0064] Because the perturbation is random, the characteristic matrix after each perturbation... They are not entirely the same. Considering that different perturbation matrices can provide diverse training samples for the model, encouraging it to capture more generalized node feature representations, this embodiment treats the DropNode method as a data augmentation method. Figure 3 It can be intuitively seen that different feature matrices will be generated when the perturbation matrix mask is different. After Q random perturbations and steps 2 and 3, Q enhanced feature matrices are generated. The inputs are sequentially fed into a multilayer perceptron for feature transformation and output.
[0065] Step 4: Input the features obtained from the hybrid propagation in Step 3 into the two-layer MLP.
[0066] Step 5: Use consistency regularization to constrain the prediction consistency of unlabeled data in Q augmented data, and use the inherent consistency of unlabeled data to improve the generalization ability and performance of the model, forcing the model to give similar predictions for different variants of the same sample.
[0067] By applying consistency constraints to different augmented versions of unlabeled data, the model can better understand the data distribution, enabling effective learning even with limited labeled data. Consistency regularization is based on the smoothness assumption, which states that similar data points should have similar outputs. In this way, the model is encouraged to learn the true distribution of the data, rather than simply learning the mapping between labels. This helps the model better understand the inherent structure of the data, enabling it to make accurate predictions even when faced with new and unseen data. First, the mean of all augmented versions is calculated as the center of the label distribution, i.e. Then, the "sharpening" method in the low-entropy prediction method is used based on Generate a label prediction probability Let represent the probability that the i-th pixel node is predicted to be of the j-th land cover type, and T be a hyperparameter controlling the class distribution. Low-entropy prediction methods help the model output a probability distribution that is as concentrated as possible; that is, the prediction for each sample should be highly concentrated in a certain class, rather than uniformly distributed across all possible classes. Such predictions are considered more deterministic and help improve the model's generalization ability. Subsequently, the output of the multilayer perceptron is calculated. With the generated predicted probability The distance between them is used as the consistency loss function:
[0068]
[0069] in, express A set of vectors predicting probabilities on Class C land features. Specifically, when Q = 2, it can be directly expressed as the L2 regularization of the Euclidean distance between the two outputs. The process of continuously optimizing the model and reducing the value of equation (1) is also a process of concentrating the prediction results of each pixel sample into a certain category.
[0070] Step 6: Combine the cross-entropy loss function and the consistency regularization loss function to perform gradient updates on the model parameters until convergence, and output the classification prediction result for each sample.
[0071] The overall loss function of the model is:
[0072] L HHPGN =L sup +λL re (2)
[0073] Where λ is a hyperparameter that controls the weights of the two loss functions.
[0074] The hyperspectral remote sensing image classification system based on a hybrid higher-order feature propagation graph network provided by this invention is described below. The hyperspectral remote sensing image classification system based on a hybrid higher-order feature propagation graph network described below can be referred to in correspondence with the hyperspectral remote sensing image classification method based on a hybrid higher-order feature propagation graph network described above.
[0075] Figure 4 This is a schematic diagram of the structure of a hyperspectral remote sensing image classification system based on a hybrid high-order feature propagation graph network provided in an embodiment of the present invention, as shown below. Figure 4 As shown, it includes: a generation module 41, a disturbance module 42, a propagation module 43, a constraint module 44, and an output module 45, wherein:
[0076] The generation module 41 is used to acquire a remote sensing image dataset and generate an initial map structure based on the spatial and spectral features of the dataset. The perturbation module 42 is used to repeatedly add random perturbation elements to the feature matrix of the initial map structure to generate multiple perturbation matrices, which are then used as several data augmentations. The propagation module 43 is used to perform multi-order feature mixing propagation on the multiple perturbation matrices to obtain several augmented feature matrices, which are then input into a two-layer multilayer perceptron for feature transformation to obtain the feature transformation output. The constraint module 44 is used to constrain the prediction consistency of unlabeled data in the several data augmentations using consistency regularization, and to train the model parameters of the feature transformation output using a preset loss function to obtain a trained hyperspectral remote sensing image classification model. The output module 45 is used to input the hyperspectral remote sensing image data to be processed into the hyperspectral remote sensing image classification model and output the image classification prediction result.
[0077] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network. This method includes: acquiring a remote sensing image dataset; generating an initial graph structure based on the spatial and spectral features of the remote sensing image dataset; repeatedly adding random perturbation elements to the feature matrix of the initial graph structure to generate multiple perturbation matrices, and using these multiple perturbation matrices as several data augmentations; performing multi-order feature hybrid propagation on the multiple perturbation matrices to obtain several augmented feature matrices; inputting these augmented feature matrices into a two-layer multilayer perceptron for feature transformation to obtain feature transformation output results; using consistency regularization to constrain the prediction consistency of unlabeled data in the several data augmentations; training the model parameters of the feature transformation output results using a preset loss function to obtain a trained hyperspectral remote sensing image classification model; inputting the hyperspectral remote sensing image data to be processed into the hyperspectral remote sensing image classification model, and outputting image classification prediction results.
[0078] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0079] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network provided by the above methods. The method includes: acquiring a remote sensing image dataset; generating an initial graph structure based on the spatial and spectral features of the remote sensing image dataset; repeatedly adding random perturbation elements to the feature matrix of the initial graph structure to generate multiple perturbation matrices, and using the multiple perturbation matrices as several data augmentations; performing multi-order feature hybrid propagation on the multiple perturbation matrices to obtain several augmented feature matrices; inputting the several augmented feature matrices into a two-layer multilayer perceptron for feature transformation to obtain feature transformation output results; using consistency regularization to constrain the prediction consistency of unlabeled data in the several data augmentations; training the model parameters of the feature transformation output results using a preset loss function to obtain a trained hyperspectral remote sensing image classification model; inputting the hyperspectral remote sensing image data to be processed into the hyperspectral remote sensing image classification model and outputting image classification prediction results.
[0080] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network provided by the above methods. The method includes: acquiring a remote sensing image dataset; generating an initial graph structure based on the spatial and spectral features of the remote sensing image dataset; repeatedly adding random perturbation elements to the feature matrix of the initial graph structure to generate multiple perturbation matrices, and using the multiple perturbation matrices as several data augmentations; performing multi-order feature hybrid propagation on the multiple perturbation matrices respectively to obtain several augmented feature matrices; inputting the several augmented feature matrices into a two-layer multilayer perceptron for feature transformation to obtain feature transformation output results; using consistency regularization to constrain the prediction consistency of unlabeled data in the several data augmentations; training the model parameters of the feature transformation output results using a preset loss function to obtain a trained hyperspectral remote sensing image classification model; inputting the hyperspectral remote sensing image data to be processed into the hyperspectral remote sensing image classification model and outputting image classification prediction results.
[0081] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network, characterized in that, include: Collect a remote sensing image dataset, and generate an initial map structure based on the spatial and spectral features of the remote sensing image dataset; Random perturbation elements are added repeatedly to the feature matrix of the initial graph structure to generate multiple perturbation matrices, which are then used as several data augmentations. The multiple perturbation matrices are subjected to multi-order feature mixing propagation to obtain several enhanced feature matrices. The several enhanced feature matrices are then input into a two-layer multilayer perceptron for feature transformation to obtain the feature transformation output result. Consistency regularization is used to constrain the prediction consistency of unlabeled data in the several data augmentations, and the model parameters of the feature transformation output are trained using a preset loss function to obtain a trained hyperspectral remote sensing image classification model. The hyperspectral remote sensing image data to be processed is input into the hyperspectral remote sensing image classification model, and the image classification prediction result is output. Random perturbation elements are added repeatedly to the feature matrix of the initial graph structure to generate multiple perturbation matrices. These multiple perturbation matrices are used as several data augmentations, including: A random mask is generated from a Bernoulli random variable, and the feature vector of each pixel in the feature matrix is determined to be retained or discarded based on the random mask. The discarded feature vectors are re-aggregated based on the random mask, perturbation probability factor, and any row vector in the feature matrix, and together with the retained feature vectors, they form the plurality of perturbation matrices. Multi-order feature mixing propagation is performed on the multiple perturbation matrices respectively to obtain several enhanced feature matrices, including: A multi-order adjacency matrix is randomly generated from the plurality of perturbation matrices, and the power exponent mean of the adjacency matrix from order 0 to multi-order is obtained based on the adjacency matrix; Multiplying the mean of the power exponents by the plurality of perturbation matrices yields the enhanced feature matrix; The enhanced feature matrix is randomly perturbed using perturbation matrices with different masks to obtain the plurality of enhanced feature matrices; Consistency regularization is used to constrain the prediction consistency of unlabeled data in the aforementioned data augmentations, including: Calculate the enhanced mean of all unlabeled data, and use the enhanced mean as the center of the label distribution; The label distribution center is sharpened using a low-entropy prediction method to generate a label prediction probability, which represents the probability that any pixel node is predicted to be any land feature. The predicted consistency is converged using a preset consistency loss function.
2. The hyperspectral remote sensing image classification method based on the hybrid high-order feature propagation graph network according to claim 1, characterized in that, The predicted consistency is converged using a preset consistency loss function, including: Calculate the distance between the feature transformation output and the sum of the label prediction probabilities, and use it as the preset consistency loss function; Wherein, the summation of the label prediction probabilities is a vector set of the prediction probabilities of the label prediction probabilities on any type of land cover.
3. The method of claim 2, wherein the method further comprises: The model parameters of the feature transformation output are trained using a preset loss function to obtain a trained hyperspectral remote sensing image classification model, including: The preset loss function is obtained by summing the product of the preset consistency loss function and the weight hyperparameter with the cross-entropy loss function; The model parameters of the feature transformation output are updated using the preset loss function. After the model converges, the hyperspectral remote sensing image classification model is output.
4. A hyperspectral remote sensing image classification system based on a hybrid high-order feature propagation graph network, based on the hyperspectral remote sensing image classification method based on the hybrid high-order feature propagation graph network in any one of claims 1 to 3, characterized in that, include: The generation module is used to acquire remote sensing image datasets and generate an initial map structure based on the spatial and spectral features of the remote sensing image datasets. The perturbation module is used to repeatedly add random perturbation elements to the feature matrix of the initial graph structure to generate multiple perturbation matrices, and to use the multiple perturbation matrices as several data augmentations. The propagation module is used to perform multi-order feature mixing propagation on the multiple perturbation matrices to obtain several enhanced feature matrices, and input the several enhanced feature matrices into a two-layer multilayer perceptron for feature transformation to obtain the feature transformation output result. The constraint module is used to constrain the prediction consistency of unlabeled data in the several data augmentations by using consistency regularization, and to train the model parameters of the feature transformation output using a preset loss function to obtain a trained hyperspectral remote sensing image classification model. The output module is used to input the hyperspectral remote sensing image data to be processed into the hyperspectral remote sensing image classification model and output the image classification prediction results.
5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network as described in any one of claims 1 to 3. 6.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network as described in any one of claims 1 to 3.
7. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the hyperspectral remote sensing image classification method based on a hybrid high-order feature propagation graph network as described in any one of claims 1 to 3.