A hyperspectral image classification method and device based on subgraph dependent neural network
The SGDNet-APE method based on subgraph-dependent neural networks solves the problems of spectral variability and scarcity of training samples in hyperspectral image classification. It achieves efficient classification of hyperspectral images through adaptive neighbor order recognition and Transformer fusion strategy.
Patent Information
- Application Number
- CN202610619799.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-08
- Publication Date
- 2026-08-25
- Estimated Expiration
- 2046-05-08
AI Technical Summary
Hyperspectral image classification suffers from spectral variability and a scarcity of training samples. Existing methods struggle to effectively handle long-range information and imbalanced topology, resulting in poor classification performance.
We employ a subgraph-dependent neural network (SGDNet-APE) to adaptively identify the order of neighbors, combine it with Transformer to construct a long-range dependency strategy, utilize a spatial-spectral unified adaptive probabilistic graph convolutional network to construct a probabilistic graph structure, enrich feature representation with embedding information in the form of residuals, and enhance position awareness through absolute position encoding.
It significantly improves the classification accuracy and robustness of hyperspectral images, and can maintain good classification performance even with few training samples, especially in regions with strong intra-class spectral variability and inter-class spectral similarity.
Smart Images
Figure CN122176423B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of hyperspectral image data processing technology, specifically to a hyperspectral image classification method and device based on a subgraph-dependent neural network. Background Technology
[0002] Hyperspectral imaging is a remote sensing technology that captures information across multiple spectral bands. Compared to traditional RGB and multispectral images, it provides more detailed spectral information and is therefore widely used in fields such as mineral exploration, precision agriculture, and vegetation monitoring. However, the performance of hyperspectral image classification is affected by two key factors: the first is the spectral variability of ground objects, which leads to differences in spectral characteristics among objects of the same category, making classification more difficult; the second is the high cost of obtaining real-world labeled objects on the ground, which can easily lead to a scarcity of training samples.
[0003] To address these issues, there are currently three main approaches for hyperspectral image classification: Convolutional Neural Networks (CNN), Transformer, and Graph Convolutional Networks (GCN). CNN, as one of the representatives of deep learning, uses convolution and pooling to perform well in tasks such as HSI classification and object detection. However, its classification performance is not particularly outstanding when dealing with long-range information extraction. In contrast, the Transformer model uses a self-attention mechanism to capture the long-range dependencies between pixels in hyperspectral images. Meanwhile, GCN also extracts information from irregular regions by using a flexible sparse graph structure and message passing mechanism, and achieves better aggregation of domain information.
[0004] The aforementioned GCN algorithm achieves feature smoothing and alleviates spectral variability by aggregating domain information through graph structures. At the same time, it can better represent complex spatial distribution relationships by utilizing irregular graph structures. In this case, GCNs, as an effective semi-supervised learning method, can establish connections between a small number of training samples and a large number of test samples through graph structures. Therefore, it can achieve good classification performance when the number of training samples is limited. However, as the aggregation depth increases, the number of neighboring nodes calculated by the root node increases exponentially, thus facing the huge challenge of neighbor node explosion.
[0005] Therefore, Subgraph Neural Networks (SGNN) were proposed. By sampling node blocks associated with a subgraph based on a subgraph, the neighborhood search is restricted to the subgraph, reducing its computational overhead. At the same time, the subgraph-based method can ensure homogeneous structural information between nodes, enabling effective intra-class information aggregation and mitigating the impact of spectral variability to some extent. Moreover, the subgraph can directly focus on regions of interest in hyperspectral images without being limited by predefined sampling neighbors, while retaining the ability to capture certain neighborhood structural information, thus preventing the loss of structural information.
[0006] While SGNN can preserve structural information between nodes and reduce information loss to some extent, the SGNN-based methods mentioned above still face a very serious problem: in the message passing paradigm, fixed K-order neighbor information is aggregated for each subgraph structure. This ignores the unbalanced topology of different subgraphs, making it difficult to effectively smooth the feature information of the subgraphs. At the same time, the intra-class spectral variability and inter-class spectral similarity in hyperspectral images lead to topological imbalance when solving the topological information within different subgraph structures.
[0007] To address this issue, this application proposes a hyperspectral image classification method and device based on a subgraph-dependent neural network, which can fully exploit the spatial, spectral, and structural features of hyperspectral images, significantly improving the accuracy and robustness of classification, thereby solving the aforementioned technical problems. Summary of the Invention
[0008] The main objective of this invention is to provide a hyperspectral image classification method and device based on a subgraph-dependent neural network. By constructing a novel subgraph-dependent neural network (SGDNet-APE), it is able to adaptively identify and capture the neighbor order of each subgraph. At the same time, it combines a transformer to construct a set of long-range dependency strategies to solve the technical problem mentioned in the background art where the global dependency relationship between nodes is destroyed due to subgraph partitioning.
[0009] The present invention solves the above-mentioned technical problems by adopting the following technical solutions: A hyperspectral image classification method based on a subgraph-dependent neural network (SGDNet-APE) is implemented via a computer device, comprising the following steps: Step S1. Input hyperspectral image data cube ,in , , They represent The height, width, and spectral dimension are optimized using the Spatial-Spectral Unified Adaptive Probabilistic Graphical Convolutional Network (SSAPGCN) method to obtain the matrix structure used to capture pairwise probabilistic connections. Finally, the overall graph structure is constructed. , This represents the number of nodes / total number of pixels; Step S2. Use the Metis algorithm to construct the overall graph structure. Divided into Each subgraph has an optimal probability connection matrix with adjacencies. Represented as There exists a corresponding connection matrix. The characteristic matrix is ; Step S3. In the feature matrix, the degree information obtained from the adjacency matrix based on the subgraph is added to the feature matrix. Embedding is used to enrich feature representation in residual form to construct residual features. A set of preset adaptive adjustment feedback thresholds Construct a set of clustering performance metrics With threshold The optimal graph convolutional layer is obtained by dynamically comparing and specifying iterative optimization methods based on the spectral differences of different subgraphs. Here, the degree information / structure degree matrix is... It is a diagonal matrix calculated based on the adjacency matrix of the subgraph, and its diagonal elements are... Represents a node The degree is calculated using the following formula: , The optimal probabilistic connection weights within the corresponding subgraph are used to reflect the importance of nodes in the subgraph topology.
[0010] Step S4. Generate a low-dimensional absolute position sequence (APS) through absolute position encoding (APE), and concatenate it with the node feature vector output by the subgraph convolution. This concatenation is used as the input to the Transformer encoder. At the same time, structural information and spatial location information are fused in the attention mechanism of the Transformer encoder. Finally, the classification result is obtained by outputting through softmax.
[0011] Preferably, the matrix structure with pairwise probability connections in step S1 The specific acquisition process includes: Step S11. Extract the spectral feature vector of each pixel from the hyperspectral image. and its spatial coordinates ; Step S12. For each pair of pixels or nodes and The comprehensive similarity metric is calculated using the following formula:
[0012] in, It is a balance parameter between spectral and spatial information. and Pixels or nodes respectively Spectral eigenvectors and their spatial coordinates and Do not use pixels or nodes Spectral eigenvectors and their spatial coordinates; Step S13. Then based on Assign probabilistic connection weights This is used to reflect the strength of the correlation between pixels; Step S14. Construct the matrix This represents a structure that captures pairwise probabilistic connections, and its computation involves a priori matrix. and adopt The F-norm is used as a regularization term to finally obtain the optimal probability connectivity matrix. The joint optimization objective function is constructed as follows: , The total number of pixels or nodes, and its constraints. That is, for nodes Its connection weights with all its neighboring nodes sum to 1, and Minimizing the objective function here results in finding an optimal set of variables. That is, A.
[0013] Preferably, the specific operation process of step S3 includes: Step S31. After obtaining the input features of each pixel or node, embed the degree information obtained from the adjacency matrix of the subgraph into the input features in the form of residuals. The residual characteristics are obtained, and their expression is:
[0014] in, For residual characteristics, The structure degree matrix represents the embedding. For balance parameters; This operation is used to enhance feature representation capabilities and alleviate the oversmoothing phenomenon in deep networks. Step S32. Using residual characteristics As initial input, within each subgraph, based on the optimized probabilistic connection weights... Perform multi-layer feature propagation and convolution, for the th For a layer, the update formula for node features is:
[0015] in, Let be the number of nodes within the subgraph, and be the th node. The information of the layer nodes is from the first It is formed by weighted aggregation of information from neighboring nodes at each layer; This step can be executed repeatedly, aggregating neighborhood information layer by layer to achieve effective feature smoothing and information fusion. Step S33. After each layer of feature propagation and convolution operation is completed, calculate the clustering performance metric of the node features in the current subgraph in real time. To dynamically assess the consistency and smoothness of features; Step S34. Measure the clustering performance. With preset adaptive threshold If a comparison is made, If the value exceeds the threshold, the next layer of feature propagation and convolution continues until... Reduced to no higher than the threshold The feature propagation and convolution operations are terminated, and the current number of convolutional layers is determined to be the optimal number of layers.
[0016] Preferably, the clustering performance metric in step S33 is... Defined as:
[0017] in, Represents the set of all categories or subgraphs. For the number of categories or subgraphs, For the first The sample set in the class or subgraph , For sample set The feature vector of the inner sample, here It reflects the similarity of node features within a homogeneous region. The smaller the value, the more consistent the features and the better the smoothing effect.
[0018] Preferably, the specific operation process of step S4 includes: Step S41. For each pixel or node, use the Absolute Position Encoding (APE) method to generate its unique Absolute Position Sequence (APS) to express its spatial location information; Step S42. Extract the original feature vector of each node. Its absolute position encoding The features are then concatenated to obtain the fused feature representation, where... This represents the feature dimension of a node after feature transformation or mapping. This dimension is preset by the network structure and is the feature dimension output by the convolutional layer. Step S43. Input the fused node feature representation into the Transformer encoder. Under the multi-head attention mechanism of the Transformer encoder, calculate the multi-head cosine similarity metric function after concatenating its features with the positional encoding. Step S44. Integrate the above multi-head cosine similarity metric function into the calculation of attention weights to obtain the final attention weights. ; Step S45. Adjust attention weights The node features are normalized by a softmax layer, and then the value vectors of neighboring nodes are weighted and aggregated based on the normalized weights to obtain updated node features. Finally, these features are input into a classifier to obtain the classification result.
[0019] Preferably, the specific process for generating the absolute position sequence in step S41 includes: The hyperspectral image is divided into several segments of size [missing information]. The number of blocks, the total number of blocks Calculated using the following formula:
[0020] in, and These are the height and width of the image, respectively. Indicates rounding up; Centered on each pixel, select a window with a size of [value missing]. A sliding window is used to count the frequency of each block number within the window, forming a sequence of length [length missing]. The frequency vector is the absolute position sequence of the pixel.
[0021] Preferably, in step S42, the concatenated result is input into a multilayer perceptron (MLP) for nonlinear mapping and dimensionality compression to obtain the final node feature representation, the expression of which is:
[0022] in, This represents a splicing operation. For structural features, For location features, It is a set of two-layer perceptron networks used to map the concatenated high-dimensional features back to the same dimension as the original features. As a group 3D space.
[0023] Preferably, the specific calculation formula for the multi-head cosine similarity measurement function in step S43 is as follows:
[0024] in, Represented as location-based features and The multi-headed cosine similarity measure function, For the number of attention heads, For the first The weight matrix of each node The original characteristics, and They are nodes and nodes Location encoding, and They are nodes node and nodes The node feature representations obtained by inputting into the multilayer perceptron.
[0025] Preferably, the attention weight in step S44 The calculation formula is:
[0026] in, and They are nodes Query vectors and nodes and key vector, For preset weighting coefficients, For feature dimension, symbol Representing vectors with vector The dot product operation.
[0027] Preferably, in step S45, the Softmax function is used to assign attention weights. Normalization is performed using a softmax layer to obtain normalized coefficients. These normalized coefficients are then compared with the value vectors of their corresponding neighboring nodes. Weighted summation is performed to aggregate and obtain the updated node features; finally, the updated node features are input into the classifier to obtain the final classification result. The specific calculation formula for updating the node features is as follows:
[0028]
[0029] In the formula, To normalize the attention coefficient, For nodes The value vector, For nodes Updated feature representation.
[0030] In another aspect, the present invention also discloses a hyperspectral image classification device based on a subgraph-dependent neural network, including a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the hyperspectral image classification method based on the subgraph-dependent neural network as described above.
[0031] As can be seen from the above technical solution, the present invention provides a hyperspectral image classification method and device based on a subgraph-dependent neural network. Compared with the prior art, the present invention has the following advantages: 1. This invention constructs a pairwise probabilistic graph structure by employing a spatial-spectral unified adaptive probabilistic graph convolutional network method. It utilizes the similarity between spectral feature information and spatial coordinate information of nodes to construct a probabilistic adjacency matrix A, which can provide rich global relational references for the partitioning of subgraphs.
[0032] 2. In feature representation, this invention embeds degree information into feature vectors in the form of residuals, which not only better represents the neighborhood connections and topological structure of nodes, but also helps to alleviate the oversmoothing problem caused by deep training.
[0033] 3. This invention proposes an adaptive subgraph convolution, which uses clustering performance metrics to dynamically determine the adaptive adjustment feedback threshold. This enables the selection of the optimal number of convolutional layers k for imbalanced subgraph structures, ultimately achieving flexible feature information aggregation.
[0034] 4. By introducing the transformer concept and adopting a fusion strategy of APE and SGDNet, this invention not only improves the location awareness and cross-subgraph representation capabilities, but also enhances the model's modeling effect on the global context, enabling it to achieve better classification performance in land class edge regions, structurally fragmented regions, and categories with scarce training samples.
[0035] 5. In summary, this invention can fully exploit the spatial, spectral, and structural features of hyperspectral images, significantly improving the accuracy and robustness of classification.
[0036] It should be understood that the descriptions in this section are not intended to identify key or essential features of embodiments of the invention, nor are they intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Of course, implementing any product of the invention does not necessarily require achieving all of the advantages described above simultaneously. Attached Figure Description
[0037] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a schematic diagram of the overall method flow architecture of the present invention; Figure 2 This is a schematic diagram comparing the ground real image (left) and false color (right) of the Indian Pine dataset of this invention; Figure 3 This is a schematic diagram comparing the ground truth image (left) and false color image (right) of the PaviaU dataset of this invention; Figure 4 This is a schematic diagram comparing a true ground image (left) and a false-color image (right) of Houston University, as presented in this invention. Figure 5 This is a schematic diagram illustrating the process of obtaining pixel block label weights (APSs) according to the present invention; Figure 6 This is a schematic diagram showing the comparison of OA (Office Automation) for different samples on the Indian Pines dataset of this invention. Detailed Implementation
[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0039] For details in the embodiments, please refer to Figures 1 to 6 .
[0040] On the one hand, such as Figure 1 As shown. The hyperspectral image classification method based on subgraph-dependent neural networks proposed in this embodiment of the invention includes the following steps: L1. Input hyperspectral image data cube H, W, and B represent respectively Height, width, and spectral dimension.
[0041] L2. The Spatial-Spectral Unified Adaptive Probabilistic Graphical Convolutional Network (SSAPGCN) method is employed to optimize and obtain the structure for capturing pairwise probabilistic connections. A∈ Thus, the overall graph structure is obtained. .
[0042] Specifically, the optimization operation here includes the following steps: L21. Extracting the spectral feature vector of each pixel from a hyperspectral image. and its spatial coordinates ; L22. For each pair of pixels (or nodes) and Calculate the overall similarity metric: in, It is a balance parameter between spectral and spatial information; L23. Constructing a matrix , representing the structure used to capture pairwise probabilistic connections, based on Assign probabilistic connection weights A prior matrix is introduced to reflect the strength of correlation between pixels. At the same time, The F-norm is used as a regularization term to construct the joint optimization objective function: The optimal probability connectivity matrix is obtained. In the optimization process of probabilistic connection weights, for each weight Apply the following constraints: The weights are ensured to be non-negative and normalization is performed.
[0043] At this point, by using the spatial-spectral unified adaptive probabilistic graph convolutional network method to construct a pairwise probabilistic graph structure, and utilizing the similarity of spectral feature information and spatial coordinate information between nodes to construct the probabilistic adjacency matrix A, rich global relational references can be provided for the partitioning of subgraphs.
[0044] L3. Use the efficient Metis algorithm to divide the large graph into... Subgraph, adjacency matrix It can be represented as The characteristic matrix is .
[0045] L4. Embedding degree information into residual form to enrich feature representation and construct residual features. An adaptive adjustment feedback threshold ξ is designed to obtain the optimal graph convolutional layer based on the spectral differences of different subgraphs, by employing clustering performance.
[0046] The specific steps involved are as follows: L41. After obtaining the input features of each pixel (or node), the degree information... Embedding input features in the form of residuals The residual characteristics are obtained; specifically: , The structure degree matrix represents the embedding. To balance parameters; this operation is used to enhance feature representation and alleviate oversmoothing in deep networks; L42. Using residual features as initial input, within each subgraph, based on the optimized probabilistic connection weights... Perform multi-layer feature propagation and convolution; for the th The update formula for layer and node features is:
[0047] in, Let be the number of nodes within the subgraph, and be the th node. The information of the layer nodes is from the first The information of neighboring nodes at each layer is weighted and aggregated; this step can be executed cyclically to aggregate neighborhood information layer by layer, achieving effective smoothing of features and information fusion. L43. After each layer of feature propagation and convolution operation is completed, the clustering performance metric of the node features within the current subgraph is calculated in real time. This is used to dynamically assess the consistency and smoothness of features; the metric is defined as:
[0048] in, Represents the set of all categories or subgraphs. The number of categories (subgraphs), For the first The sample set in the class (subgraph), , For sample set Feature vectors of internal samples; It reflects the similarity of node features within a homogeneous region; the smaller the value, the more consistent the features and the better the smoothing effect. L44. The above intra-class distance metric With preset adaptive threshold Compare; when Reduced to no higher than the threshold When the time is right, automatically terminate feature propagation and convolution operations, and determine the current number of convolutional layers as the optimal number of layers; if If the value is still higher than the threshold, continue with the next layer of feature propagation and convolution until the termination condition is met.
[0049] By embedding degree information as a residual into the feature vector in the feature representation, we can not only better represent the neighborhood connectivity and topology of nodes, but also help alleviate the oversmoothing problem caused by deep training.
[0050] Furthermore, by proposing adaptive subgraph convolution, a clustering performance metric is used to dynamically determine the adaptive adjustment feedback threshold, thus enabling the selection of the optimal number of convolutional layers k for imbalanced subgraph structures, ultimately achieving flexible feature information aggregation.
[0051] L5. A low-dimensional absolute position sequence (APS) is generated through absolute position encoding (APE), and combined with the node feature vector output by the subgraph convolution in a concatenated manner. This is used as the input of the Transformer encoder. At the same time, structural information and spatial position information are fused in the attention mechanism. Finally, the classification result is obtained through softmax.
[0052] The specific data processing of the Transformer encoder at this time includes the following steps: L51. For each pixel (or node), an absolute position encoding (APE) method is used to generate its unique absolute position sequence (APS) to represent its spatial location information; specifically, the hyperspectral image is first divided into several sub-arrays of size [missing information]. The number of blocks, the total number of blocks Calculated using the following formula: ,in and These are the height and width of the image, respectively. This indicates rounding up; subsequently, a window of size is selected, centered on each pixel. A sliding window is used to count the frequency of each block number within the window, forming a sequence of length [length missing]. The frequency vector; this frequency vector is the absolute position sequence (APS) of the pixel; L52. Extract the original feature vector of each node. Its absolute position encoding The features are concatenated to obtain the fused feature representation; specifically, this is achieved through a concatenation operation. Structural and location features are combined and then input into a multilayer perceptron (MLP) for nonlinear mapping and dimensionality compression to obtain the final node feature representation: MLP is a two-layer perceptron network used to map the concatenated high-dimensional features back to the same dimension as the original features. L53. The fused node feature representation is input into the Transformer encoder. Under the multi-head attention mechanism, the multi-head cosine similarity metric function of its features and positional encoding concatenation is calculated. The specific calculation formula is as follows: in, For the number of attention heads, For the first The weight matrix of each node The original characteristics, and They are nodes and nodes Location encoding, and They are nodes node and nodes Input the node feature representations obtained from the multilayer perceptron. This is represented as a splicing operation; L54. The above multi-head cosine similarity measurement function This is incorporated into the calculation of attention weights to obtain the final attention weights. The calculation formula is as follows: ,in, and They are nodes and The query vector and key vector, For preset weighting coefficients, For feature dimensions; L55. Finally, the attention score is passed through a softmax layer to obtain the classification result.
[0053] In practical applications, this method introduces the transformer concept and adopts a fusion strategy of APE and SGDNet, which not only improves the location awareness and cross-subgraph representation capabilities, but also enhances the model's modeling effect on the global context, enabling it to achieve better classification performance in land class edge regions, structurally fragmented regions, and categories with scarce training samples.
[0054] In a specific implementation process, by constructing such Figure 2 , 3 Data comparison was performed on the classic dataset images shown in Figure 4, using methods such as... Figure 5 The diagram shows the process of obtaining pixel block label weights. Low-dimensional absolute position sequences are obtained through absolute position encoding, where r and s are used to define the width and height of the hyperspectral image (HSI), respectively. In this implementation, the (3r+5)th pixel is specifically focused on, and its block label weight is obtained through absolute position encoding (APE). At this time, the patch label weight is regarded as APS and will be used as the feature input to the subsequent position encoder.
[0055] After processing using the method described in this application, and through data comparison, the comparison results of different methods and the SGDNet-APE method on three datasets regarding OA(STD) (%), AA(STD) (%), and KC(STD) (%) are shown in Tables 1, 2, and 3 below: Table 1: Comparison results of different methods and the SGDNet-APE method on the Indian Pine dataset (partial data)
[0056] Table 2: Comparison results of different methods and the SGDNet-APE method on the PaviaU dataset (partial data)
[0057] Table 3: Comparison results of different methods and the SGDNet-APE method on the Houston University dataset (partial data)
[0058] The result at this time is as follows Figure 6The line graph showing the OA comparison of different methods on different samples on the Indian Pines dataset illustrates that the OA of the SGDNet-APE model is superior to other methods under different numbers of labeled samples.
[0059] Furthermore, compared to the GCN method, the SGDNet-APE model can adaptively perform graph convolutions, while optimizing graph structure information through output features and the graph's own structure, thereby achieving better convolution results to adapt to sample information under different graph structures.
[0060] Finally, it can be observed that even with relatively small sample sizes, such as 0.5% and 1%, the improved SGDNet-APE method can significantly outperform other classification methods in terms of classification performance.
[0061] In summary, due to the limited training samples and spectral variability of HSI, most popular classification methods perform poorly. However, SGDNet-APE with Transformer can handle this situation, enhancing the model's long-range dependency information and ensuring the extraction of global information, thereby improving classification accuracy.
[0062] In summary, this method achieves efficient feature extraction and classification of hyperspectral images by organically combining multiple techniques such as spatial-spectral feature fusion, graph structure modeling, subgraph partitioning, absolute position encoding, and self-attention mechanism.
[0063] In summary, the method of this invention can fully exploit the spatial, spectral, and structural features of hyperspectral images, significantly improving the accuracy and robustness of classification, and has broad application prospects.
[0064] On the other hand, the present invention also discloses a hyperspectral image classification device based on a subgraph-dependent neural network, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.
[0065] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.
[0066] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the hyperspectral image classification methods based on subgraph-dependent neural networks described above.
[0067] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above methods.
[0068] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus. Memory, used to store computer programs; When the processor executes the program stored in memory, it implements the above-mentioned hyperspectral image classification method based on subgraph-dependent neural networks.
[0069] The communication bus mentioned in the above-mentioned electronic devices can be a standard bus for interconnecting peripheral components or an extended industrial standard structure bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.
[0070] The communication interface is used for communication between the aforementioned electronic devices and other devices.
[0071] The memory may include random access memory or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0072] The processors mentioned above can be general-purpose processors, including central processing units, network processors, etc.; they can also be digital signal processors, application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0073] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, an optical medium, or a semiconductor medium, etc.
[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
[0075] Furthermore, it should be noted that if any directional indication (such as up, down, left, right, front, back, etc.) is involved in the embodiments of the present invention, the directional indication is only used to explain the relative positional relationship and movement of each component in a specific posture. If the specific posture changes, the directional indication will also change accordingly.
[0076] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions. Taking "A and / or B" as an example, it includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.
Claims
1. A hyperspectral image classification method based on a subgraph-dependent neural network, characterized in that, include: Step S1. Input hyperspectral image data cube The matrix structure is obtained by optimizing a spatial-spectral unified adaptive probabilistic graphical convolutional network. Construct the overall graph structure ; Step S2. Use the Metis algorithm to construct the overall graph structure. Partitioning a subgraph, an optimal probability connection matrix exists for adjacency. and corresponding feature matrix ; Step S3. In the feature matrix, the degree information obtained from the adjacency matrix based on the subgraph is added to the feature matrix. Embedded in residual form, residual features are constructed. Construct a set of clustering performance metrics With preset adaptive adjustment feedback threshold Dynamic comparison to obtain the optimal graph convolutional layer; Step S4. Generate a low-dimensional absolute position sequence through absolute position encoding, concatenate it with the node feature vector output from the subgraph convolution as input to the Transformer encoder, fuse structural information and spatial position information, and finally output the classification result; The specific operation process of step S4 includes: Step S41. For each pixel or node in the hyperspectral image, generate its unique absolute position sequence using the absolute position encoding method; Step S42. Extract the original feature vector of each node. Its absolute position encoding The features are then concatenated to obtain the fused feature representation. Step S43. Input the fused node feature representation into the Transformer encoder. Under the multi-head attention mechanism of the Transformer encoder, calculate the multi-head cosine similarity metric function after concatenating its features with the positional encoding. Step S44. Integrate the above multi-head cosine similarity metric function into the calculation of attention weights to obtain the final attention weights. ; Step S45. Adjust attention weights The node features are normalized by a softmax layer, and then the value vectors of neighboring nodes are weighted and aggregated based on the normalized weights to obtain updated node features. Finally, the updated features are input into a classifier to obtain the classification result. The specific calculation formula for the multi-head cosine similarity measurement function in step S43 is as follows: in, Represented as location-based features and The multi-headed cosine similarity measure function, For the number of attention heads, For the first The weight matrix of each node The original characteristics, and They are nodes and nodes Location encoding, and They are nodes node and nodes The node feature representation obtained by inputting into the multilayer perceptron; Attention weights in step S44 The calculation formula is: in, and They are nodes Query vectors and nodes and key vector, For preset weighting coefficients, For feature dimension, express and The dot product operation.
2. The hyperspectral image classification method based on subgraph-dependent neural networks as described in claim 1, characterized in that, The matrix structure of pairwise probability connections in step S1 The specific acquisition process includes: Step S11. Extract the spectral feature vector of each pixel from the hyperspectral image. and its spatial coordinates ; Step S12. For each pair of pixels or nodes and The comprehensive similarity metric is calculated using the following formula: in, It is a balance parameter between spectral and spatial information. and Pixels or nodes respectively Spectral eigenvectors and their spatial coordinates and Do not use pixels or nodes Spectral eigenvectors and their spatial coordinates; Step S13. Then based on Assign probabilistic connection weights This is used to reflect the strength of the correlation between pixels; Step S14. Construct the matrix This represents a structure that captures pairwise probabilistic connections, and its computation involves a priori matrix. and adopt The F-norm is used as a regularization term to finally obtain the optimal probability connectivity matrix. The joint optimization objective function is constructed as follows: , The total number of pixels or nodes.
3. The hyperspectral image classification method based on subgraph-dependent neural networks as described in claim 2, characterized in that, The specific operation process of step S3 includes: Step S31. After obtaining the input features of each pixel or node, embed the degree information obtained from the adjacency matrix of the subgraph into the input features in the form of residuals. The residual characteristics are obtained, and their expression is: in, For residual characteristics, The structure degree matrix represents the embedding. For balance parameters; Step S32. Using residual characteristics As initial input, within each subgraph, based on the optimized probabilistic connection weights... Perform multi-layer feature propagation and convolution, for the th For a layer, the update formula for node features is: in, Let be the number of nodes in the subgraph, and be the th node. The information of the layer nodes is from the first It is formed by weighted aggregation of information from neighboring nodes at each layer; Step S33. After each layer of feature propagation and convolution operation is completed, calculate the clustering performance metric of the node features in the current subgraph in real time. To dynamically assess the consistency and smoothness of features; Step S34. Measure the clustering performance. With preset adaptive threshold If a comparison is made, If the value exceeds the threshold, the next layer of feature propagation and convolution continues until... Reduced to no higher than the threshold The feature propagation and convolution operations are terminated, and the current number of convolutional layers is determined to be the optimal number of layers.
4. The hyperspectral image classification method based on subgraph-dependent neural networks as described in claim 3, characterized in that, The clustering performance metrics in step S33 Defined as: in, Represents the set of all categories or subgraphs. For the number of categories or subgraphs, For the first The sample set in the class or subgraph , For sample set The feature vector of the internal sample.
5. The hyperspectral image classification method based on subgraph-dependent neural networks as described in claim 3, characterized in that, The specific process for generating the absolute position sequence in step S41 includes: The hyperspectral image is divided into several segments of size [missing information]. The number of blocks, the total number of blocks Calculated using the following formula: in, and These are the height and width of the original hyperspectral image data cube, respectively. Indicates rounding up; Centered on each pixel, select a window with a size of [value missing]. A sliding window is used to count the frequency of each block number within the window, forming a sequence of length [length missing]. The frequency vector is the absolute position sequence of the pixel.
6. The hyperspectral image classification method based on subgraph-dependent neural networks as described in claim 3, characterized in that, In step S42, the concatenated result is input into a multilayer perceptron for nonlinear mapping and dimensionality compression to obtain the final node feature representation, the expression of which is: in, This represents a splicing operation. For structural features, For location features, It is a set of two-layer perceptron networks used to map the concatenated high-dimensional features back to the same dimension as the original features. As a group 3D space.
7. A hyperspectral image classification device based on a subgraph-dependent neural network, characterized in that, The device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the hyperspectral image classification method based on a subgraph-dependent neural network as described in any one of claims 1 to 6.