Hyperspectral image classification method and system based on dynamic causal graph

Abstract

The invention relates to an image processing technology, in particular to a hyperspectral image classification method and system based on a dynamic causal graph. The method comprises the following steps: constructing an input sequence feature from a hyperspectral image block; constructing a dynamic causal structure learning module, and performing directed relation modeling on the input sequence features to obtain a sparse dynamic causal adjacent structure; constructing an intervention enhancement graph attention spreading module, and introducing a sparse dynamic causal adjacency structure into a graph attention spreading process as explicit causal priori; building a multi-scale causal feature pyramid network, repeatedly executing dynamic causal structure learning and intervention enhancement graph attention propagation on different levels, and realizing hierarchical modeling from a local relation to a global layout relation; forming a dynamic causal graph network model by the pyramid network and the classifier, and performing joint training; and inputting a to-be-detected hyperspectral image block into the trained network model, and outputting a classification result. According to the invention, the accuracy, stability and reliability of hyperspectral image classification are improved.

Description

A Hyperspectral Image Classification Method and System Based on Dynamic Causal Graphs Technical Field

[0001] This invention relates to image processing technology, specifically to a hyperspectral image classification method and system based on dynamic causal graphs. Background Technology

[0002] Hyperspectral imaging can acquire continuous and rich spectral information at every spatial location while preserving the spatial structure information of the scene. Therefore, it has important application value in fields such as land cover mapping, ecological environment monitoring, precision agriculture, urban analysis, and forestry resource surveys. Compared with ordinary remote sensing images, hyperspectral images combine high spectral resolution and spatial texture information, enabling precise identification of complex land cover types.

[0003] As sensor spatial resolution continues to improve and remote sensing observation range expands, the distribution of ground features in large-scale hyperspectral scenes is becoming increasingly complex. Different types of ground features often exhibit similar local textures with only slight spectral differences, while features within the same category may show strong intra-class differences due to variations in illumination, mixed pixels, background interference, and imaging conditions. Therefore, large-scale hyperspectral image classification not only needs to characterize local spatial-spectral details but also needs to model long-range dependencies and spatial layout relationships over a larger scale.

[0004] In recent years, convolutional neural networks (CNNs), Transformer networks, and Mamba-like models have been increasingly applied to hyperspectral image classification. CNNs can effectively extract local spectral-spatial features, but their modeling capabilities are often limited by their local receptive field, resulting in insufficient ability to express long-distance spatial dependencies in large-scale scenes. Transformer networks, relying on self-attention mechanisms, can explicitly model long-range interactions, showing promising application prospects in large-scale hyperspectral image classification. Mamba-like models, on the other hand, are highly efficient in modeling long sequences and are also gradually being used in hyperspectral image classification tasks.

[0005] However, existing methods still primarily rely on statistical correlations in observational data for feature interaction modeling. In complex, large-scale scenes, the relationships between different regions that are truly beneficial for classification change dynamically with variations in spatial context, background composition, imaging conditions, and local structure. If only statistical correlations are relied upon, the model may easily mistake accidental co-occurrence relationships brought about by confounding factors such as background, shadows, and mixed pixels for stable discriminative relationships, leading to unstable decisions and insufficient robustness.

[0006] Furthermore, existing methods typically lack a structural learning mechanism that can explicitly describe the directed dependencies between spatial locations, and also lack an integrated modeling approach that can directly use this structure to constrain the feature propagation process. Although attention mechanisms can provide weight assignments between different locations, attention weights obtained solely from statistical correlation are not equivalent to stable discriminative dependencies. It is difficult to distinguish which spatial locations truly have a lasting impact on the classification of the current location, and which spatial locations only show strong correlation in the current sample due to background co-occurrence.

[0007] Meanwhile, large-scale hyperspectral scenes simultaneously contain local detail relationships, regional structural relationships, and global layout relationships. Single-scale modeling methods struggle to accommodate discriminative information at different levels, limiting classification accuracy and stability. Therefore, it is necessary to propose a novel classification scheme for large-scale hyperspectral images. Summary of the Invention

[0008] To address the technical problems existing in the prior art, this invention proposes a hyperspectral image classification method and system based on dynamic causal graphs. This method can explicitly learn sparse directed structural relationships that dynamically adjust with changes in spatial context, and directly apply these sparse directed structural relationships to the feature propagation process. At the same time, it achieves multi-scale relationship modeling from local to global through a hierarchical approach, thereby improving the accuracy, stability, and reliability of large-scale hyperspectral image classification.

[0009] On one hand, embodiments of the present invention provide a hyperspectral image classification method based on dynamic causal graphs, comprising the following steps:

[0010] S1. Extract the initial spectral spatial feature map from the hyperspectral image patch and convert the initial spectral spatial feature map into a sequence feature representation to construct the input sequence feature; during the conversion into a sequence feature representation, expand the initial spectral spatial feature map according to the spatial location so that each spatial location corresponds to a spatial location feature unit;

[0011] S2. Construct a dynamic causal structure learning module to model the directed relationship of the input sequence features under spatial constraints, learn the dynamic causal adjacency structure between spatial location feature units, and obtain a sample-adaptive sparse dynamic causal adjacency structure.

[0012] S3. Construct an intervention-enhanced graph attention propagation module, introducing a sparse dynamic causal adjacency structure as an explicit causal prior into the graph attention propagation process to obtain query vectors, key vectors, and value vectors; and in the training phase, randomly replacing some value vectors to obtain updated feature representations.

[0013] S4. Construct a multi-scale causal feature pyramid network, which includes multiple causal blocks stacked layer by layer. Each causal block includes a dynamic causal structure learning module, an intervention enhancement graph attention propagation module, and a residual connection and normalization processing unit connected in sequence. The multi-scale causal feature pyramid network repeatedly performs dynamic causal structure learning and intervention enhancement graph attention propagation at different levels to achieve hierarchical modeling from local relationships to global layout relationships.

[0014] S5. Combine the constructed multi-scale causal feature pyramid network with the classifier to form a dynamic causal graph network model; jointly train the dynamic causal graph network model, set the total objective function as a weighted sum of classification loss and causal regularization loss, and update the parameters of each module in the dynamic causal graph network model end-to-end.

[0015] S6. Input the hyperspectral image patch to be tested into the trained dynamic causal graph network model. After dynamic causal structure learning, intervention enhancement graph attention propagation and multi-scale causal feature modeling, the corresponding classification result is output.

[0016] Among them, the sparse dynamic causal adjacency structure refers to the dynamic directed dependency relationship between feature units of different spatial locations learned for classification tasks; the dynamic directed dependency relationship is used to characterize the relative influence of each spatial location on the classification judgment of the target location, and to guide the information flow path in the attention propagation process.

[0017] On the other hand, embodiments of the present invention also provide a hyperspectral image classification system based on a dynamic causal graph, implemented by sampling the above-mentioned hyperspectral image classification method, wherein the hyperspectral image classification system includes:

[0018] A convolutional feature extraction network is used to extract initial spectral spatial feature maps from hyperspectral image patches and convert the initial spectral spatial feature maps into sequence feature representations to construct input sequence features. During the conversion into sequence feature representations, the initial spectral spatial feature maps are expanded according to spatial locations, so that each spatial location corresponds to a spatial location feature unit.

[0019] The multi-scale causal feature pyramid network consists of multiple stacked causal blocks. Each causal block includes a dynamic causal structure learning module, an intervention enhancement graph attention propagation module, and a residual connection and normalization processing unit connected in sequence. The multi-scale causal feature pyramid network repeatedly performs dynamic causal structure learning and intervention enhancement graph attention propagation at different levels to achieve hierarchical modeling from local relationships to global layout relationships.

[0020] A classifier is used to connect with a multi-scale causal feature pyramid network to form a dynamic causal graph network model;

[0021] For the hyperspectral image patch to be tested, the corresponding classification result is output after dynamic causal structure learning, intervention-enhanced graph attention propagation and multi-scale causal feature modeling in the dynamic causal graph network model;

[0022] Among them, the dynamic causal structure learning module is used to model the directed relationship of the input sequence features under spatial constraints, learn the dynamic causal adjacency structure between spatial location feature units, and obtain the sample adaptive sparse dynamic causal adjacency structure.

[0023] The intervention-enhanced graph attention propagation module is used to introduce sparse dynamic causal adjacency structure as an explicit causal prior into the graph attention propagation process to obtain query vector, key vector and value vector; and to perform random replacement intervention on some value vectors during the training phase to obtain updated feature representations.

[0024] Sparse dynamic causal adjacency structure refers to the dynamic directed dependency relationship between feature units of different spatial locations learned for classification tasks; the dynamic directed dependency relationship is used to characterize the relative influence of each spatial location on the classification judgment of the target location, and to guide the information flow path in the attention propagation process.

[0025] Compared with the prior art, the technical effects achieved by the present invention include:

[0026] 1. This invention learns a sparse dynamic causal adjacency structure that adapts to samples under spatial constraints through a dynamic causal structure learning mechanism. This enables the model to explicitly characterize the sparse directed structural relationships that dynamically adjust with changes in spatial context in complex, large-scale scenes, serving as a discriminative dependency relationship. This effectively enhances the structural expressiveness and classification stability in complex scenes.

[0027] 2. This invention employs an intervention-enhanced graph attention propagation mechanism, introducing sparse dynamic causal adjacency structures as explicit priors into the graph attention propagation process, and combining random replacement interventions to suppress the influence of confounding factors and accidental co-occurrence information, thereby enhancing the robustness of feature propagation and the reliability of classification decisions under complex background conditions.

[0028] 3. This invention proposes a multi-scale causal feature pyramid modeling strategy, which achieves hierarchical modeling from local detail relationships and regional structural relationships to global semantic layout relationships by stacking multiple causal blocks layer by layer. This strategy can simultaneously take into account local spectral details and global structural information, thereby improving the overall discrimination ability in complex large-scale hyperspectral image classification tasks.

[0029] 4. In this invention, the relationships between spatially located feature units in a hyperspectral image are learned as a dynamic directed dependency structure for classification tasks. This dynamic directed dependency structure is not a physical causal description of the original image generation mechanism, but rather a structured expression of "which spatially located feature unit has a more stable influence on the classification of which" among different spatially located feature units. After the dynamic directed dependency structure is learned, it does not exist independently, but is directly introduced as an explicit prior into the attention propagation process to constrain the calculation of attention weights and subsequent feature information aggregation, so that feature propagation proceeds more along stable discriminative relationships, rather than being simply dominated by accidental statistical correlations.

[0030] 5. In this invention, attention propagation refers to calculating the attention weights between spatial locations based on the query vector and key vector, and then using these attention weights to perform weighted aggregation on the value vectors, thereby achieving information transfer and updating between features of different spatial locations. Therefore, the dynamic causal structure in this invention is essentially a propagation prior acting on the attention propagation process, used to indicate which spatial locations should establish information interaction and which information flow paths have greater discriminative value. Attached Figure Description

[0031] Figure 1 is a flowchart of the hyperspectral image classification method based on dynamic causal graph in an embodiment of the present invention;

[0032] Figure 2 is a schematic diagram of the overall framework of the hyperspectral image classification system based on dynamic causal graph in an embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of the dynamic causal structure learning module in an embodiment of the present invention;

[0034] Figure 4 is a schematic diagram of the structure of the intervention-enhanced graph attention module in an embodiment of the present invention. Detailed Implementation

[0035] In summary, this invention addresses the challenges of explicitly characterizing dynamic discriminative relationships, susceptibility to confounding factors, and the difficulty in unifying the modeling of local and global structural information in large-scale hyperspectral image classification. It proposes a hyperspectral image classification method and system based on dynamic causal graphs. Specifically, a dynamic causal structure learning mechanism is used to learn an adaptive sparse directed adjacency structure for samples under spatial constraints. Simultaneously, an intervention-enhanced graph attention propagation mechanism is introduced, using the dynamic causal adjacency structure as an explicit prior in feature propagation, and randomly replacing some value vectors during the training phase. Furthermore, a multi-scale causal feature pyramid network is constructed, achieving hierarchical modeling from local relationships to global layout relationships through the layered stacking of multiple causal blocks. This enables the model to complete hyperspectral image classification in complex large-scale scenes.

[0036] The technical solution of the present invention will now be clearly and completely described in conjunction with the embodiments and accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0037] Example

[0038] This embodiment proposes a hyperspectral image classification method based on dynamic causal graphs. Through dynamic causal structure learning, intervention-enhanced graph attention propagation, and multi-scale causal feature pyramid modeling, it achieves explicit relationship modeling and hierarchical feature extraction in complex large-scale scenes. The overall process includes five main stages: initial spectral spatial feature extraction, dynamic causal structure learning, intervention-enhanced graph propagation, multi-scale causal block concatenation, and joint training (see Figures 1-4). Specifically, it includes the following steps:

[0039] S1. Extract the initial spectral spatial feature map from the hyperspectral image patch and convert the initial spectral spatial feature map into a sequence feature representation to construct the input sequence feature. In the process of converting into a sequence feature representation, expand the initial spectral spatial feature map according to the spatial location so that each spatial location corresponds to a spatial location feature unit.

[0040] Because complex local dependencies and long-distance correlations exist between different spatial locations in large-scale hyperspectral images, relying solely on traditional convolutional operations or ordinary attention mechanisms for feature interaction can easily lead to the direct use of spatially accidental co-occurring patterns as the basis for judgment, making it difficult to explicitly distinguish between stable and interfering relationships. Therefore, this step first extracts initial spectral-spatial feature maps from the input hyperspectral image patch to obtain the input sequence features. Subsequent steps then model the directed relationships of the input sequence features under spatial constraints.

[0041] Let the input hyperspectral image patch be ,in Indicates batch size. This indicates the number of input bands for the hyperspectral image patch. Indicates spatial height, Indicates the width of the space.

[0042] This embodiment employs a convolutional feature extraction network to extract an initial spectral spatial feature map from hyperspectral image patches. The initial convolutional feature extraction network maps the hyperspectral image patches to a unified latent feature space, resulting in the initial spectral spatial feature map.

[0043] ;

[0044] in, This represents the initial convolutional feature extraction network. Indicates the dimension of the hidden features. This represents the initial spectral space feature map. Specifically, the convolutional feature extraction network consists of multiple layers of convolutions, normalization layers, and activation functions connected in sequence.

[0045] Subsequently, the initial spectral spatial feature map is converted into a sequence feature representation to construct the input sequence features. Specifically, the initial spectral spatial feature map is expanded according to spatial location, so that each spatial location corresponds to a spatial location feature unit, resulting in:

[0046] ;

[0047] in, Representing sequence features, Indicates the number of spatial location feature units. This represents the feature dimension of each spatial location feature unit.

[0048] In this embodiment, the "spatial location feature unit" can be understood as each position node of the input sequence feature after spatial expansion. If the input sequence feature is expanded in a pixel-level manner, it can be approximately understood as the feature unit corresponding to each pixel position; if the input sequence feature is expanded with the processed feature positions, its essence is still the feature node in spatial location.

[0049] S2. Construct a Dynamic Causal Structure Learning (DCSL) module to model the directed relationships of the input sequence features under spatial constraints, learn the dynamic causal adjacency structure between spatial location feature units, and obtain a sample-adaptive sparse dynamic causal adjacency structure.

[0050] The sparse dynamic causal adjacency structure in this embodiment does not refer to a pixel physically causing another pixel, but rather to a dynamic directed dependency learned for a classification task between feature units at different spatial locations. This dependency is used to characterize the relative strength of the influence of each spatial location on the classification of the target location (e.g., which has a more stable influence on the classification of which), and to guide the information flow path in the subsequent attention propagation process. Therefore, the sparse dynamic causal adjacency structure learned in this embodiment is a structured propagation dependency relationship oriented towards a classification task, rather than an ordinary undirected similarity relationship.

[0051] The dynamic causal structure learning module constructed in this step includes a spatially constrained candidate graph construction unit, a query projection unit, a key projection unit, a multi-head directed association modeling unit, a temperature scaling unit, and a Top-k sparse connection filtering unit. Specifically, this step further includes:

[0052] S21. In the spatial constraint candidate graph construction unit, each spatial location feature unit is assigned two-dimensional spatial coordinates. The spatial distance between spatial locations is calculated based on the two-dimensional coordinates corresponding to each spatial location feature unit. Candidate connections between any two spatial locations are determined according to a preset distance threshold, and a spatial constraint mask matrix is ​​constructed.

[0053] To avoid establishing unconstrained connections between all spatial locations, this embodiment first constructs a spatial constraint candidate graph using a spatial constraint candidate graph construction unit. The spatial constraint candidate graph construction unit first performs gridded encoding on the spatial coordinates of the initial spectral spatial feature map to obtain the row and column coordinates corresponding to each spatial location feature unit; then, it calculates the spatial distance between spatial locations based on the row and column coordinates, and compares the calculated spatial distance with a preset distance threshold to determine candidate connections between any two spatial locations, thus obtaining a binarized spatial constraint mask matrix.

[0054] The binarized spatial constraint mask matrix is ​​used in subsequent directed association modeling to restrict candidate edges, allowing connections only between spatial location feature units located within a preset neighborhood.

[0055] Specifically, by constructing units through spatially constrained candidate graphs, each spatial location feature unit is assigned two-dimensional spatial coordinates, let the first unit be... The two-dimensional spatial coordinates of each spatial location feature unit are: , No. The two-dimensional spatial coordinates of each spatial location feature unit are: Then spatial location Spatial location Euclidean distance between for:

[0056] ;

[0057] Euclidean distance That is, the spatial distance between spatial locations.

[0058] Based on spatial distance, spatial location is determined according to a preset distance threshold. Spatial location Whether candidate connections are allowed is determined to identify candidate connections between any two spatial locations, constructing a spatial constraint mask matrix to form a spatial constraint candidate graph. The determination of whether candidate connections are allowed is as follows:

[0059] ;

[0060] in, Indicates spatial location Spatial location Are candidate connections allowed between them? Indicates the preset distance threshold; Construct a spatial constraint mask matrix for the matrix elements. .

[0061] The role of the spatial constraint mask matrix is ​​to first limit the potential connection range in the geometric space, avoid the model from searching for relationships indiscriminately in all spatial locations, thereby reducing obviously unreasonable long-range noise connections, and also providing a candidate edge basis for subsequent learning of more stable directed dependencies.

[0062] S22. The input sequence features are obtained by querying the projection unit and the key projection unit to obtain the query vector and key vector of spatial location, respectively, and the query vector and key vector are divided into multiple heads according to the preset number of heads; in the multi-head directed association modeling unit, the directed association score between feature units of different spatial locations under each head is calculated.

[0063] Let the features of the current input sequence be:

[0064] ;

[0065] in, This represents the sequence feature representation of the input layer. Indicates batch size. Indicates the number of spatial location feature units. Indicates the feature dimension.

[0066] S221. Obtain the query vector of the spatial location through the query projection unit, and obtain the key vector of the spatial location through the key projection unit. Let the spatial location... The corresponding query vector is Spatial location The corresponding key vector is Calculate spatial location Spatial location The directed association score between them; in this embodiment, the directed association score can be expressed as:

[0067] ;

[0068] in, Indicates spatial location and spatial location The directed association score between them This indicates the transpose operation.

[0069] The "direction" here is reflected in: spatial location Spatial location The strength of the relationship with spatial location Spatial location The strength of the relationship does not need to be the same. Therefore, this score is not ordinary symmetric similarity, but a structural score that can represent directional influence.

[0070] S222. Divide the query vector and key vector into multiple heads according to the preset number of heads, and calculate the directed association score between different spatial location feature units under each head in the multi-head directed association modeling unit.

[0071] The space is divided into multiple heads (i.e., multiple subspaces) according to a preset number of heads, and the feature dimensions of each head are:

[0072] ;

[0073] in, This represents the feature dimension of each head. This indicates the number of long positions, i.e., the preset number of long positions.

[0074] For the Size, calculating spatial position Spatial location The directed association score between them is:

[0075] ;

[0076] in, Indicates the first Space under the size Spatial location The directed association score between them Indicates the first The query vector under the size Indicates the first The key vector under each head; This indicates the feature dimension, used to scale the dot product result.

[0077] S23. In the temperature scaling unit, the directed correlation score under each head is scaled and adjusted, and then the spatial constraint mask matrix is ​​applied to the scaled and adjusted directed correlation score to obtain the directed correlation score after applying spatial constraints, thereby masking the positions that do not meet the spatial constraint conditions.

[0078] In this embodiment, in order to adjust the score distribution, the directed correlation scores obtained in step S22 are first further subjected to temperature scaling:

[0079] ;

[0080] in, This represents the directed association score after scaling adjustment. This represents the temperature coefficient.

[0081] Then, the spatial constraint mask matrix is ​​applied to the scaled directed correlation score to obtain the directed correlation score with spatial constraints applied:

[0082] ;

[0083] in, This represents the directed association score after spatial constraints are applied.

[0084] S24. The sparse connection filtering unit performs Top-k sparse filtering on the candidate connections corresponding to each spatial location feature unit, and averages and aggregates the filtering results under multiple heads to obtain a dynamic causal adjacency structure.

[0085] In this embodiment, after performing Top-k sparse filtering on the candidate connections, the connection with the highest score is retained. Candidate connections:

[0086] ;

[0087] in, Indicates the first Sparse directed adjacency structure under the size; This indicates that the previous line should be retained. Set the largest element to zero, and set the rest to zero.

[0088] Finally, the selection results of multiple heads are averaged and aggregated to obtain the dynamic causal adjacency matrix:

[0089] ;

[0090] in, The final dynamic causal adjacency matrix represents the dynamic directed dependency between feature units at different spatial locations in the input sequence features, and is dynamically adjusted as the input sequence features change.

[0091] As can be seen, in step S2, the input sequence features are processed by the query projection unit and the key projection unit and then divided into low-dimensional sub-representations under multiple heads. Each head models the relationship between spatial locations and aggregates them after applying spatial constraints and sparse filtering to generate a unified sparse dynamic causal adjacency structure.

[0092] S3. Construct an Intervention Enhanced Graph Attention Propagation Module (IAGA), which introduces a sparse dynamic causal adjacency structure as an explicit causal prior into the graph attention propagation process to obtain query vectors, key vectors, and value vectors; and performs random replacement intervention on some value vectors during the training phase to obtain updated feature representations.

[0093] After obtaining the sparse dynamic causal adjacency structure, this step further utilizes the intervention-enhanced graph attention propagation module to complete feature propagation. This module includes a query-key-value joint projection unit, a random intervention unit, a causal prior enhanced attention calculation unit, a normalization and feature aggregation unit, and an output mapping unit.

[0094] Let the features of the input sequence still be denoted as:

[0095] ;

[0096] in, This represents the sequence features of the current input.

[0097] S31. In the query-key-value joint projection unit, the input sequence features are processed through a joint linear mapping to obtain the query vector, key vector, and value vector simultaneously, and then rearranged according to multiple heads.

[0098] The query vector, key vector, and value vector are specifically defined as follows:

[0099] ;

[0100] in, This indicates a joint linear projection operation. , , These represent the query vector, key vector, and value vector, respectively.

[0101] S32. During the training phase, the random intervention unit generates an intervention mask according to the preset intervention probability and randomly generates alternative features with the same dimension as the value vector; some value vectors are randomly replaced, the selected value vectors are replaced by the randomly generated alternative features, and the unselected value vectors remain unchanged, thus obtaining the overall value vector after random intervention.

[0102] The specific method for generating the intervention mask is as follows:

[0103] ;

[0104] in, Indicates an intervention mask. Indicates the probability of intervention. This represents the Bernoulli distribution.

[0105] Randomly generate alternative features with the same dimension as the value vector, specifically:

[0106] ;

[0107] in, This represents randomly generated alternative features. This represents a standard Gaussian distribution.

[0108] The final result is the overall value vector after random intervention:

[0109] ;

[0110] in, This represents the overall value vector after random intervention. This represents element-wise multiplication. A mask representing the spatial location that has not been interfered with.

[0111] It is important to emphasize that the random intervention in this embodiment applies to the value vector channel, that is, to the feature carriers that actually participate in information aggregation. The purpose of this is to prevent the model from over-relying on locally occurring, randomly stable value patterns, and to make the model more reliant on stable relationships supported by dynamic causal structures during propagation.

[0112] In this embodiment, a random intervention unit is used to perform replacement processing on a portion of the value vectors during the training phase. The random intervention unit generates an intervention mask based on the input batch, number of heads, and number of spatial locations. When a spatial location corresponding to a certain spatial location feature unit is selected, its original value vector is replaced with a randomly generated feature; otherwise, the original value vector remains unchanged. The random intervention unit is only enabled during the training phase and disabled during the inference phase.

[0113] S33. Causal Prior Enhanced Attention Propagation: In the causal prior enhanced attention calculation unit, the original attention score is calculated based on the query vector and the key vector, and the sparse dynamic causal adjacency structure obtained in step S2 is added to the original attention score to obtain the attention score after introducing the sparse dynamic causal adjacency structure.

[0114] For each head, first calculate the raw attention score based on the query vector and key vector:

[0115] ;

[0116] in, Indicates the first Space under the body Spatial location The original attention scores between them.

[0117] Then, the sparse dynamic causal adjacency matrix obtained in step S2 is directly added to the original attention score to obtain the attention score after introducing the sparse dynamic causal adjacency matrix:

[0118] ;

[0119] in, This represents the attention score after introducing a sparse dynamic causal adjacency matrix. It is a sparse dynamic causal adjacency matrix.

[0120] S34. Normalize the attention score after introducing the sparse dynamic causal adjacency structure to obtain the attention weight; the feature aggregation unit uses the attention weight to perform weighted aggregation of the value vector.

[0121] The attention weights are obtained after processing with the softmax normalized exponential function:

[0122] ;

[0123] in, Indicates the first Attention weight based on size This indicates a normalization operation.

[0124] Then, the attention weights are used to perform weighted aggregation on the value vectors after intervention to obtain the intermediate propagation features:

[0125] ;

[0126] in, Indicates the first The results of transmission under different sizes Indicates the first The value vector after random intervention under the given size.

[0127] As can be seen, the causal prior enhanced attention calculation unit in this embodiment first calculates the original attention score based on the query vector and key vector, then extends the dynamic causal adjacency structure to each head and superimposes it with the original attention score, and obtains the final attention weight after normalization.

[0128] S35. In the output mapping unit, the propagation results from multiple heads are concatenated and linearly mapped to obtain the updated feature representation.

[0129] In this embodiment, the propagation results of multiple heads are concatenated and linearly mapped to obtain the updated intermediate feature representation:

[0130] ;

[0131] in, This represents the intermediate features after attention propagation in the graph. Indicates multi-head splicing. This indicates the output linear mapping operation.

[0132] As can be seen from the above, the dynamic causal adjacency structure in this embodiment is not a replacement for the attention mechanism, but rather serves as an explicit prior constraint on the attention propagation process. The original attention mainly reflects statistical correlation, while the dynamic causal adjacency structure reflects a more stable directed discriminative dependency relationship under the current sample. When the two are combined, the propagation process is no longer dominated by positions that "appear to be relevant", but rather tends to transmit information along stable paths that are conducive to classification.

[0133] In other words, the attention propagation used in this embodiment does not refer to ordinary attention computation, but rather to a feature information aggregation process based on attention weights. Specifically, it involves first determining the attention weights between different spatial locations based on the query vector and key vector, and then using these attention weights to aggregate the value vectors of relevant spatial locations to the current spatial location to achieve feature updates. The dynamic causal adjacency structure does not replace the attention mechanism itself in this attention propagation process; rather, it imposes explicit constraints on the attention mechanism, ensuring that the propagation process considers not only statistical correlation but also the more stable directed discriminative dependencies under the current sample.

[0134] S4. Construct a multi-scale causal feature pyramid network (MCFP), which includes multiple causal blocks stacked layer by layer. Each causal block includes a dynamic causal structure learning module, an intervention enhancement graph attention propagation module, and a residual connection and normalization processing unit connected in sequence. The multi-scale causal feature pyramid network repeatedly performs dynamic causal structure learning and intervention enhancement graph attention propagation at different levels to achieve hierarchical modeling from local relationships to global layout relationships.

[0135] To enable the model to progressively expand its relational perception range at different levels, this embodiment constructs a multi-scale causal feature pyramid network. Within each causal block, the dynamic causal structure learning module is constructed in step S2, and the intervention enhancement graph attention propagation module is constructed in step S3.

[0136] S41. In each causal block, the dynamic causal structure learning module learns the sparse dynamic causal adjacency structure corresponding to the current causal block based on the features of the current layer input sequence.

[0137] Let the first The input sequence features of each causal block are: Learn the first [sequence] based on the features of the current input sequence. The dynamic causal adjacency matrix corresponding to each causal block:

[0138] ;

[0139] in, Indicates the first The dynamic causal adjacency matrix output by each causal block. This represents a dynamic causal structure learning module. Represents a space constraint mask.

[0140] S42. The intervention enhancement graph attention propagation module performs intervention enhancement graph attention propagation under the constraint of sparse dynamic causal adjacency structure to obtain the intermediate propagation features of the current causal block.

[0141] In this embodiment, the first Intermediate propagation characteristics of causal blocks:

[0142] ;

[0143] in, Indicates the first The intermediate propagation characteristics of each causal block, i.e., the propagation output of the current layer; This indicates an intervention to enhance the attention dissemination module.

[0144] S43, the residual connection and normalization processing unit adds the intermediate propagation features of the current causal block to the input sequence features of the current layer by residual addition, and performs normalization processing and feedforward transformation to obtain the output features of the current causal block; the output features of the current causal block are used as the input of the next causal block to finally obtain the multi-scale causal feature representation.

[0145] In this embodiment, the residual connection and normalization processing unit includes a residual connection network, a first normalization unit, a feedforward network, and a second normalization unit. The residual connection network adds the current layer propagation output (i.e., the intermediate propagation features of the current causal block) to the current layer input sequence features using residual addition, and the first normalization unit performs normalization processing to obtain normalized features. The normalized features are then input to the feedforward network, and the output of the feedforward network is again added to the normalized features using residual addition, and the second normalization unit performs normalization processing to form the output features of the current causal block. The output features of each causal block are:

[0146] ;

[0147] in, Indicates the first Output features of each causal block; Indicates the first Output features of each causal block; Indicates the first normalized unit. Indicates a feedforward network. This represents the second normalized unit.

[0148] In summary, step S4, based on the constructed multi-scale causal feature pyramid network, uses a cascaded causal block approach to continuously model relationships at different scales. A hierarchical relationship representation is obtained through a repetitive process of multi-layered "structure learning—propagation update—feedforward transformation." Shallow causal blocks primarily model local neighborhood relationships, and the modeled relationships mainly correspond to local neighborhood relationships. Subsequent causal blocks continue to perform new dynamic causal structure learning and graph attention propagation based on the output features of the previous layer. As the number of causal block layers increases, features are continuously updated, and the relationship information contained in the features gradually expands from local neighborhood relationships to regional relationships and spatial layout relationships of a larger scope. This gradually expands the scope of relationship modeling to regional relationships and spatial layout relationships of a larger scope. In other words, this invention does not only learn a single-level relationship structure, but continuously updates the dynamic causal adjacency structure in a layer-by-layer evolution, and continuously constrains attention propagation through the dynamic causal adjacency structure, enabling unified modeling of local relationships, regional relationships, and global layout relationships.

[0149] S5. Combine the constructed multi-scale causal feature pyramid network with the classifier to form a dynamic causal graph network model; jointly train the dynamic causal graph network model, set the total objective function as a weighted sum of classification loss and causal regularization loss, and update the parameters of each module in the dynamic causal graph network model end-to-end.

[0150] This embodiment constructs a unified backbone framework consisting of a multi-scale causal feature pyramid network and a classifier, namely a dynamic causal graph network model. This unified backbone framework is not a simple concatenation of causal structure learning, attention modeling, and multi-scale modeling. Instead, it forms a complete technical chain by first learning a dynamic directed structure, then performing attention propagation based on that structure, and finally repeating this propagation process within a hierarchical network. As a result, the model can effectively suppress the propagation of spurious correlations caused by confounding factors in complex, large-scale scenarios, improving the robustness of classification and overall discriminative ability.

[0151] The joint training process of the dynamic causal graph network model includes the calculation of classification loss and causal regularization loss. The final output features of the multi-scale causal feature pyramid network are restored to a spatial feature map and input into the classifier. The classifier outputs a class prediction result, and the classification loss is calculated based on the class prediction result and the true class label. Simultaneously, sparse constraint losses are calculated for the dynamic causal adjacency structure output by each causal block, and these are summarized to form the causal regularization loss. The classification loss and the causal regularization loss are then weighted and summed as the overall objective function for joint optimization of the entire network model. The causal regularization loss is used to apply L1 sparse constraints to the dynamic causal adjacency structure. The classifier includes an adaptive pooling unit, a flattening unit, a fully connected layer, a normalization layer, an activation layer, and an output layer.

[0152] Specifically, after the multi-scale causal feature pyramid network completes feature updates, it will finally output the features. The resulting spatial feature map is then input into the classifier. The corresponding classification output can be represented as:

[0153] ;

[0154] in, Indicates the category prediction result, This represents a classifier.

[0155] Calculate the classification loss based on the category prediction results and the true category labels:

[0156] ;

[0157] in, Represents classification loss. Represents the cross-entropy loss function. This represents the actual category label.

[0158] Simultaneously, the sparse constraint loss is calculated for the dynamic causal adjacency matrix output by each causal block, and then summarized to form the causal regularization loss:

[0159] ;

[0160] in, Indicates causal regularity loss. Represents the regularity coefficient. Indicates the total number of causal blocks. Indicates the first Each causal block corresponds to the absolute value of the dynamic causal adjacency matrix. This indicates an averaging operation.

[0161] Finally, the classification loss and the causal regularization loss are weighted and summed to obtain the overall objective function:

[0162] ;

[0163] in, This represents the overall objective function. During the training phase, the overall objective function is minimized, and the parameters of the dynamic causal structure learning module, the intervention enhancement graph attention propagation module, the multi-scale causal feature pyramid network, and the classifier are updated end-to-end.

[0164] S6. In the inference stage, the hyperspectral image patch to be tested is input into the trained dynamic causal graph network model. After dynamic causal structure learning, intervention-enhanced graph attention propagation, and multi-scale causal feature modeling, the corresponding classification result is output.

[0165] During the inference phase, the trained model parameters are fixed, and the hyperspectral image patch to be tested is input into the model. First, an initial spectral-space feature map is obtained through spectral-space feature extraction. Then convert it into sequence feature representation. Subsequently, a spatial constraint mask is constructed based on the spatial coordinates, and dynamic causal structure learning and intervention-enhanced graph attention propagation are sequentially completed through multiple causal blocks to obtain the final output features. During the inference phase, the random intervention unit is turned off, and only the dynamic causal adjacency matrix is ​​retained as an explicit prior to participate in the propagation; finally, the classification result is output by the classifier.

[0166] Based on the same inventive concept, this embodiment also provides a hyperspectral image classification system based on a dynamic causal graph, implemented using the aforementioned hyperspectral image classification method. The hyperspectral image classification system includes:

[0167] A convolutional feature extraction network is used to extract initial spectral spatial feature maps from hyperspectral image patches and convert the initial spectral spatial feature maps into sequence feature representations to construct input sequence features. During the conversion into sequence feature representations, the initial spectral spatial feature maps are expanded according to spatial locations, so that each spatial location corresponds to a spatial location feature unit.

[0168] The multi-scale causal feature pyramid network consists of multiple stacked causal blocks. Each causal block includes a dynamic causal structure learning module, an intervention enhancement graph attention propagation module, and a residual connection and normalization processing unit connected in sequence. The multi-scale causal feature pyramid network repeatedly performs dynamic causal structure learning and intervention enhancement graph attention propagation at different levels to achieve hierarchical modeling from local relationships to global layout relationships.

[0169] A classifier is used to connect with a multi-scale causal feature pyramid network to form a dynamic causal graph network model;

[0170] For the hyperspectral image patch to be tested, the corresponding classification result is output after dynamic causal structure learning, intervention-enhanced graph attention propagation and multi-scale causal feature modeling in the dynamic causal graph network model;

[0171] Among them, the dynamic causal structure learning module is used to model the directed relationship of the input sequence features under spatial constraints, learn the dynamic causal adjacency structure between spatial location feature units, and obtain the sample adaptive sparse dynamic causal adjacency structure.

[0172] The intervention-enhanced graph attention propagation module is used to introduce sparse dynamic causal adjacency structure as an explicit causal prior into the graph attention propagation process to obtain query vector, key vector and value vector; and to perform random replacement intervention on some value vectors during the training phase to obtain updated feature representations.

[0173] Sparse dynamic causal adjacency structure refers to the dynamic directed dependency relationship between feature units of different spatial locations learned for classification tasks; the dynamic directed dependency relationship is used to characterize the relative influence of each spatial location on the classification judgment of the target location, and to guide the information flow path in the attention propagation process.

[0174] Each of the above modules can be implemented using the aforementioned steps S1 to S6, and the detailed implementation process will not be repeated here.

[0175] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, and simplifications made by those skilled in the art without departing from the spirit and principle of the present invention should be considered equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. A hyperspectral image classification method based on dynamic causal graphs, characterized in that, Includes the following steps: S1. Extract the initial spectral spatial feature map from the hyperspectral image patch, and convert the initial spectral spatial feature map into a sequence feature representation to construct the input sequence feature; In the process of converting to sequence feature representation, the initial spectral spatial feature map is expanded according to spatial location, so that each spatial location corresponds to a spatial location feature unit; S2. Construct a dynamic causal structure learning module to model the directed relationships of input sequence features under spatial constraints, learn the dynamic causal adjacency structure between spatial location feature units, and obtain a sample-adaptive sparse dynamic causal adjacency structure. S3. Construct an intervention-enhanced graph attention propagation module, introducing the sparse dynamic causal adjacency structure as an explicit causal prior into the graph attention propagation process to obtain query vectors, key vectors, and value vectors; and implementing random replacement intervention on some value vectors during the training phase to obtain updated feature representations. S4. Construct a multi-scale causal feature pyramid network, including multiple stacked causal blocks. Each causal block includes a dynamically causal structure learning module, an intervention-enhanced graph attention propagation module, and residual connection and normalization processing units connected sequentially. The multi-scale causal feature pyramid network repeatedly performs dynamic causal structure learning and intervention-enhanced graph attention propagation at different levels. S5. The constructed multi-scale causal feature pyramid network and the classifier are combined to form a dynamic causal graph network model. The dynamic causal graph network model is jointly trained, and the overall objective function is set as the weighted sum of classification loss and causal regularization loss. The parameters of each module in the dynamic causal graph network model are updated end-to-end. S6. The hyperspectral image patch to be tested is input into the trained dynamic causal graph network model. After dynamic causal structure learning, intervention enhancement graph attention propagation and multi-scale causal feature modeling, the corresponding classification result is output. Among them, the sparse dynamic causal adjacency structure refers to the dynamic directed dependency relationship between feature units of different spatial locations learned for the classification task. The dynamic directed dependency relationship is used to characterize the relative influence of each spatial location on the classification judgment of the target location and to guide the information flow path in the attention propagation process.

2. The hyperspectral image classification method according to claim 1, characterized in that, The dynamic causal structure learning module constructed in step S2 includes a spatial constraint candidate graph construction unit, a query projection unit, a key projection unit, a multi-head directed association modeling unit, a temperature scaling unit, and a sparse connection filtering unit. Step S2 includes: S21, in the spatial constraint candidate graph construction unit, assigning two-dimensional spatial coordinates to each spatial location feature unit, calculating the spatial distance between spatial locations based on the two-dimensional coordinates corresponding to each spatial location feature unit, and determining candidate connections between any two spatial locations based on a preset distance threshold, thus constructing a spatial constraint mask matrix; S22, the input sequence features are obtained through the query projection unit and the key projection unit, respectively. The query vector and key vector of spatial location are obtained, and the query vector and key vector are divided into multiple heads according to a preset number of heads; in the multi-head directed association modeling unit, the directed association score between different spatial location feature units under each head is calculated respectively; S23, in the temperature scaling unit, the directed association score under each head is scaled and adjusted, and then the spatial constraint mask matrix is ​​applied to the scaled and adjusted directed association score to obtain the directed association score after applying spatial constraints; S24, the sparse connection filtering unit performs sparse filtering on the candidate connections corresponding to each spatial location feature unit, and the filtering results under multiple heads are averaged and aggregated to obtain the dynamic causal adjacency structure.

3. The hyperspectral image classification method according to claim 2, characterized in that, In step S21, the spatial constraint candidate map construction unit first performs gridded encoding on the spatial coordinates of the initial spectral spatial feature map to obtain the row and column coordinates corresponding to each spatial location feature unit; then, it calculates the spatial distance between spatial locations based on the row and column coordinates, and compares the calculated spatial distance with a preset distance threshold to determine the candidate connection between any two spatial locations, thereby obtaining a binarized spatial constraint mask matrix.

4. The hyperspectral image classification method according to claim 3, characterized in that, Step S21 determines the spatial location based on the spatial distance and a preset distance threshold. Spatial location Whether candidate connections are allowed between any two spatial locations is determined as follows: ;in, Indicates spatial location Spatial location Are candidate connections allowed between them? Indicates the preset distance threshold; Construct a spatial constraint mask matrix for the matrix elements. 。 5. The hyperspectral image classification method according to claim 4, characterized in that, Step S22 includes: S221, obtaining the query vector of the spatial location through the query projection unit, and obtaining the key vector of the spatial location through the key projection unit; let the spatial location... The corresponding query vector is Spatial location The corresponding key vector is Calculate spatial location Spatial location Directed association score between for: ;in, S222: Divide the query vector and key vector into multiple heads according to a preset number of heads, with each head having a feature dimension of 1. ,in This represents the feature dimension of each head. Indicates the preset number of heads; calculates the directed association score between feature units at different spatial locations under each head in the multi-head directed association modeling unit; for the th Size, calculating spatial position Spatial location The directed association score between them is: ;in, Indicates the first Space under the body Spatial location The directed association score between them Indicates the first The query vector under the size Indicates the first The key vector under each head.

6. The hyperspectral image classification method according to claim 5, characterized in that, Step S23 includes: First, performing temperature scaling on the directed correlation score obtained in step S22: ;in, This represents the directed association score after scaling adjustment. This represents the temperature coefficient; then, the spatial constraint mask matrix is ​​applied to the scaled directed correlation score to obtain the directed correlation score with spatial constraints applied. ;in, This represents the directed association score after spatial constraints are applied.

7. The hyperspectral image classification method according to claim 1, characterized in that, The intervention-enhanced graph attention propagation module in step S3 includes a query-key-value joint projection unit, a random intervention unit, a causal prior enhanced attention calculation unit, a normalization and feature aggregation unit, and an output mapping unit. Step S3 includes: S31, in the query-key-value joint projection unit, the input sequence features are processed through a joint linear mapping to simultaneously obtain the query vector, key vector, and value vector, and then rearranged according to multiple heads; S32, during the training phase, the random intervention unit generates an intervention mask according to a preset intervention probability, and randomly generates alternative features with the same dimension as the value vector; some value vectors are randomly replaced, and the selected value vectors are replaced by the randomly generated alternative features. The unselected value vectors remain unchanged, resulting in the overall value vector after random intervention; S33, in the causal prior enhanced attention calculation unit, the original attention score is calculated based on the query vector and key vector, and the sparse dynamic causal adjacency structure is added to the original attention score to obtain the attention score after introducing the sparse dynamic causal adjacency structure; S34, the attention score after introducing the sparse dynamic causal adjacency structure is normalized to obtain the attention weight; the feature aggregation unit uses the attention weight to perform weighted aggregation on the value vector; S35, in the output mapping unit, the propagation results under multiple heads are concatenated and linearly mapped to obtain the updated feature representation.

8. The hyperspectral image classification method according to claim 1, characterized in that, Step S4 includes: S41, in each causal block, the dynamic causal structure learning module learns the sparse dynamic causal adjacency structure corresponding to the current causal block based on the current layer input sequence features; S42, the intervention enhancement graph attention propagation module performs intervention enhancement graph attention propagation under the constraint of the sparse dynamic causal adjacency structure to obtain the intermediate propagation features of the current causal block; S43, the residual connection and normalization processing unit adds the intermediate propagation features of the current causal block to the current layer input sequence features, and performs normalization processing and feedforward transformation to obtain the output features of the current causal block; the output features of the current causal block are used as the input of the next causal block to finally obtain a multi-scale causal feature representation.

9. The hyperspectral image classification method according to claim 1, characterized in that, Step S5 involves the joint training of the dynamic causal graph network model, including the calculation of classification loss and causal regularization loss. The final output features of the multi-scale causal feature pyramid network are then restored to a spatial feature map, which is input into the classifier. The classifier outputs the class prediction results, and the classification loss is calculated based on the class prediction results and the true class labels. ;in, Represents classification loss. Represents the cross-entropy loss function. This represents the true category label; simultaneously, sparse constraint loss is calculated for the sparse dynamic causal adjacency structure output by each causal block, and the results are summarized to form the causal regularization loss: ;in, Indicates causal regularity loss. Represents the regularity coefficient. This represents the total number of causal blocks; the sparse dynamic causal adjacency structure output by each causal block is a dynamic causal adjacency matrix. Indicates the first Each causal block corresponds to the absolute value of the dynamic causal adjacency matrix. This represents the averaging operation; finally, the classification loss and the causal regularization loss are weighted and summed to obtain the overall objective function: ;in, This represents the overall objective function, used for joint optimization of dynamic causal graph network models.

10. A hyperspectral image classification system based on dynamic causal graphs, implemented using any one of the hyperspectral image classification methods of claims 1-9, characterized in that, The hyperspectral image classification system includes: a convolutional feature extraction network for extracting initial spectral spatial feature maps from hyperspectral image patches and converting the initial spectral spatial feature maps into sequence feature representations to construct input sequence features; during the conversion into sequence feature representations, the initial spectral spatial feature maps are expanded according to spatial location, so that each spatial location corresponds to a spatial location feature unit; a multi-scale causal feature pyramid network, including multiple stacked causal blocks, each causal block including a dynamically connected dynamic causal structure learning module, an intervention enhancement graph attention propagation module, and a residual connection and normalization processing unit; the multi-scale causal feature pyramid network repeatedly performs dynamic causal structure learning and intervention enhancement graph attention propagation at different levels to achieve hierarchical modeling from local relationships to global layout relationships; a classifier for connecting with the multi-scale causal feature pyramid network to form a dynamic causal graph network model; for the hyperspectral image patch to be tested, in the dynamic causal graph network model... In this process, after dynamic causal structure learning, intervention-enhanced graph attention propagation, and multi-scale causal feature modeling, the corresponding classification results are output. Specifically, the dynamic causal structure learning module models directed relationships between input sequence features under spatial constraints, learning the dynamic causal adjacency structure between spatial location feature units to obtain a sample-adaptive sparse dynamic causal adjacency structure. The intervention-enhanced graph attention propagation module introduces the sparse dynamic causal adjacency structure as an explicit causal prior into the graph attention propagation process to obtain query vectors, key vectors, and value vectors. During the training phase, random replacement interventions are applied to some value vectors to obtain updated feature representations. The sparse dynamic causal adjacency structure refers to the dynamic directed dependency relationship learned between different spatial location feature units for the classification task. This dynamic directed dependency relationship characterizes the relative strength of each spatial location's influence on the target location's classification and guides the information flow path during the attention propagation process.

Images