A hyperspectral image registration-fusion-classification integrated system and method thereof

Abstract

The application discloses a hyperspectral image registration-fusion-classification integrated system and a method thereof, realizes mutual promotion and end-to-end joint optimization of three tasks, fundamentally overcomes the defects of error amplification and insufficient utilization of fusion semantic information in a traditional cascade mode, a spatial perception registration Mamba network is provided in the application, pixel coordinates are mapped into a high-dimensional feature space, and a global rigidity and local non-rigidity double-branch decoupling architecture is designed to accurately process complex mixed deformation between images, and a classification semantic driven dynamic fusion module is further provided, so that deep classification semantic information can guide shallow image fusion, and the difficulty that a fused image lacks semantic information is broken through.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to an integrated system and method for hyperspectral image registration-fusion-classification. Background Technology

[0002] Due to the physical hardware constraints of spaceborne or airborne sensors, hyperspectral images, while achieving high spectral resolution, often suffer from insufficient spatial resolution. To compensate for this deficiency, it is typically necessary to utilize high spatial resolution panchromatic images of the same region, injecting spatial details from the panchromatic images into the hyperspectral images through image fusion techniques, thereby obtaining hyperspectral images with both high spatial and hyperspectral resolution. In actual data acquisition, hyperspectral and panchromatic images are acquired by different sensor payloads. Due to objective factors such as differences in sensor perspective and satellite platform jitter, complex geometric misregistration problems commonly exist between hyperspectral and panchromatic images. This misregistration is usually not a simple linear offset, but a complex mixture of global rigid transformations (such as translation and rotation) and local non-rigid deformations (such as local distortion caused by terrain), posing a significant challenge to subsequent image fusion and classification.

[0003] To address the aforementioned problems, existing methods typically rely on a traditional cascaded processing approach, following a cascaded flow of registration, fusion, and classification. However, this cascaded approach has two drawbacks in practical applications: 1. Pixel-level perfect registration is difficult to achieve in complex real-world scenarios. Minor deviations in the registration stage are directly fed into the fusion network, which is sensitive to spatial alignment. This results in severe edge ghosting and spectral aliasing in high-frequency regions such as ground object boundaries in the fused image, thus affecting the subsequent classification results.

[0004] 2. The optimization objective of traditional registration and fusion tasks is usually to minimize pixel-level visual errors between images, while neglecting the preservation of deep semantic information. This results in fused images that appear clear to human eyes but actually destroy semantic information and lack the semantic features required for the final land cover classification task. Summary of the Invention

[0005] In view of this, the present invention provides an integrated system and method for hyperspectral image registration-fusion-classification, which at least solves the problems of amplified inter-task errors and weak inter-task connections in the traditional cascade method in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A hyperspectral image registration-fusion-classification integrated system, comprising: Image registration module, used for high-resolution panchromatic images and low-resolution hyperspectral images Preliminary fusion characteristics Injecting spatial location encoding yields location-enhanced fusion features Two-branch decoupled Mamba network according to Output global rigid deformation characteristics and local elastic deformation characteristics After aggregation, the aggregation feature is obtained. ,based on and Image registration was completed, resulting in a registered hyperspectral image. ; The dynamic fusion module is used for... and Deep feature extraction is performed using a Mamba network, and pixel-level classification probability maps corresponding to the classification results returned by the classification module are generated. As a high-level semantic prior, a semantically driven spatial attention modulation matrix is ​​generated. ,pass To each and The corresponding deep features are weighted according to category-specific features to obtain the weighted features. ,Will After further fusion with the Mamba network and residual reconstruction, a high spatial resolution hyperspectral image is output. ; A classification module is used to classify high spatial resolution hyperspectral images. Deep mining of joint features of spatial texture and high-dimensional spectrum using an enhanced Mamba classification network. This yields the final pixel-level classification probability map.

[0007] Preferably, the image registration module includes: Shallow feature extraction units are used to extract low-resolution hyperspectral images through bilinear interpolation. Upsampling to high-resolution panchromatic images The same spatial dimensions yielded an upsampled hyperspectral image. High-resolution panchromatic images are processed through convolutional layers respectively. and upsampled hyperspectral images Projecting onto the same feature channel dimension yields the panchromatic image features. and hyperspectral features ; The preliminary fusion unit is used to calculate panchromatic image features through an adaptive gating mechanism. and hyperspectral features Fusion weight graph , fusion weight graph Preliminary fusion characteristics were obtained : ; in, This indicates element-wise multiplication; Spatial location coding unit, used in the initial feature fusion Based on this, spatial location coding is injected to obtain position-enhanced fusion features after injecting spatial perception information. .

[0008] Preferably, the specific content of the fusion weight map obtained by the preliminary fusion unit is as follows: Features of panchromatic images and hyperspectral features After concatenation along the channel dimensions, an adaptive gating mechanism is used to calculate the fusion weight map. : ; in, This represents the Sigmoid activation function. This indicates a splicing operation at the channel level. This is a gated convolution operation. , H is the height, W is the width, and D is the number of channels. It is the set of real numbers.

[0009] Preferably, the specific content of the spatial location coding unit includes: Generate a normalized two-dimensional coordinate grid in the range [-1, 1]. , and These represent the horizontal and vertical coordinates of the grid, respectively, and are concatenated into a coordinate matrix. Define learnable position scaling parameters ,calculate Spatial location features and will and By summing the residuals, we obtain the position-enhanced fusion feature after injecting spatial awareness information. : ; ; in, This indicates a coordinate-encoded convolutional layer.

[0010] Preferably, the image registration module further includes a spatial dimension flattening unit, a dual-branch decoupled Manba network, and a reshaping and aggregation unit; Spatial dimension flattening unit, used to flatten Flattened into a one-dimensional sequence in spatial dimension Where N = H × W; The two-branch decoupled Mamba network includes rigid deformation branches and non-rigid deformation branches, wherein the rigid deformation branch is determined according to... Extracting global rigid deformation features Non-rigid deformation branch extracts local elastic deformation features ; The reshaping and aggregation units utilize global average pooling and multilayer perceptron to generate rigid deformation branches, respectively. Dynamic weights of non-rigid deformation branches , Global rigid deformation characteristics and local elastic deformation characteristics After reshaping back to two-dimensional spatial features, aggregation is performed: ; in, This represents the feature matrix in a two-dimensional space, H×W×D, that reshapes a one-dimensional sequence back into its H×W×D form. Affine sampling network building blocks are used to... After global average pooling and regression multilayer perceptron, the affine transformation parameters are obtained. , A two-dimensional affine matrix containing global geometric transformation information such as translation, rotation, scaling, and shearing, through... Construct an affine sampling mesh; Registration unit for upsampling hyperspectral images based on affine sampling grid Perform geometric resampling to complete image registration and obtain the registered hyperspectral image. .

[0011] Preferably, the dynamic fusion module includes a deep feature extraction unit, the specific contents of which include: High-resolution panchromatic image Registered hyperspectral image Sequence features are extracted by feeding the data into a Mamba network, and deep spatial features with a global receptive field are output. With spectral characteristics : ; ; in, and For the initial mapping layer, and It is a deep sequence feature extraction network.

[0012] Preferably, the dynamic fusion module also includes: Spatial smoothing and normalization processing units are used to process pixel-level classification probability maps. Perform spatial smoothing and normalization to obtain a smoothed semantic probability map. : ; in, This is a smoothing convolution operation used to suppress local high-frequency noise; Spatial attention modulation matrix generation unit, used to generate a smooth semantic probability map Generate a semantically driven spatial attention modulation matrix : ; in, ; Feature weighting unit, used to... and The data is concatenated using a semantic modulation matrix. Perform category-specific feature weighting, and the weighted features : ; in, For element-wise multiplication of tensors For channel fusion convolution, It serves as a residual semantic attention gating mechanism; The image reconstruction unit is used to re-fuse the weighted features through a Mamba network and residual reconstruction to output a high spatial resolution hyperspectral image. : ; in, This indicates a convolution operation.

[0013] Preferably, the classification module includes a classification feature extraction network, the specific content of which is as follows: Channel compression layer, used to compress high-resolution hyperspectral images After inputting into the classification feature extraction network, initial dimensionality reduction is performed, resulting in... : ; in, Indicates a channel compression layer; An enhanced Mamba classification network for deep mining of joint features of spatial texture and high-dimensional spectrum. : ; in, A Mamba backbone network optimized for classification tasks. , This indicates the number of channels in the output feature map.

[0014] Preferably, the classification module includes: Mapping convolutional layers are used to determine the appropriate convolutional layer based on the given conditions. Generate the original classification logits. ; in, This represents a convolutional layer that maps the number of channels to the number of classes K; The classification head is used to generate a normalized final probability distribution map along the category dimension using the Softmax function. : ; in, , The k-th channel value represents the confidence level that the pixel belongs to the k-th type of land cover.

[0015] A hyperspectral image registration-fusion-classification integrated method includes the following steps: S1. For high-resolution panchromatic images and low-resolution hyperspectral images Preliminary fusion characteristics Injecting spatial location encoding yields location-enhanced fusion features Two-branch decoupled Mamba network according to Output global rigid deformation characteristics and local elastic deformation characteristics After aggregation, the aggregation feature is obtained. ,based on and Image registration was completed, resulting in a registered hyperspectral image. ; S2. To and Deep feature extraction is performed using the Mamba network, and the pixel-level classification probability map corresponding to the classification results fed back by S3 is generated. As a high-level semantic prior, a semantically driven spatial attention modulation matrix is ​​generated. ,pass To each and The corresponding deep features are weighted according to category-specific features to obtain the weighted features. ,Will After further fusion with the Mamba network and residual reconstruction, a high spatial resolution hyperspectral image is output. ; S3. Based on high spatial resolution hyperspectral images Deep mining of joint features of spatial texture and high-dimensional spectrum using an enhanced Mamba classification network. This yields the final pixel-level classification probability map.

[0016] As can be seen from the above technical solution, compared with the prior art, the present invention discloses an integrated system and method for hyperspectral image registration-fusion-classification, which has the following beneficial effects: 1. This invention constructs a one-stop end-to-end joint optimization framework for registration-fusion-classification, which avoids the amplification of registration errors and ensures that the generated fused image better serves the final hyperspectral image classification task.

[0017] 2. This invention designs a spatially aware registration Mamba network, which solves the problem of accurate registration under complex mixed deformation. By explicitly injecting pixel coordinates into the feature space, the Mamba network is endowed with position awareness. Furthermore, by synchronously modeling global rigid and local non-rigid deformations through a dual-branch network, accurate prediction of geometric transformation fields is achieved.

[0018] 3. This invention proposes a classification semantic-driven dynamic fusion module, which uses deep classification semantic priors to adaptively optimize the fusion network parameters, thereby significantly improving spatial resolution while accurately preserving spectral semantic features. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 The flowchart of the integrated hyperspectral image registration-fusion-classification system provided by this invention is as follows: Figure 2 This invention provides a flowchart of the image registration module in an integrated hyperspectral image registration-fusion-classification system. Figure 3 This invention provides a flowchart of the dynamic fusion module in an integrated hyperspectral image registration-fusion-classification system. Figure 4 The flowchart of the classification module in the hyperspectral image registration-fusion-classification integrated system provided by the present invention is shown. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] This invention provides an integrated hyperspectral image registration-fusion-classification system, such as... Figure 1 As shown, it includes: Image registration module, used for high-resolution panchromatic images and low-resolution hyperspectral images Preliminary fusion characteristics Injecting spatial location encoding yields location-enhanced fusion features Two-branch decoupled Mamba network according to Output global rigid deformation characteristics and local elastic deformation characteristics After aggregation, the aggregation feature is obtained. ,based on and Image registration was completed, resulting in a registered hyperspectral image. ; The dynamic fusion module is used for... and Deep feature extraction is performed using a Mamba network, and pixel-level classification probability maps corresponding to the classification results returned by the classification module are generated. As a high-level semantic prior, a semantically driven spatial attention modulation matrix is ​​generated. ,pass To each and The corresponding deep features are weighted according to category-specific features to obtain the weighted features. ,Will After further fusion with the Mamba network and residual reconstruction, a high spatial resolution hyperspectral image is output. ; A classification module is used to classify high spatial resolution hyperspectral images. Deep mining of joint features of spatial texture and high-dimensional spectrum using an enhanced Mamba classification network. This yields the final pixel-level classification probability map.

[0023] It should be noted that: The image fusion module aims to overcome the complex mixed deformations (global rigid deformation and local non-rigid deformation) between hyperspectral and panchromatic images. It enhances the spatial awareness of the Mamba network by explicitly injecting coordinate information and outputs accurate affine transformation parameters, ultimately obtaining the registered hyperspectral image. The process is as follows: Figure 2 As shown.

[0024] The dynamic fusion module utilizes the Mamba network to extract deep features from the registered image and the panchromatic image, and focuses on using the classification probability map fed back by the subsequent classification network as high-level semantic guidance for dynamic parameter modulation, thereby achieving dynamic semantic fusion at the feature level. Figure 3 As shown.

[0025] The classification module utilizes the fused and reconstructed high-fidelity, high-resolution hyperspectral images. To perform the final high-precision prediction of land cover categories, specifically as follows: Figure 4 As shown.

[0026] To further implement the above technical solution, the image registration module includes: Shallow feature extraction units are used to extract low-resolution hyperspectral images through bilinear interpolation. Upsampling to high-resolution panchromatic images The same spatial dimensions yielded an upsampled hyperspectral image. High-resolution panchromatic images are processed through convolutional layers respectively. and upsampled hyperspectral images Projecting onto the same feature channel dimension yields the panchromatic image features. and hyperspectral features ; The preliminary fusion unit is used to calculate panchromatic image features through an adaptive gating mechanism. and hyperspectral features Fusion weight graph , fusion weight graph Preliminary fusion characteristics were obtained : ; in, This indicates element-wise multiplication; Spatial location coding unit, used in the initial feature fusion Based on this, spatial location coding is injected to obtain position-enhanced fusion features after injecting spatial perception information. .

[0027] To further implement the above technical solution, the specific content of the fusion weight map obtained by the preliminary fusion unit is as follows: Features of panchromatic images and hyperspectral features After concatenation along the channel dimensions, an adaptive gating mechanism is used to calculate the fusion weight map. : ; in, This represents the Sigmoid activation function. This indicates a splicing operation at the channel level. This is a gated convolution operation. , H is the height, W is the width, and D is the number of channels. It is the set of real numbers.

[0028] To further implement the above technical solution, the specific content of the spatial location coding unit includes: To enable the Mamba network based on one-dimensional sequence modeling to possess absolute two-dimensional spatial location awareness, a normalized two-dimensional coordinate grid in the range [-1, 1] is generated. , and These represent the horizontal and vertical coordinates of the grid, respectively, and are concatenated into a coordinate matrix. Define learnable position scaling parameters ,calculate Spatial location features and will and By summing the residuals, we obtain the position-enhanced fusion feature after injecting spatial awareness information. : ; ; in, This indicates a coordinate-encoded convolutional layer.

[0029] To further implement the above technical solutions, the image registration module also includes a spatial dimension flattening unit, a dual-branch decoupled Manba network, and a reshaping and aggregation unit; Spatial dimension flattening unit, used to flatten Flattened into a one-dimensional sequence in spatial dimension , where N = H×W; in order to simultaneously model global rigid deformation and local elastic deformation, it is input into a two-branch decoupled Mamba network; The two-branch decoupled Mamba network includes rigid deformation branches and non-rigid deformation branches, wherein the rigid deformation branch is determined according to... Extracting global rigid deformation features Non-rigid deformation branch extracts local elastic deformation features ; The rigid deformation branch focuses on global long-range dependencies, employing a smaller local convolutional receptive field and a larger state dimension. ; The non-rigid deformation branch focuses on local feature extraction, enhancing local correlations and increasing the expansion rate through one-dimensional convolution: ; in, and These are Mamba sequence processing networks with two independent parameters; The reshaping and aggregation units utilize global average pooling and multilayer perceptron to generate rigid deformation branches, respectively. Dynamic weights of non-rigid deformation branches , Global rigid deformation characteristics and local elastic deformation characteristics After reshaping back to two-dimensional spatial features, aggregation is performed: ; in, This represents the feature matrix in a two-dimensional space, H×W×D, that reshapes a one-dimensional sequence back into its H×W×D form. Affine sampling network building blocks are used to... After global average pooling and regression multilayer perceptron, the affine transformation parameters are obtained. , respectively, represent two-dimensional affine matrices for global geometric transformations such as translation, rotation, scaling, and shearing, through Construct an affine sampling mesh; Registration unit for upsampling hyperspectral images based on affine sampling grid Perform geometric resampling to complete image registration and obtain the registered hyperspectral image. .

[0030] To further implement the above technical solution, the dynamic fusion module includes a deep feature extraction unit, the specific contents of which are as follows: High-resolution panchromatic image Registered hyperspectral image Sequence features are extracted by feeding the data into a Mamba network, and deep spatial features with a global receptive field are output. With spectral characteristics : ; ; in, and For the initial mapping layer, and It is a deep sequence feature extraction network.

[0031] To further implement the above technical solutions, traditional image fusion typically aims only to minimize pixel-level reconstruction errors. This easily leads to severe "spectral smoothing" and "semantic aliasing" at high-frequency ground object boundaries, thus losing the spectral separability most important for classification tasks. This invention receives pixel-level classification probability maps fed back by the classification module. (K represents the number of land cover categories) serves as a high-level semantic prior, dynamically guiding the shallow image fusion process. The dynamic fusion module also includes: The spatial smoothing and normalization unit is used to smooth and normalize the pixel-level classification probability map to avoid negative interference from classification prediction noise in the early stages of training. Perform spatial smoothing and normalization to obtain a smoothed semantic probability map. : ; in, This is a smoothing convolution operation used to suppress local high-frequency noise; Spatial attention modulation matrix generation unit, used to generate a smooth semantic probability map Generate a semantically driven spatial attention modulation matrix : ; in, This matrix is ​​used to indicate the "semantic saliency" and "boundary uncertainty" of the current pixel in the classification task; Feature weighting unit, used to... and The data is concatenated using a semantic modulation matrix. Perform category-specific feature weighting, and the weighted features : ; in, For element-wise multiplication of tensors For channel fusion convolution, It serves as a residual semantic attention gating mechanism; The image reconstruction unit is used to re-fuse the weighted features through a Mamba network and residual reconstruction to output a high spatial resolution hyperspectral image. : ; in, This indicates a convolution operation.

[0032] It should be noted that: In end-to-end joint optimization, the gradient generated by the classification loss will be passed through... It is directly sent back to the dynamic fusion module. When a pixel is located at a complex boundary of features (i.e., ... When the prediction entropy is high and the classification uncertainty is large, It adaptively enhances the spectral fidelity weights of the region and reduces overly smoothed texture injection. This mechanism, which "guides fusion parameters with classification semantics," breaks down the barriers of information isolation between traditional cascaded tasks, fundamentally ensuring the quality of the generated data. It can best serve the final classification task.

[0033] To further implement the above technical solution, the classification module includes a classification feature extraction network, the specific contents of which are as follows: Channel compression layer, used to compress high-resolution hyperspectral images After inputting into the classification feature extraction network, initial dimensionality reduction is performed, resulting in... : ; in, Indicates a channel compression layer; An enhanced Mamba classification network for deep mining of joint features of spatial texture and high-dimensional spectrum. : ; in, A Mamba backbone network optimized for classification tasks. , This indicates the number of channels in the output feature map.

[0034] To further implement the above technical solution, the classification module includes: Mapping convolutional layers are used to determine the appropriate convolutional layer based on the given conditions. Generate the original classification logits. ; in, This represents a convolutional layer that maps the number of channels to the number of classes K; The classification head is used to generate a normalized final probability distribution map along the category dimension using the Softmax function. : ; in, , The k-th channel value represents the confidence level that the pixel belongs to the k-th type of land cover.

[0035] It should be noted that: This invention mentions that parameters are not shared between multiple Mamba networks, and each network is optimized independently.

[0036] In the initial state before training, the weights of the Mamba network in the classification module are randomly initialized. At this time, the output classification probabilities approach a disordered uniform distribution. In this initial state, the dynamic fusion module is equivalent to performing uniform, unbiased conventional feature fusion; as joint training progresses rapidly, the Mamba network in the classification module converges quickly and gradually acquires a highly deterministic semantic orientation, thereby effectively driving feature weighting.

[0037] Joint Training Process: Joint training employs an end-to-end backpropagation mechanism. In one forward propagation, data flows sequentially through the three modules of registration, fusion, and classification. Subsequently, the total loss function is calculated, which is the sum of the fusion loss and the classification loss. Through the backpropagation algorithm, the error gradient generated by the classification task is not only used to update the classification Mamba network but also further injected into the dynamic fusion module. This joint training mechanism breaks the isolation of traditional cascaded tasks, enabling the network parameters in the registration and fusion stages to be jointly optimized and updated in a direction that "improves the final classification accuracy."

[0038] A hyperspectral image registration-fusion-classification integrated method includes the following steps: S1. For high-resolution panchromatic images and low-resolution hyperspectral images Preliminary fusion characteristics Injecting spatial location encoding yields location-enhanced fusion features Two-branch decoupled Mamba network according to Output global rigid deformation characteristics and local elastic deformation characteristics After aggregation, the aggregation feature is obtained. ,based on and Image registration was completed, resulting in a registered hyperspectral image. ; S2. To and Deep feature extraction is performed using the Mamba network, and the pixel-level classification probability map corresponding to the classification results fed back by S3 is generated. As a high-level semantic prior, a semantically driven spatial attention modulation matrix is ​​generated. ,pass To each and The corresponding deep features are weighted according to category-specific features to obtain the weighted features. ,Will After further fusion with the Mamba network and residual reconstruction, a high spatial resolution hyperspectral image is output. ; S3. Based on high spatial resolution hyperspectral images Deep mining of joint features of spatial texture and high-dimensional spectrum using an enhanced Mamba classification network. This yields the final pixel-level classification probability map.

[0039] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A hyperspectral image registration-fusion-classification integrated system, characterized in that, include: Image registration module, used for high-resolution panchromatic images and low-resolution hyperspectral images Preliminary fusion characteristics Injecting spatial location encoding yields location-enhanced fusion features Two-branch decoupled Mamba network according to Output global rigid deformation characteristics and local elastic deformation characteristics After aggregation, the aggregation feature is obtained. ,based on and Image registration was completed, resulting in a registered hyperspectral image. ; The dynamic fusion module is used for... and Deep feature extraction is performed using a Mamba network, and pixel-level classification probability maps corresponding to the classification results returned by the classification module are generated. As a high-level semantic prior, a semantically driven spatial attention modulation matrix is ​​generated. ,pass To each and The corresponding deep features are weighted according to category-specific features to obtain the weighted features. ,Will After further fusion with the Mamba network and residual reconstruction, a high spatial resolution hyperspectral image is output. ; A classification module is used to classify high spatial resolution hyperspectral images. Deep mining of joint features of spatial texture and high-dimensional spectrum using an enhanced Mamba classification network. This yields the final pixel-level classification probability map.

2. The hyperspectral image registration-fusion-classification integrated system according to claim 1, characterized in that, The image registration module includes: Shallow feature extraction units are used to extract low-resolution hyperspectral images through bilinear interpolation. Upsampling to high-resolution panchromatic images The same spatial dimensions yielded an upsampled hyperspectral image. High-resolution panchromatic images are processed through convolutional layers respectively. and upsampled hyperspectral images Projecting onto the same feature channel dimension yields the panchromatic image features. and hyperspectral features ; The preliminary fusion unit is used to calculate panchromatic image features through an adaptive gating mechanism. and hyperspectral features Fusion weight graph , fusion weight graph Preliminary fusion characteristics were obtained : ； in, This indicates element-wise multiplication; Spatial location coding unit, used in the initial feature fusion Based on this, spatial location coding is injected to obtain position-enhanced fusion features after injecting spatial perception information. .

3. The hyperspectral image registration-fusion-classification integrated system according to claim 2, characterized in that, The specific content of the fusion weight map obtained by the initial fusion unit is as follows: Features of panchromatic images and hyperspectral features After concatenation along the channel dimensions, an adaptive gating mechanism is used to calculate the fusion weight map. : ； in, This represents the Sigmoid activation function. This indicates a splicing operation at the channel level. This is a gated convolution operation. , H is the height, W is the width, and D is the number of channels. It is the set of real numbers.

4. The hyperspectral image registration-fusion-classification integrated system according to claim 1, characterized in that, The specific contents of the spatial location coding unit include: Generate a normalized two-dimensional coordinate grid in the range [-1, 1]. , and These represent the horizontal and vertical coordinates of the grid, respectively, and are concatenated into a coordinate matrix. Define learnable position scaling parameters ,calculate Spatial location features and will and By summing the residuals, we obtain the position-enhanced fusion feature after injecting spatial awareness information. : ； ； in, This indicates a coordinate-encoded convolutional layer.

5. The hyperspectral image registration-fusion-classification integrated system according to claim 2, characterized in that, The image registration module also includes a spatial dimension flattening unit, a two-branch decoupled Manba network, and a reshaping and aggregation unit; Spatial dimension flattening unit, used to flatten Flattened into a one-dimensional sequence in spatial dimension Where N = H×W; The two-branch decoupled Mamba network includes rigid deformation branches and non-rigid deformation branches, wherein the rigid deformation branch is determined according to... Extracting global rigid deformation features Non-rigid deformation branch extracts local elastic deformation features ; The reshaping and aggregation unit utilizes global average pooling and a multilayer perceptron to generate dynamic weights for rigid deformation branches, respectively. Dynamic weights of non-rigid deformation branches , Global rigid deformation characteristics and local elastic deformation characteristics After reshaping back to two-dimensional spatial features, aggregation is performed: ； in, This represents the feature matrix in a two-dimensional space, H×W×D, that reshapes a one-dimensional sequence back into its H×W×D form. Affine sampling network building blocks are used to... After global average pooling and regression multilayer perceptron, the affine transformation parameters are obtained. , A two-dimensional affine matrix containing global geometric transformation information, including translation, rotation, scaling, and shearing, is obtained through... Construct an affine sampling mesh; Registration unit for upsampling hyperspectral images based on affine sampling grid Perform geometric resampling to complete image registration and obtain the registered hyperspectral image. .

6. The hyperspectral image registration-fusion-classification integrated system according to claim 1, characterized in that, The dynamic fusion module includes a deep feature extraction unit, the specific contents of which include: High-resolution panchromatic image Registered hyperspectral image Sequence features are extracted by feeding the data into a Mamba network, and deep spatial features with a global receptive field are output. With spectral characteristics : ； ； in, and For the initial mapping layer, and It is a deep sequence feature extraction network.

7. The hyperspectral image registration-fusion-classification integrated system according to claim 1, characterized in that, The dynamic fusion module also includes: Spatial smoothing and normalization processing units are used to process pixel-level classification probability maps. Perform spatial smoothing and normalization to obtain a smoothed semantic probability map. : ； in, This is a smoothing convolution operation used to suppress local high-frequency noise; Spatial attention modulation matrix generation unit, used to generate a smooth semantic probability map Generate a semantically driven spatial attention modulation matrix : ； in, ; Feature weighting unit, used to... and The data is concatenated using a semantic modulation matrix. Perform category-specific feature weighting, and the weighted features : ； in, For element-wise multiplication of tensors For channel fusion convolution, It serves as a residual semantic attention gating mechanism; The image reconstruction unit is used to re-fuse the weighted features through a Mamba network and residual reconstruction to output a high spatial resolution hyperspectral image. : ； in, This indicates a convolution operation.

8. The hyperspectral image registration-fusion-classification integrated system according to claim 1, characterized in that, The classification module includes a classification feature extraction network, the specific contents of which are as follows: Channel compression layer, used to compress high-resolution hyperspectral images After inputting into the classification feature extraction network, initial dimensionality reduction is performed, resulting in... : ； in, Indicates a channel compression layer; An enhanced Mamba classification network for deep mining of joint features of spatial texture and high-dimensional spectrum. : ； in, A Mamba backbone network optimized for classification tasks. , This indicates the number of channels in the output feature map.

9. The hyperspectral image registration-fusion-classification integrated system according to claim 1, characterized in that, The classification module includes: Mapping convolutional layers are used to determine the appropriate convolutional layer based on the given conditions. Generate the original classification logits. ； in, This represents a convolutional layer that maps the number of channels to the number of classes K; The classification head is used to generate a normalized final probability distribution map along the category dimension using the Softmax function. : ； in, , The k-th channel value represents the confidence level that the pixel belongs to the k-th type of land cover.

10. A hyperspectral image registration-fusion-classification integrated method, based on the hyperspectral image registration-fusion-classification integrated system according to any one of claims 1-9, characterized in that, Includes the following steps: S1. For high-resolution panchromatic images and low-resolution hyperspectral images Preliminary fusion characteristics Injecting spatial location encoding yields location-enhanced fusion features Two-branch decoupled Mamba network according to Output global rigid deformation characteristics and local elastic deformation characteristics After aggregation, the aggregation feature is obtained. ,based on and Image registration was completed, resulting in a registered hyperspectral image. ; S2. To and Deep feature extraction is performed using the Mamba network, and the pixel-level classification probability map corresponding to the classification results fed back by S3 is generated. As a high-level semantic prior, a semantically driven spatial attention modulation matrix is ​​generated. ,pass To each and The corresponding deep features are weighted according to category-specific features to obtain the weighted features. ,Will After further fusion with the Mamba network and residual reconstruction, a high spatial resolution hyperspectral image is output. ; S3. Based on high spatial resolution hyperspectral images Deep mining of joint features of spatial texture and high-dimensional spectrum using an enhanced Mamba classification network. This yields the final pixel-level classification probability map.

Images