Hyperspectral image reconstruction method and system based on multi-domain neural network modeling
By using multi-domain neural network modeling, combined with multi-scale spatial fusion and U-shaped spectral enhancement modules, the accuracy and efficiency problems of existing hyperspectral image reconstruction methods are solved, realizing high-precision reconstruction and flexible application of hyperspectral images.
Patent Information
- Application Number
- CN202511543850.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-28
- Publication Date
- 2026-01-30
AI Technical Summary
Existing hyperspectral image reconstruction methods have shortcomings in reconstruction accuracy, computational efficiency, and utilization of hyperspectral data, and have not fully explored the correlation between spatial-spectral prior information.
A multi-domain neural network model is adopted, and a multi-scale spatial fusion module and a U-shaped spectral enhancement module are combined with a spectral Fourier modulation module to synergistically utilize spatial, spectral and frequency domain information for hyperspectral image reconstruction.
It improves the reconstruction accuracy and spectral fidelity of hyperspectral images, has good flexibility and scalability, is suitable for different hardware environments, and supports the acquisition of hyperspectral image data in fields such as environmental monitoring, precision agriculture, and geological exploration.
Smart Images
Figure CN121437656A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of remote sensing image processing and hyperspectral reconstruction, in particular to a hyperspectral image reconstruction method and system based on multi-domain neural network modeling. BACKGROUND
[0002] In the field of hyperspectral image reconstruction, existing methods are mainly divided into two categories: prior knowledge-based methods and data-driven methods. Prior knowledge-based methods utilize physical or mathematical models and rely on the understanding of the spectral characteristics of ground objects and the imaging process. However, these methods are often limited in terms of reconstruction accuracy and sensitive to information loss due to their reliance on prior knowledge and lack of adaptability to complex scenes. Data-driven methods train models with large amounts of data to learn the reconstruction mapping relationship. Although these methods improve the reconstruction results to some extent, existing models generally have problems such as excessive number of parameters, insufficient computational efficiency and scalability, and inadequate utilization of spatial and temporal priors inherent in hyperspectral data. Furthermore, frequency domain information has not been systematically explored, resulting in deficiencies in high-frequency detail preservation and spatial-spectral consistency in the reconstruction results. In addition, attention mechanism models, although improving the reliance on modeling, often come with higher computational complexity and memory usage, and fail to fully exploit the relevance of spatial-spectral prior information inherent in hyperspectral data. These problems have prompted the need for more efficient, accurate, and fully utilizing multi-domain information hyperspectral image reconstruction methods and systems.
[0003] To this end, the present application provides a hyperspectral image reconstruction method and system based on multi-domain neural network modeling to solve the above problems. SUMMARY
[0004] In view of the deficiencies of the prior art, the present application provides a hyperspectral image reconstruction method and system based on multi-domain neural network modeling to solve the problems raised in the background art.
[0005] To achieve the above purpose, the present application is implemented by the following technical solution: a hyperspectral image reconstruction method based on multi-domain neural network modeling, comprising the following steps:
[0006] Convolve the input multispectral image to increase its spectral dimension to the number of bands of the target hyperspectral image, obtaining initial hyperspectral features;
[0007] Use a multi-scale spatial fusion module to process the initial hyperspectral features, extract and fuse global spatial context information and local multi-scale spatial features, obtaining spatial fusion features;
[0008] Use a U-shaped spectral enhancement modeling module to process the spatial fusion features, extract, enhance and fuse spectral information at different scales, obtaining reconstructed spectral features;
[0009] The reconstructed hyperspectral image is obtained by convolving and restoring the reconstructed spectral features through a reconstruction image output module.
[0010] Preferably, the step of processing the initial hyperspectral features by using the multi-scale spatial fusion module comprises:
[0011] The global spatial context information of the initial hyperspectral features is extracted through a Mamba module;
[0012] The local multi-scale spatial features of the initial hyperspectral features are extracted through at least two parallel convolution branches with different scale convolution kernels;
[0013] The global spatial context information is fused with each local multi-scale spatial feature, and the fused multiple features are finally integrated.
[0014] Preferably, the U-shaped spectral enhancement modeling module comprises a plurality of stacked spectral enhancement modules, each of which is used to process input features to enhance spectral information.
[0015] Preferably, the spectral enhancement module comprises:
[0016] At least one local modeling branch for modeling local spectral information of features;
[0017] At least one global modeling branch for modeling global spectral information of features;
[0018] A fusion unit for fusing outputs of the local modeling branch and the global modeling branch.
[0019] Preferably, the local modeling branch comprises a multi-head spectral self-attention mechanism for capturing local inter-band correlation, and the global modeling branch comprises a SpectralMamba module for modeling long-range dependency between different spectral groups.
[0020] Preferably, after the step of processing the spatial fusion features by using the U-shaped spectral enhancement modeling module, the step of modulating the reconstructed spectral features in the frequency domain by using a spectral Fourier modulation module is further included to improve the spectral reconstruction accuracy.
[0021] Preferably, the step of modulating the reconstructed spectral features in the frequency domain by using the spectral Fourier modulation module comprises:
[0022] One-dimensional Fourier transform is performed on the input spectral features along the spectral dimension to obtain their amplitude spectrum and phase spectrum;
[0023] The amplitude adjustment parameter and the phase offset parameter are respectively generated by a learnable network module, and the amplitude spectrum and the phase spectrum are modulated;
[0024] The modulated frequency domain representation is subjected to one-dimensional inverse Fourier transform to obtain a frequency-corrected spectral feature.
[0025] A hyperspectral image reconstruction system based on multi-domain neural network modeling comprises:
[0026] An initial spectral dimension increasing module is configured to perform convolution processing on an inputted multispectral image to increase the spectral dimension of the multispectral image to the number of bands of a target hyperspectral image, thereby obtaining an initial hyperspectral feature.
[0027] A multi-scale spatial fusion module is configured to process the initial hyperspectral feature, extract and fuse global spatial context information and local multi-scale spatial features, and obtain a spatial fusion feature.
[0028] A U-shaped spectral enhancement modeling module is configured to process the spatial fusion feature, extract, enhance and fuse spectral information at different scales, and obtain a reconstructed spectral feature.
[0029] A reconstructed image output module is configured to perform convolution recovery on the reconstructed spectral feature, and obtain a reconstructed hyperspectral image.
[0030] Preferably, the multi-scale spatial fusion module comprises:
[0031] A global spatial context extraction unit is configured to extract global spatial context information of the initial hyperspectral feature by a Mamba module.
[0032] A local multi-scale feature extraction unit is configured to extract local multi-scale spatial features of the initial hyperspectral feature by at least two parallel convolution branches with different scale convolution kernels.
[0033] A feature fusion unit is configured to fuse the global spatial context information with each local multi-scale spatial feature, and finally integrate the fused features.
[0034] Preferably, the U-shaped spectral enhancement modeling module comprises a plurality of stacked spectral enhancement modules, each of which comprises at least one local modeling branch, at least one global modeling branch and a fusion unit, the local modeling branch is configured to model local spectral information of the feature, the global modeling branch is configured to model global spectral information of the feature, and the fusion unit is configured to fuse the outputs of the local modeling branch and the global modeling branch.
[0035] The present application provides a hyperspectral image reconstruction method and system based on multi-domain neural network modeling.
[0036] 1.The present application can effectively improve the accuracy and spectral fidelity of hyperspectral image reconstruction from multispectral image by synergistically utilizing multi-domain information such as space, spectrum and frequency. The initial spectral dimensionality increasing module lays the foundation for subsequent processing, the multi-scale spatial fusion module can comprehensively extract and fuse spatial information, the U-shaped spectral enhancement modeling module can model spectral information in different scales in detail, and the reconstructed image output module realizes accurate hyperspectral image reconstruction. At the same time, the spectral Fourier modulation module further optimizes the spectral characteristics in the frequency domain, so that the reconstructed hyperspectral image is closer to the real situation in terms of spectral accuracy.
[0037] 2.The present application adopts a deep learning framework, has good flexibility and scalability, and is convenient for adjustment and optimization according to actual application requirements. Various processing units in the multi-scale spatial fusion module and the U-shaped spectral enhancement modeling module can be conveniently modularized and integrated, which is conducive to efficient operation under different hardware environments and computing resource conditions. In addition, the system can provide strong support for related field research and application, such as environmental monitoring, precision agriculture, geological exploration, etc., helping users to more accurately obtain hyperspectral image data, so as to provide more reliable basis for decision making. BRIEF DESCRIPTION OF DRAWINGS
[0038] Figure 1 The hyperspectral image reconstruction algorithm provided by the present application is shown in the figure;
[0039] Figure 2 a is a schematic diagram of different information aggregation methods in the present application, Figure 2 b is a schematic diagram of the MMSAF core algorithm proposed by the present application;
[0040] Figure 3 The comparison table of the calculation cost of the present application and other related models is shown in the figure;
[0041] Figure 4 a is a comparison table of reconstruction error of other related models of the present application under the IndianPines data set, Figure 4 b is a comparison table of reconstruction error of the present application and other related models under the PaviaUniversity data set;
[0042] Figure 5 The comparison figure of single-point spectrum of the present application and other related models for reconstructing hyperspectral image is shown in the figure;
[0043] Figure 6 The comparison figure of classification results of the present application and other related models for reconstructing hyperspectral image is shown in the figure. DETAILED DESCRIPTION
[0044] With reference to the drawings of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.
[0045] Please refer to the drawings of the present application Figure 1 - the drawings of the present application Figure 6 The embodiment of the present application provides a hyperspectral image reconstruction method based on multi-domain neural network modeling, comprising the following steps:
[0046] The input multispectral image is subjected to convolution processing, the spectral dimension is raised to the number of bands of the target hyperspectral image, and an initial hyperspectral feature is obtained;
[0047] The initial hyperspectral feature is processed by using a multi-scale spatial fusion module, global spatial context information and local multi-scale spatial features are extracted and fused, and a spatial fusion feature is obtained;
[0048] The step of processing the initial hyperspectral feature by using the multi-scale spatial fusion module comprises:
[0049] The global spatial context information of the initial hyperspectral feature is extracted by the Mamba module;
[0050] The local multi-scale spatial features of the initial hyperspectral feature are extracted by at least two parallel convolution branches with different scale convolution kernels;
[0051] The global spatial context information is fused with each local multi-scale spatial feature, and the fused multiple features are finally integrated.
[0052] Specifically, according to the processing procedure of space first and spectrum second, the first part MMSAF is responsible for extracting multi-scale spatial information and performing multi-level channel fusion. The second part SEM adopts a U-shaped architecture to combine intra-group and inter-group spectral modeling to extract multi-scale spectral information, and embeds a frequency domain information modulation module to avoid the problem of falling into a local optimal solution when only modeling in the spectral domain. The overall flow of the model is as follows:
[0053] F HSI =SEM(MMSAF(F MSI ))
[0054] By connecting MMSAF and SEM, the network can fully learn spatial, spectral and frequency domain information, realize multi-domain joint modeling, and alleviate the problem of poor curve fitting effect and pseudo curve caused by poor reconstruction caused by single extraction mode. Finally, the model restores the output in space and band number by a 3x3 convolution, so that the model can output any band number.
[0055] (1)Lightweight multi-scale information fusion guided by global spatial information
[0056] To improve the multi-scale modeling capability of spatial features, we designed a multi-scale spatial fusion module (MMSAF) based on Mamba. The module first uses Mamba to extract global spatial context information, and then extracts local multi-scale spatial features through three different scale deep convolutions (3x3, 5x5, 7x7). The features of each scale branch are concatenated with the Mamba output and then fused through point-wise convolution. Finally, all features are concatenated again and integrated using a 3x3 convolution to output.
[0057] F0 = PWConv 1×1 (Mamba(X In ))
[0058] F k = PWConv 1×1 (Concat(DWConv k×k (X in ),F0))
[0059] F out = Conv 3×3 (Concat(F0,F3,F5,F7))
[0060] The spatial fusion features are processed using a U-shaped spectral enhancement modeling module to extract, enhance, and fuse spectral information at different scales to obtain reconstructed spectral features. The U-shaped spectral enhancement modeling module includes multiple stacked spectral enhancement modules, each of which is used to process input features to enhance spectral information.
[0061] Specifically, to achieve efficient hyperspectral reconstruction modeling, a U-shaped spectral enhancement modeling network U-SEM is introduced, which is based on a dual-branch SpectralEnhancementModule (SEM) to jointly model the intra-group and inter-group spectral dependence relationships and fuse and regulate them through a gating mechanism. The overall framework adopts a one-layer U-Net structure, including three modules of symmetric encoder, decoder, and bottleneck, to extract and fuse spectral information at different scales. Since there are more long-range linear dependence relationships between bands in most remote sensing images, and local bands are more highly correlated, local and long-range band dependence relationships need to be modeled separately, and a gating mechanism is used to adaptively select long-range and short-range dependence.
[0062] SEM aims to jointly capture fine-grained relationships in the spectral dimension and cross-group modeling capabilities. To this end, the input features Firstly, the data is determined to be divided into N groups (i.e., tokens), and the feature dimension C' = N · d after the Pad dimension upgrade is used to meet the modeling requirements of N groups, each group contains channels, forming the completed representation:
[0063]
[0064] Then enter two branches:
[0065] 1. Local modeling branch (MS-MSA)
[0066] Due to the high linear correlation between the local bands of hyperspectral data, we use the multi-head spectral self-attention mechanism as the method of local spectral modeling. The local modeling adopts the multi-head spectral self-attention mechanism (MS-MSA), which applies attention within each head, that is, within each spectral group, to model the local spectral interdependence. When the input spectrum is , X In is first deformed to , and X is respectively multiplied by These learnable parameters, linearly projected to the query key value
[0067] Q = XW Q , K = XW K , V = XW V
[0068] Q, K, V are divided into N heads along the spectral dimension, becoming Q = [Q1,..., Q N ], K = [K1,..., K N ], V = [V1,..., V N ], and the dimension of each head is Each head is sent to the self-attention mechanism to calculate the attention weight, and the process of splicing each new head obtained after weighting is as follows:
[0069]
[0070] Here we use to align the inter-spectral sparsity, and after splicing, a linear layer is used to add PE(V) for position encoding.
[0071] 2. Global modeling branch (SpectralMamba)
[0072] In addition to the high linear correlation of local spectrum, there is still linear correlation between certain remote spectral bands. In addition, the non-linear correlation between spectra needs to be further explored. Therefore, the SpectralMamba module is designed to use the State Space Model (SSM) to model the relationship between groups continuously. Similarly, the input is divided into N groups to model the relationship between different groups by the standard mamba module, and the corresponding relationship is used to guide the spectral feature update. The whole group spectral feature update process is as follows:
[0073] H G = SplitSpectralGroup (X In )
[0074] H F = Flatten (H G )
[0075] H M = Mamba (H F )
[0076] H Output = Reshape (H M ) + X In
[0077] wherein respectively refer to the split spectral group feature, the stretched feature, the Mamba modeling feature, and the residual output feature. N refers to the number of spectral groups, M is the number of wavebands of each spectral group, and Mamba refers to the standard Mamba block.
[0078] The spectral enhancement module includes:
[0079] at least one local modeling branch for modeling the local spectral information of the feature;
[0080] The local modeling branch includes a multi-head spectral self-attention mechanism for capturing the correlation between local wavebands; the global modeling branch includes the SpectralMamba module for modeling the long-range dependency relationship between different spectral groups.
[0081] at least one global modeling branch for modeling the global spectral information of the feature;
[0082] Specifically, the dual-branch output is and a two-layer fully connected gating module is used to predict the fusion weight:
[0083] W = σ (FC (Concat (X S-MSA , X Mamba )))
[0084] X fsd =W⊙X S-MSA +(1-W)⊙X Mamba
[0085] X result =FFN(X fsd )+X fsd
[0086] Where FC represents a gated network consisting of two linear layers, and σ refers to the Sigmoid activation function. The adaptive choice here depends more on local or global modeling, assigning different preferences to different regions.
[0087] To improve the spectral accuracy of reconstructed hyperspectral images, this paper proposes a Spectral Fourier Modulation (SFM) module for fine-tuning the output spectral features in the frequency domain. This module can adjust the amplitude and phase information in the spectrum separately, thereby effectively correcting frequency distortion in the network reconstruction process and enhancing the continuity and smoothness of the spectrum.
[0088] Let the reconstructed hyperspectral image output by the network be... A one-dimensional Fourier transform (FFT) is performed on it along the spectral dimension C to obtain its complex frequency domain representation:
[0089] F = Fc(X) = A·e jΦ
[0090] in, For spectral amplitude, This is the phase spectrum.
[0091] Then, a lightweight multilayer linear sensing machine (MLP) is used to generate amplitude adjustment parameters and phase shift parameters respectively:
[0092]
[0093] Each MLP contains several linear layers and ReLU activation functions, with its output dimension consistent with the number of spectral bands C. To ensure stable training in the initial stage of the model, we implemented an initialization strategy for the tuning parameters: when adjusting the amplitude vector S... A Initialize to 1 to preserve the original amplitude structure: phase adjustment vector S Φ Initialize to 0 to avoid initial phase disturbances: After broadcasting and adjusting these parameters across the spectrum, we have:
[0094] A′=A·S A
[0095] Φ′=Φ+S Φ
[0096] F' = A' e jΦ′ = A'(cosΦ' + jsinΦ')
[0097] Finally, the original spatial-spectral domain is recovered by inverse Fourier transform:
[0098] X' = F c -1 (F')
[0099] The frequency domain modulation strategy can be regarded as a learnable frequency correction mechanism, which can finely correct spectral distortion. Since the adjustment parameter is a global vector, the overall structural parameter amount is extremely low, almost no additional calculation is added, and the spectral reconstruction accuracy can be effectively improved.
[0100] The fusion unit is configured to fuse the outputs of the local modeling branch and the global modeling branch.
[0101] After the step of processing the spatial fusion feature by the U-shaped spectral enhancement modeling module, the method further includes a step of modulating the reconstructed spectral feature in the frequency domain by the spectral Fourier modulation module to improve the spectral reconstruction accuracy.
[0102] The step of modulating the reconstructed spectral feature in the frequency domain by the spectral Fourier modulation module includes:
[0103] performing one-dimensional Fourier transform on the input spectral feature along the spectral dimension to obtain an amplitude spectrum and a phase spectrum thereof;
[0104] generate an amplitude adjustment parameter and a phase offset parameter by the learnable network module, and modulate the amplitude spectrum and the phase spectrum;
[0105] performing one-dimensional inverse Fourier transform on the modulated frequency domain representation to obtain the spectral feature after frequency correction.
[0106] convolve the reconstructed spectral feature by the reconstructed image output module to obtain the reconstructed hyperspectral image.
[0107] A hyperspectral image reconstruction system based on multi-domain neural network modeling includes:
[0108] An initial spectral dimension increasing module is configured to perform convolution processing on the input multispectral image to increase the spectral dimension of the multispectral image to the number of bands of the target hyperspectral image, and obtain an initial hyperspectral feature;
[0109] A multi-scale spatial fusion module is configured to process the initial hyperspectral feature, extract and fuse global spatial context information and local multi-scale spatial features, and obtain a spatial fusion feature;
[0110] The multi-scale spatial fusion module includes:
[0111] a global spatial context extraction unit configured to extract global spatial context information of the initial hyperspectral features by a Mamba module;
[0112] Specifically, the multispectral image is convolved by the initial spectral dimension increasing module to increase its spectral dimension to the number of bands of the target hyperspectral image, thereby constructing a more matched feature basis for subsequent processing. On this basis, the multiscale spatial fusion module captures the global spatial information of the initial hyperspectral features by the Mamba module through the global spatial context extraction unit, and fuses the local multiscale spatial features, realizes comprehensive extraction and integration of spatial information, and obtains more rich and expressive spatial fusion features, which provides strong support for accurate reconstruction of the hyperspectral image.
[0113] a local multiscale feature extraction unit configured to extract local multiscale spatial features of the initial hyperspectral features by at least two parallel convolution branches with different scale convolution kernels;
[0114] a feature fusion unit configured to fuse the global spatial context information with each local multiscale spatial feature, and to finally integrate the fused multiple features.
[0115] Specifically, the local multiscale feature extraction unit processes the initial hyperspectral features by using parallel convolution branches with multiple different scale convolution kernels. This design can capture local spatial features at different scales, because convolution kernels of different sizes can perceive local information in different ranges, thereby retaining more rich details and textures in the feature map. The feature fusion unit then combines the global spatial context information with these local multiscale features, realizing information complementation and enhancement. This fusion mechanism helps the network to understand the overall structure of the image while retaining local details, thereby improving the richness and accuracy of feature expression and ultimately improving the reconstruction quality of the hyperspectral image.
[0116] a U-shaped spectral enhancement modeling module configured to process the spatial fusion features, extract, enhance and fuse spectral information at different scales to obtain reconstructed spectral features; the U-shaped spectral enhancement modeling module includes a plurality of stacked spectral enhancement modules, each spectral enhancement module includes at least one local modeling branch, at least one global modeling branch and a fusion unit, the local modeling branch is configured to model local spectral information of the features, the global modeling branch is configured to model global spectral information of the features, and the fusion unit is configured to fuse outputs of the local modeling branch and the global modeling branch.
[0117] Specifically, the U-shaped spectral enhancement modeling module plays a key role in the hyperspectral image reconstruction system, mainly used for in-depth processing of spatial fusion features to extract, enhance and fuse spectral information. This module draws on the idea of U-Net architecture, through multiple stacked spectral enhancement modules, which can effectively capture spectral information at different scales. Each spectral enhancement module contains at least one local modeling branch and one global modeling branch, as well as a fusion unit. The local modeling branch, for example, adopts a multi-head spectral self-attention mechanism, focuses on fine modeling of the local spectral information of the features, and can capture the high correlation between local bands in the spectral data, thus retaining more detailed information. The global modeling branch, such as through the SpectralMamba module, is responsible for modeling the global spectral information of the features, and can effectively model the long-range dependency between different spectral groups, thus ensuring the consistency and coherence of the overall spectral features. The fusion unit fuses the outputs of the local modeling branch and the global modeling branch, dynamically balances the contributions of local and global spectral information by adaptively learning the fusion weights, so that the final reconstructed spectral features contain not only rich local details but also global consistency. This multi-scale spectral information extraction and fusion method significantly improves the accuracy and quality of hyperspectral image reconstruction, making the reconstruction results effectively improved in terms of spectral fidelity and spatial-spectral consistency.
[0118] The reconstructed image output module is used to perform convolution recovery on the reconstructed spectral features to obtain the reconstructed hyperspectral image.
[0119] Specifically, the reconstructed image output module plays a crucial role in the present hyperspectral image reconstruction system. Its main function is to perform convolution recovery operations on the reconstructed spectral features processed by the U-shaped spectral enhancement modeling module, thereby outputting the final hyperspectral image. This module performs a series of carefully designed convolution operations on the multi-channel feature maps, performing linear and nonlinear transformations, mapping the high-dimensional feature representation back to the band dimension corresponding to the original hyperspectral image, while retaining and highlighting the key information extracted in the previous modules and removing possible redundant or noisy features. This process not only realizes the conversion from feature space to original data space, but also further improves the spatial resolution and spectral fidelity of the reconstructed image through the integration and optimization of features, ensuring that the output hyperspectral image is as close as possible to the real scene in terms of details, structure and spectral characteristics, providing high-quality reconstruction results for subsequent application analysis.
[0120] Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the application, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for hyperspectral image reconstruction based on multi-domain neural network modeling, characterized in that, The method comprises the following steps: convolution processing is performed on the input multispectral image to increase the spectral dimension to the number of bands of the target hyperspectral image, to obtain initial hyperspectral features; the initial hyperspectral features are processed by a multiscale spatial fusion module to extract and fuse global spatial context information and local multiscale spatial features, to obtain spatial fusion features; a U-shaped spectral enhancement modeling module is used to process the spatial fusion features to extract, enhance and fuse spectral information at different scales, to obtain reconstructed spectral features; the reconstructed spectral features are recovered by a reconstructed image output module to obtain a reconstructed hyperspectral image.
2. The hyperspectral image reconstruction method based on multi-domain neural network modeling according to claim 1, characterized in that, The step of processing the initial hyperspectral features by the multiscale spatial fusion module comprises: global spatial context information of the initial hyperspectral features is extracted by a Mamba module; local multiscale spatial features of the initial hyperspectral features are extracted by at least two parallel convolution branches with different scale convolution kernels; the global spatial context information is fused with each local multiscale spatial feature, and the fused multiple features are finally integrated. 3.The hyperspectral image reconstruction method based on multi-domain neural network modeling according to claim 1, characterized in that, The U-shaped spectral enhancement modeling module comprises a plurality of stacked spectral enhancement modules, each of which is used to process input features to enhance spectral information.
4. The hyperspectral image reconstruction method based on multi-domain neural network modeling according to claim 1, characterized in that, The spectral enhancement module comprises: at least one local modeling branch for modeling local spectral information of features; at least one global modeling branch for modeling global spectral information of features; a fusion unit for fusing outputs of the local modeling branch and the global modeling branch.
5. The hyperspectral image reconstruction method based on multi-domain neural network modeling according to claim 1, characterized in that, The local modeling branch comprises a multi-head spectral self-attention mechanism for capturing local inter-band correlation, and the global modeling branch comprises a SpectralMamba module for modeling long-range dependency between different spectral groups.
6. The hyperspectral image reconstruction method based on multi-domain neural network modeling according to claim 1, characterized in that, After the step of processing the spatial fusion features by the U-shaped spectral enhancement modeling module, a step of modulating the reconstructed spectral features in the frequency domain by a spectral Fourier modulation module is further included to improve spectral reconstruction accuracy.
7. The hyperspectral image reconstruction method based on multi-domain neural network modeling according to claim 1, characterized in that, The step of modulating the reconstructed spectral features in the frequency domain by the spectral Fourier modulation module comprises: one-dimensional Fourier transform is performed on the input spectral features along the spectral dimension to obtain amplitude spectrum and phase spectrum; a learnable network module is used to generate amplitude adjustment parameters and phase offset parameters respectively, and to modulate the amplitude spectrum and the phase spectrum; one-dimensional inverse Fourier transform is performed on the modulated frequency domain representation to obtain frequency-corrected spectral features.
8. A hyperspectral image reconstruction system based on multi-domain neural network modeling, according to the hyperspectral image reconstruction method based on multi-domain neural network modeling of any one of claims 1-7, characterized in that, The method comprises: an initial spectral dimension increasing module for performing convolution processing on the input multispectral image to increase the spectral dimension to the number of bands of the target hyperspectral image, to obtain initial hyperspectral features; a multiscale spatial fusion module for processing the initial hyperspectral features to extract and fuse global spatial context information and local multiscale spatial features, to obtain spatial fusion features; a U-shaped spectral enhancement modeling module for processing the spatial fusion features to extract, enhance and fuse spectral information at different scales, to obtain reconstructed spectral features; and a reconstructed image output module for recovering the reconstructed spectral features. The reconstructed image output module is configured to perform convolution recovery on the reconstructed spectral features to obtain a reconstructed hyperspectral image.
9. The hyperspectral image reconstruction system based on multi-domain neural network modeling according to claim 8, characterized in that, The multi-scale spatial fusion module comprises: a global spatial context extraction unit configured to extract global spatial context information of the initial hyperspectral features by a Mamba module; a local multi-scale feature extraction unit configured to extract local multi-scale spatial features of the initial hyperspectral features by at least two parallel convolution branches with different scale convolution kernels; a feature fusion unit configured to fuse the global spatial context information with each local multi-scale spatial feature and finally integrate the fused features.
10. The hyperspectral image reconstruction system based on multi-domain neural network modeling according to claim 8, wherein, The U-shaped spectral enhancement modeling module comprises a plurality of stacked spectral enhancement modules, each of which comprises at least one local modeling branch, at least one global modeling branch, and a fusion unit, the local modeling branch is configured to model local spectral information of the features, the global modeling branch is configured to model global spectral information of the features, and the fusion unit is configured to fuse outputs of the local modeling branch and the global modeling branch.