A hyperspectral fusion imaging method and system of a two-branch diffusion model framework

Abstract

This invention discloses a hyperspectral fusion imaging method and system based on a dual-branch diffusion model framework. The method includes: inputting a hyperspectral image and a corresponding high-resolution multispectral image from an observed scene; constructing a fusion image representing the target image for modeling using a representation model; optimizing and training a one-dimensional U-Net spectral diffusion model based on the hyperspectral image to learn and generate high-fidelity spectral components; combining the generated features of the pre-trained diffusion model derived from remote sensing images with the observation features extracted from the high-resolution multispectral image to obtain high-resolution spatial components; constructing a dual-branch diffusion framework, guiding the generation process of the two components through a shared guiding function, and iteratively optimizing the observation features to fuse and reconstruct a high-resolution hyperspectral image. This invention achieves high-quality image fusion through a dual-branch diffusion model generation mechanism, providing an innovative solution for hyperspectral image fusion imaging.

Description

Technical Field

[0001] This invention relates to hyperspectral image fusion imaging technology, and in particular to a hyperspectral fusion imaging method and system based on a dual-branch diffusion model framework. Background Technology

[0002] Hyperspectral imaging technology, by finely sampling the reflectance information of ground objects across continuous spectral bands, can acquire rich spectral feature information while providing spatial structure information of a scene, making it valuable for applications in agricultural monitoring, environmental remote sensing, fine mapping, and mineral exploration. However, due to limitations in the physical manufacturing processes and costs of existing imaging sensors, there is an inherent constraint between spatial resolution and spectral resolution, making it difficult to simultaneously achieve high spatial and spectral resolution. Typically, hyperspectral sensors have low spatial resolution but high spectral resolution, while multispectral sensors have high spatial resolution but low spectral resolution. Based on this complementarity, by fusing low-spatial-resolution hyperspectral images with high-spatial-resolution multispectral images, spatial resolution can be significantly improved while maintaining the integrity of spectral information, thus obtaining high-spatial-resolution hyperspectral images. This fusion imaging technology has become an effective way to overcome the physical limitations of sensors and improve the quality of remote sensing images.

[0003] Existing fusion imaging techniques can be broadly categorized into two types: model-driven methods and data-driven methods. Model-driven methods typically rely on mathematical models such as matrix factorization, sparse representation, low-rank constraints, or tensor decomposition to achieve the fusion of hyperspectral and multispectral images. While these methods offer good interpretability, they often struggle to fully characterize the complex coupling between high-dimensional space and spectral features when faced with complex ground cover distributions, nonlinear observation degradation, or noise interference. This can easily lead to spatial texture blurring or spectral distortion in the fusion imaging results. With the development of deep learning, data-driven methods based on convolutional neural networks and Transformers have been widely proposed. These methods learn end-to-end mappings from observed images to high-resolution hyperspectral images through large-scale sample learning, significantly improving the quality of fusion imaging in several scenarios. However, data-driven methods generally rely on large-scale paired training samples. In practical applications, obtaining high-resolution hyperspectral images that are precisely paired with low-spatial-resolution hyperspectral images as training labels is extremely difficult, severely limiting the practicality and generalization ability of these methods.

[0004] In recent years, with the rise of generative artificial intelligence, diffusion models, with their powerful data distribution modeling and image generation capabilities, have been gradually introduced into the field of hyperspectral and multispectral image fusion imaging. Methods based on diffusion models can gradually generate high-quality fused images by simulating the reverse diffusion process from noise to image. However, existing diffusion model methods still have several shortcomings in hyperspectral fusion imaging tasks. On the one hand, many methods directly use spatial diffusion models pre-trained on natural images as priors. However, natural images only contain a limited number of spectral channels, and the learned distribution mainly reflects the spatial texture features of natural images, making it difficult to fully adapt to the complex spatial-spectral structural features in hyperspectral data, thus causing deviations in practical fusion imaging tasks. On the other hand, existing methods typically use fully connected structures to construct spectral diffusion models. However, fully connected networks structurally treat spectral vectors as disordered, flat features, making it difficult to effectively capture the continuous absorption features between adjacent bands in hyperspectral images, which may lead to the loss of spectral details and distortion of absorption peaks in the fusion results. In summary, there is an urgent need to provide a fusion imaging framework that can effectively model spatial and spectral components and generate them collaboratively to improve the spatial fidelity and spectral consistency of fused images. Summary of the Invention

[0005] The purpose of this invention is to provide a hyperspectral fusion imaging method and system based on a dual-branch diffusion model framework.

[0006] The technical solution to achieve the purpose of this invention is: a hyperspectral fusion imaging method based on a dual-branch diffusion model framework, comprising the following steps:

[0007] Step 1: Input a low spatial resolution hyperspectral image and a high spatial resolution multispectral image of the corresponding observation area. Decompose the high resolution hyperspectral image to be reconstructed into a spectral component and a spatial component, wherein the spatial component consists of two parts: generated features and observation features.

[0008] Step 2: Construct a one-dimensional U-Net spectral diffusion model that can learn and generate continuous sequence features, and train it using the acquired low spatial resolution hyperspectral images to generate spectral components that can accurately describe the observation scene.

[0009] Step 3: Construct a spatial component generation branch that combines spatial information from two different sources: one is the generation features derived from a spatial diffusion model pre-trained on an external remote sensing image dataset, and the other is the observation features extracted from high spatial resolution multispectral images.

[0010] Step 4: The fusion result is generated using the spectral diffusion model in Step 2 and the spatial component generation branch in Step 3. During the sampling process of the dual-branch diffusion model, a shared guiding function is used to guide the generation of spectral components and generated features, and the observation features are iteratively optimized using the observation network to obtain a high-resolution hyperspectral image.

[0011] A hyperspectral fusion imaging system based on a dual-branch diffusion model framework is provided for implementing the above method. The system includes:

[0012] The first module takes as input a low spatial resolution hyperspectral image and a high spatial resolution multispectral image of the corresponding observation area, and decomposes the high resolution hyperspectral image to be reconstructed into a spectral component and a spatial component, wherein the spatial component consists of two parts: generated features and observed features.

[0013] The second module constructs a one-dimensional U-Net spectral diffusion model that can learn and generate continuous sequence features, and trains it using the acquired low spatial resolution hyperspectral images to generate spectral components that can accurately describe the observation scene.

[0014] The third module constructs a spatial component generation branch, which combines spatial information from two different sources: one is the generation features derived from a spatial diffusion model pre-trained on an external remote sensing image dataset, and the other is the observation features extracted from high spatial resolution multispectral images.

[0015] The fourth module utilizes the spectral diffusion model from the second module and the spatial component generation branch from the third module to generate the fusion result. During the sampling process of the dual-branch diffusion model, a shared guiding function is used to guide the generation of spectral components and generated features, and the observation network is used to iteratively optimize the observation features to obtain a high-resolution hyperspectral image.

[0016] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the hyperspectral fusion imaging method based on the dual-branch diffusion model framework described above.

[0017] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the hyperspectral fusion imaging method based on the aforementioned dual-branch diffusion model framework.

[0018] A computer program product includes a computer program that, when executed by a processor, implements the hyperspectral fusion imaging method based on the aforementioned dual-branch diffusion model framework.

[0019] Compared with existing technologies, the significant advantages of this invention are as follows: (1) This invention introduces a pre-trained spatial diffusion model and optimizes the spectral diffusion model using a self-supervised training paradigm, effectively overcoming the problem of weak generalization ability caused by the difficulty in obtaining large-scale training data in existing supervised learning methods; (2) In response to the problem that existing fully connected spectral diffusion models cannot effectively capture the continuous features of spectral curves, this invention proposes a spectral diffusion model based on a one-dimensional U-Net architecture. This model utilizes a hierarchical convolutional structure and a self-attention mechanism to effectively extract local correlations and long-distance continuous features in spectral curves; (3) This invention uses multiple pre-trained spatial diffusion models and observation networks to collaboratively generate spatial components, making full use of prior knowledge obtained from natural images and specific scene features extracted from observation images, effectively avoiding the problem that traditional single pre-trained spatial diffusion models cannot represent complex spatial-spectral structures in hyperspectral data; (4) This invention constructs a shared guiding function to achieve unified constraints and interactive optimization of the spatial generation features and spectral component generation process. This mechanism effectively promotes the collaborative fusion of spatial and spectral information, thereby further improving the spatial fidelity and spectral consistency of the fused imaging results.

[0020] The hyperspectral fusion imaging method based on the dual-branch diffusion model framework provided by this invention will be described in detail below with reference to the accompanying drawings. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the hyperspectral fusion imaging method within the dual-branch diffusion model framework of this invention.

[0022] Figure 2 This is a residual block structure diagram of the denoising network used in the one-dimensional U-Net diffusion model described in this invention.

[0023] Figure 3 This is a diagram of the denoising network structure used in the one-dimensional U-Net diffusion model described in this invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] like Figure 1 As shown, the present invention provides a hyperspectral fusion imaging method based on a dual-branch diffusion model framework, the method comprising:

[0026] Input a low spatial resolution hyperspectral image and a high spatial resolution multispectral image of the corresponding observation area, and decompose the high resolution hyperspectral image to be reconstructed into a spectral component and a spatial component, wherein the spatial component consists of two parts: generated features and observation features.

[0027] To address the continuous nature of spectral curves in hyperspectral images, a one-dimensional U-Net spectral diffusion model capable of learning and generating continuous sequence features is constructed. This model is trained using acquired low spatial resolution hyperspectral images to generate high-fidelity spectral components that accurately describe the observed scene.

[0028] A spatial component generation branch is constructed, which combines spatial information from two different sources: one is the generation features derived from a spatial diffusion model pre-trained on an external remote sensing image dataset, and the other is the observation features extracted from high spatial resolution multispectral images.

[0029] The fusion results are generated by using a spectral diffusion model and a spatial component generation branch. During the diffusion model sampling process, a shared guiding function is used to guide the generation of spectral components and generated features, and the observation features are iteratively optimized using an observation network to obtain a high-resolution hyperspectral image.

[0030] The overall structural framework of the hyperspectral fusion imaging method based on the dual-branch diffusion model of the present invention is shown in the diagram. The method specifically includes the following steps:

[0031] (1) Data input and target fusion image decomposition representation:

[0032] Step 1.1: Represent the input low spatial resolution hyperspectral image as a third-order tensor. ,in and These represent the width and height of the hyperspectral image, respectively. The number of bands in the hyperspectral image is represented; the input high spatial resolution multispectral image is represented as a third-order tensor. ,in and These represent the width and height of the multispectral image, respectively. This indicates the number of bands in a multispectral image.

[0033] Step 1.2: Using the idea of ​​low-rank decomposition, the high-resolution hyperspectral image to be reconstructed... It is decomposed into two low-rank components: a spatial component and a spectral component. The specific mathematical expression is:

[0034]

[0035] in, Represents spatial components, Indicates spectral components, This represents the number of channels after low-rank decomposition. This represents tensor mode-3 multiplication. Further, the spatial components are jointly characterized by generated features from a pre-trained spatial diffusion model and observed features extracted from multispectral images. Ultimately, the target high-resolution hyperspectral image... The representation model of can be defined by the following mathematical formula:

[0036]

[0037] in This represents the concatenation operation of two tensors along the channel dimension. This represents the extracted observation features. This represents the generated features from the pre-trained spatial diffusion model.

[0038] (2) Construct a one-dimensional U-Net spectral diffusion model for generating spectral components.

[0039] Step 2.1: Construct a spectral vector diffusion model based on a one-dimensional U-Net structure. The denoising network in the model outputs... Configured to receive a noisy one-dimensional spectral vector A time-step encoding using sinusoidal position embedding As input, to predict the noisy vector at the next time step Denoising signal Furthermore, the denoising network is constructed using a series of residual blocks and self-attention modules to learn continuous sequence features in the spectral curve. Figure 2 As shown, the residual block is characterized by and time step encoding For input, generate an output feature. Its internal data processing steps can be specifically represented as follows:

[0040]

[0041]

[0042]

[0043] in The vector represents the embedding vector at each time step, and MLP stands for Multilayer Perceptron. This represents a one-dimensional convolution with a kernel length of 3. SiLU represents the Sigmoid linear unit activation function, and GroupNorm represents group normalization. This indicates element-wise addition. Represents the first [unit] inside the residual block A convolutional block feature mapping function with a kernel length of 3, specifically... and These represent the first and second layer feature mapping functions in the residual block, respectively. This residual block can capture local correlations in the spectral vector using convolution operations. The self-attention module inputs features... and output features The specific steps can be represented as follows:

[0044] ; ; ;

[0045] in Let represent the query vector, key vector, and value vector in the self-attention mechanism, respectively; Matmul represents matrix multiplication; and GroupNorm represents group normalization. Here, attn represents the output features of the self-attention module, and softmax represents the normalized exponential function. Input features Feature dimensions, The projected feature dimension is represented by `Linear`, indicating a linear mapping layer. The self-attention module effectively captures long-range dependencies in the spectral curve by calculating the correlation weights between all bands within the spectral vector. The complete denoising network structure is as follows: Figure 3 As shown.

[0046] Step 2.2: Utilize low spatial resolution hyperspectral images For the output of the denoising network Optimize the training. Specifically, use hyperspectral images... Deconstructing in the spatial dimension, dividing the position of each pixel... spectral vectors on Each instance is considered an independent training instance. The model's parameters are optimized within this... The denoising network parameters are calculated by using a set of samples and aiming to accurately predict the noise added to the spectral vector. The optimization gradient is:

[0047]

[0048] in Indicates the parameter The gradient operator for differentiation, This represents a randomly sampled training sample instance. This represents the noise added to the spectral vector. This represents the noise scheduling coefficient. Specifically, the optimization training process of the network model is executed using the AdamW optimizer, and the training process lasts for a total of 50,000 epochs. In each iteration step, the batch size is 128, and the initial learning rate is set to... Furthermore, the spectral component It is a by The matrix consists of independent spectral vectors, therefore its denoising network output can be expressed as... A combination of the outputs of individual spectral vector denoising networks:

[0049]

[0050] in It is represented as a set of spectral vectors.

[0051] (3) Constructing spatial components to generate branches

[0052] Step 3.1: Obtain the DDPM-CD pre-trained spatial diffusion model for generating image texture structures. This pre-trained model was trained and optimized using 500,000 remote sensing RGB images cropped from the Google Earth engine platform. To further improve the generative model's ability to represent complex ground structures, this invention employs a multi-diffusion model parallel mechanism. The output of the denoising network for generating features can be represented as:

[0053]

[0054] in This represents the output of a single pre-trained denoising network. Indicates time step Time A noisy input sample, The number of pre-trained spatial diffusion models is set to 3 in practice.

[0055] Step 3.2: Construct an observation network to obtain observation features This observation network provides high spatial resolution multispectral images. As input, deep features are first extracted. Then learn spatial attention maps For the deep features The data is weighted and then output through a fully connected layer to obtain the observed features. The observation network is initialized with an identity mapping to accelerate the convergence of the method. Final spatial components. Represented as a combination of generated features and observed features:

[0056]

[0057] (4) Optimization of Sampling and Observation Features Guided by Diffusion Model

[0058] Step 4.1: Utilize the input hyperspectral image and multispectral images Build a shared bootstrap function Its mathematical expression is:

[0059] ,in This represents the tensor mode-3 product. and Indicates the final component and The estimate, Represents the space degradation operator, The spectral response function representing spectral degradation, and This represents the weighting coefficient. and In practice, it is always set to 1.

[0060] Step 4.2: Using the spectral diffusion model from Step 2.2 and the pre-trained spatial diffusion model from Step 3.1, and applying the shared guiding function constructed in Step 4.1 for correction. The output of the corrected denoising network is expressed as:

[0061]

[0062]

[0063] in and This represents the corrected outputs of the spatial denoising network and the spectral denoising network. and Indicates information about space components and spectral components gradient operator, and This indicates the step size of the shared bootstrap function, in actual implementation and Set them to 0.05 and 0.03 respectively.

[0064] Step 4.3: Sample and initialize components from a standard normal distribution. and ,in For the termination time step of the bi-branch diffusion model, noise is gradually removed using the corrected denoising network output obtained in step 4.2. This process is applied to any current time step during the backsampling process. The denoising sampling process can be expressed as:

[0065]

[0066]

[0067] in and Indicates the first Spatial and spectral components of the time step. Furthermore, , It is a predefined time-step noise ratio. Step sampling to obtain the final component and In actual implementation The time step noise ratio is expressed in exponential form, and the number of sampling steps is... Set it to 400.

[0068] Step 4.4: The result generated by combining the two-branch diffusion model and The observation features are iteratively optimized using the observation network described in step 3.2. The loss function of the observation network. for:

[0069]

[0070] To minimize To optimize the target, update the observation features. After optimization, the hyperspectral image was reconstructed.

[0071]

[0072] Step 4.5, iterate through steps 4.3 and 4.4 until the reconstructed hyperspectral image is obtained. Convergence. Outputs high-resolution hyperspectral images. .

[0073] Based on the same inventive concept, the present invention also provides a hyperspectral fusion imaging system with a dual-branch diffusion model framework, comprising:

[0074] The first module takes as input a low spatial resolution hyperspectral image and a high spatial resolution multispectral image of the corresponding observation area, and decomposes the high resolution hyperspectral image to be reconstructed into a spectral component and a spatial component, wherein the spatial component consists of two parts: generated features and observed features.

[0075] The second module addresses the continuous spectral curve characteristic of hyperspectral images by constructing a one-dimensional U-Net spectral diffusion model capable of learning and generating continuous sequence features. This model is trained using acquired low spatial resolution hyperspectral images to generate high-fidelity spectral components that accurately describe the observed scene.

[0076] The third module constructs a spatial component generation branch that combines spatial information from two different sources: one is the generation features derived from a spatial diffusion model pre-trained on an external remote sensing image dataset, and the other is the observation features extracted from high spatial resolution multispectral images.

[0077] The fourth module utilizes the spectral diffusion model from the second module and the spatial component generation branch from the third module to generate the fusion result. During the diffusion model sampling process, a shared guiding function is used to guide the generation of spectral components and generated features, and the observation network is used to iteratively optimize the observation features to obtain a high-resolution hyperspectral image.

[0078] The specific implementation methods of the first to fourth modules mentioned above are the same as those of the hyperspectral fusion imaging method in the aforementioned dual-branch diffusion model framework, and will not be repeated here.

[0079] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hyperspectral fusion imaging method of a dual-branch diffusion model framework, characterized in that, Includes the following steps: Step 1: Input a low spatial resolution hyperspectral image and a high spatial resolution multispectral image of the corresponding observation area. Decompose the high resolution hyperspectral image to be reconstructed into a spectral component and a spatial component, wherein the spatial component consists of two parts: generated features and observation features. Step 2 involves constructing a one-dimensional U-Net spectral diffusion model capable of learning and generating continuous sequence features, and training it using acquired low spatial resolution hyperspectral images to generate spectral components that accurately describe the observed scene. This includes the following steps: Step 2.1, constructing a one-dimensional U-Net spectral diffusion model, the denoising network output in the model is configured to receive a noisy one-dimensional spectrum vector and a time step encoding with sinusoidal positional embeddings as input to predict a denoised signal of the noisy vector at the next time step ;​ The denoising network is constructed using residual blocks and self-attention modules to learn continuous sequence features in the spectral curve; the residual blocks use features... and time step encoding For input, generate an output feature. The internal data processing steps are specifically represented as follows: ； ； ； in The vector represents the embedding vector at each time step, and MLP stands for Multilayer Perceptron. This represents a one-dimensional convolution with a kernel length of 3. SiLU represents the Sigmoid linear unit activation function, and GroupNorm represents group normalization. This indicates element-wise addition; Represents the first [unit] inside the residual block A convolutional block feature mapping function with a kernel length of 3. and These represent the first-layer feature mapping function and the second-layer feature mapping function in the residual block, respectively. Step 2.2: Utilize low spatial resolution hyperspectral images For the output of the denoising network Perform optimized training; convert low spatial resolution hyperspectral images Deconstructing in the spatial dimension, dividing the position of each pixel... spectral vectors on All are treated as independent training instances; parameter optimization of the spectral diffusion model is performed in... This is completed in a set of samples. , and Let represent the width, height, and number of bands of a low spatial resolution hyperspectral image, respectively. The denoising network parameters are defined with the goal of predicting the noise added to the spectral vector. The optimization gradient is: ； in Indicates the parameter The gradient operator for differentiation, This represents a randomly sampled training sample instance. This represents the noise added to the spectral vector. Noise scheduling coefficient; spectral component It is a by The matrix consists of independent spectral vectors, therefore its denoising network output is expressed as... A combination of the outputs of individual spectral vector denoising networks: ； in Represented as a set of spectral vectors; Step 3: Construct a spatial component generation branch that combines spatial information from two different sources: one is the generation features derived from a spatial diffusion model pre-trained on an external remote sensing image dataset, and the other is the observation features extracted from high spatial resolution multispectral images. Step 4: The fusion result is generated using the spectral diffusion model in Step 2 and the spatial component generation branch in Step 3. During the sampling process of the dual-branch diffusion model, a shared guiding function is used to guide the generation of spectral components and generated features, and the observation features are iteratively optimized using the observation network to obtain a high-resolution hyperspectral image.

2. The hyperspectral fusion imaging method based on a dual-branch diffusion model framework according to claim 1, characterized in that, Step 1: The specific steps are as follows: Step 1.1: Represent the input low spatial resolution hyperspectral image as a third-order tensor. The input high spatial resolution multispectral image is represented as a third-order tensor. ,in , and These represent the width, height, and number of bands of a high spatial resolution multispectral image, respectively. Step 1.2: Obtain the high-resolution hyperspectral image to be reconstructed. It is decomposed into two low-rank components: a spatial component and a spectral component. The spatial component is jointly characterized by the generated features of a pre-trained spatial diffusion model and the observed features extracted from the multispectral image. The target high-resolution hyperspectral image... The representation model of is defined by the following mathematical formula: ； in This represents the concatenation operation of two tensors along the channel dimension. Indicates spectral components, This represents the number of channels after low-rank decomposition. This represents the extracted observation features. This represents the generated features from the pre-trained spatial diffusion model. This represents tensor mode-3 multiplication.

3. The hyperspectral fusion imaging method based on a dual-branch diffusion model framework according to claim 2, characterized in that, Step 3, which generates a branch for the spatial component, combines spatial information from two different sources and specifically includes the following steps: Step 3.1: Obtain the DDPM-CD pre-trained spatial diffusion model to generate the texture structure of the image; using a multi-diffusion model parallel mechanism, the output of the feature-generating denoising network is represented as follows: ； in This represents the output of a single pre-trained denoising network. Indicates time step Time A noisy input sample, This represents the number of pre-trained spatial diffusion models; Step 3.2: Construct an observation network to obtain observation features This observation network uses high spatial resolution multispectral images. As input, deep features are first extracted. Then learn spatial attention maps For deep features The data is weighted and then output through a fully connected layer to obtain the observed features. ; The observation network is initialized with an identity mapping to accelerate the convergence of the method; the final spatial component Represented as a combination of generated features and observed features: 。 4. The hyperspectral fusion imaging method based on a dual-branch diffusion model framework according to claim 3, characterized in that, Step 4 specifically includes the following steps: Step 4.1: Utilize the input hyperspectral image and multispectral images Build a shared bootstrap function The mathematical expression is: ； in, and Indicates the final component and The estimate, Represents the space degradation operator, The spectral response function representing spectral degradation, and Indicates the weighting coefficient; Step 4.2: Using the spectral diffusion model from Step 2.2 and the pre-trained spatial diffusion model from Step 3.1, and correcting it using the shared guiding function constructed in Step 4.1, the output of the corrected denoising network is expressed as: ； ； in and This represents the corrected outputs of the spatial denoising network and the spectral denoising network. and Indicates information about space components and spectral components gradient operator, and Indicates the step size of the shared bootstrap function; Step 4.3: Sample and initialize components from a standard normal distribution. and ,in For the termination time step of the bi-branch diffusion model, noise is gradually removed using the corrected denoising network output obtained in step 4.

2. This process is applied to any current time step during the backsampling process. The denoising sampling process can be represented as: ； ； in and Indicates the first Spatial and spectral components of the time step; , It is a predefined time step noise ratio; through Step sampling to obtain the final component and ; Step 4.4: The result generated by combining the two-branch diffusion model and The observation features are iteratively optimized using the observation network described in step 3.

2. Loss function of the observation network for: ； To minimize To optimize the target, update the observation features. After optimization, the hyperspectral image was reconstructed. ； Step 4.5, iterate through steps 4.3 and 4.4 until the reconstructed hyperspectral image is obtained. Convergence; Output high-resolution hyperspectral images .

5. A hyperspectral fusion imaging system based on a dual-branch diffusion model framework, used to implement the method described in any one of claims 1 to 4, characterized in that, The system includes: The first module takes as input a low spatial resolution hyperspectral image and a high spatial resolution multispectral image of the corresponding observation area, and decomposes the high resolution hyperspectral image to be reconstructed into a spectral component and a spatial component, wherein the spatial component consists of two parts: generated features and observed features. The second module constructs a one-dimensional U-Net spectral diffusion model that can learn and generate continuous sequence features, and trains it using the acquired low spatial resolution hyperspectral images to generate spectral components that can accurately describe the observation scene. The third module constructs a spatial component generation branch, which combines spatial information from two different sources: one is the generation features derived from a spatial diffusion model pre-trained on an external remote sensing image dataset, and the other is the observation features extracted from high spatial resolution multispectral images. The fourth module utilizes the spectral diffusion model from the second module and the spatial component generation branch from the third module to generate the fusion result. During the sampling process of the dual-branch diffusion model, a shared guiding function is used to guide the generation of spectral components and generated features, and the observation network is used to iteratively optimize the observation features to obtain a high-resolution hyperspectral image.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the hyperspectral fusion imaging method based on the dual-branch diffusion model framework as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the hyperspectral fusion imaging method based on the dual-branch diffusion model framework as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the hyperspectral fusion imaging method based on the dual-branch diffusion model framework of any one of claims 1-4.

Images