A Two-Stage Diffusion Remote Sensing Hyperspectral Reconstruction Method for Disaster Scenarios

Abstract

This disclosure presents a two-stage diffusion-guided hyperspectral reconstruction method for disaster scenarios, comprising: acquiring datasets of multispectral and hyperspectral images; generating a priori noisy images; constructing conditional priors; iteratively generating intermediate hyperspectral images through reverse diffusion; and further optimizing the hyperspectral images through a spectral correction network to obtain more refined hyperspectral images. This invention employs a priori-guided diffusion method, utilizing spectral prior information to achieve accurate hyperspectral reconstruction with a single sample, effectively generating hyperspectral data in scenarios where hyperspectral data is scarce. The spectral correction process uses a progressive design, dividing the input image into multiple levels, gradually eliminating noise introduced during diffusion at different resolution levels, thus achieving high-quality hyperspectral image generation. This invention can complete spectral super-resolution generation tasks based on a single sample, and on the other hand, it can improve the structural accuracy of the hyperspectral images generated by the algorithm, significantly improving the hyperspectral data generation effect.

Description

Technical Field

[0001] This disclosure relates to the field of spectral image reconstruction technology, and in particular to a two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios. Background Technology

[0002] With the development of imaging sensors in recent years, such as microwave radar, multispectral sensors, and hyperspectral sensors, microwave images, multispectral images, and hyperspectral images can be generated respectively.

[0003] Different types of satellite imaging can provide complementary and comprehensive information about disaster sites, thus offering reliable information for emergency disaster relief. In particular, hyperspectral images, with their hundreds of spectral bands, can capture rich spectral information within a scene, aiding in rapid assessment of disaster situations during disaster relief missions. However, the high imaging cost and low spatial resolution of hyperspectral images hinder their application in emergency relief. In contrast, multispectral images, represented by RGB images, offer lower costs and higher spatial resolution. Given the complementary imaging characteristics between the two, spectral super-resolution methods that generate hyperspectral images from RGB images have significant application value.

[0004] Spectral super-resolution is a complex inverse problem. Solving it relies not only on simple interpolation methods, but also on advanced methods such as machine learning and deep learning to achieve high-precision spectral super-resolution. By constructing a model, complex spectral information is learned, thereby recovering higher-resolution spectral band information. Based on specific technical approaches, different methods can be categorized into sparse representation-based spectral super-resolution methods and deep learning-based spectral super-resolution methods. The former utilizes the sparsity of hyperspectral data, establishing a mapping relationship between hyperspectral data and RGB images to generate hyperspectral data through sparse mapping. This type of method requires a spectral response function, which often necessitates obtaining relevant parameters from the imaging sensor, thus limiting its application scenarios. With the rapid development of deep learning in recent years, end-to-end training allows for better learning of the characteristic distribution of target data. The network architecture of this type of method typically includes key components such as convolutional neural networks, generative adversarial networks, and attention mechanisms.

[0005] Although the above methods can effectively learn the representation of the data itself, accurate hyperspectral construction requires a large amount of data and lacks targeted network structure design for hyperspectral data structures. Therefore, the above methods still have significant limitations in the field of spectral super-resolution.

[0006] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0007] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0008] The purpose of this disclosure is to provide a two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios, thereby overcoming at least to some extent one or more problems caused by the limitations and defects of related technologies.

[0009] This disclosure first provides a two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios, including: Acquire raw multispectral-hyperspectral data in multiple bands, and divide the raw multispectral-hyperspectral data into a test set, a training set, and a validation set, which include multispectral images and hyperspectral images; The multispectral images in the test set are processed by band difference to form a multiband image, and the multiband image is concatenated with the noise image in the spectral dimension to form a prior noise image. Singular value decomposition is performed on the spectral response function, and then the conditional prior is constructed using the obtained singular value pseudo-inverse matrix, spectral singular vector, and spatial singular vector. The back diffusion iteration process is constrained by the aforementioned prior conditions. The back diffusion iteration is performed on the prior noise image to generate an intermediate hyperspectral image. The intermediate hyperspectral image is optimized using a loss function consisting of the root mean square error function and the total variation loss, as well as an optimizer, to obtain the optimized intermediate hyperspectral image. The optimized intermediate hyperspectral image is spectrally corrected. During the spectral correction process, downsampling is used to obtain aggregated feature maps of three different spectral dimensions. The three aggregated feature maps are connected in the spectral dimension to obtain a fused feature map. The fused feature map is then convolved and activated to obtain the reconstructed hyperspectral image.

[0010] In one embodiment of this disclosure, the step of performing band interpolation processing on the multispectral images in the test set to form a multiband image, and concatenating the multiband image and a noise image in parallel along the spectral dimension to form a priori noise image, includes: The first difference image is obtained by subtracting the blue band and the green band of the multispectral images in the test set; The second difference image is obtained by subtracting the green band and the red band of the multispectral images in the test set; The five-band image is formed in the band order of blue band, first difference image, green band, second difference image, and red band; The five-band image and the noise image are concatenated in parallel along the spectral dimension to form a priori noise image.

[0011] In one embodiment of this disclosure, the step of performing singular value decomposition on the spectral response function and then constructing a conditional prior using the obtained singular value pseudo-inverse matrix, spectral singular vector, and spatial singular vector includes: Singular value decomposition is performed on the spectral response function to obtain the spatial singular vector, the spectral singular vector, and the singular value matrix. The singular value pseudo-inverse matrix is ​​then obtained from the singular value matrix. Conditional priors are constructed using prior noisy images, spatial singular vectors, and singular value pseudo-inverse matrices. The conditional prior variance is obtained based on the stated conditional prior.

[0012] In one embodiment of this disclosure, the step of constraining the back-diffusion iteration process using the conditional prior and performing back-diffusion iteration on the prior noise image to generate an intermediate hyperspectral image includes: The aforementioned prior conditions are embedded into the reverse diffusion iteration process for constraint; Using the prior noisy image as the initial input for back diffusion, the following multiple rounds of back diffusion iterations are performed: For each time step, the current variable is input into the noise prediction network, which is a U-shaped structure without skip connections and includes a channel-level multi-head attention mechanism to focus on spectral dimension features and estimate the noise component of the current time step. Based on the inverse recursive formula, the aforementioned prior conditions and estimated noise components are used to eliminate some noise in the current variable, thus obtaining the variable for the next time step. During the iteration process, the spatial structure prior information of the multi-band image is preserved through the aforementioned conditional prior, while noise in the noisy image is removed, so that the variables gradually shift from noise-dominated to hyperspectral feature-dominated. When the iteration reaches time step 0, the output variable is the intermediate hyperspectral image.

[0013] In one embodiment of this disclosure, during the optimization of the intermediate hyperspectral image using a loss function consisting of the root mean square error function and the total variation loss, 1000 rounds of iterative optimization are performed, and the loss in the first 500 rounds does not participate in gradient backpropagation.

[0014] In one embodiment of this disclosure, the step of performing spectral correction on the optimized intermediate hyperspectral image, and obtaining aggregated feature maps of three different spectral dimensions by downsampling during the spectral correction process, includes: Downsampling the intermediate hyperspectral image yields three first feature maps with different spectral and spatial dimensions. The three types of first feature maps are convolved to obtain the corresponding three types of second feature maps; The first feature map is subjected to multi-head attention processing to obtain the third feature map; The second feature map and the third feature map are summed to obtain aggregated feature maps of three different spectral dimensions.

[0015] In one embodiment of this disclosure, the reconstruction method further includes: By using the root mean square error function as the loss function and employing the Adam optimizer for self-supervised learning, the spectral correction process is optimized.

[0016] In one embodiment of this disclosure, the root mean square error function is used as the loss function, and the Adam optimizer is used to carry out self-supervised learning to perform 500 rounds of iterative optimization on the spectral correction process.

[0017] This disclosure further provides a two-stage diffusion remote sensing hyperspectral reconstruction system for disaster scenarios, comprising: The dataset construction and processing module is used to acquire raw multispectral-hyperspectral data of multiple bands and divide the raw multispectral-hyperspectral data into a test set, a training set, and a validation set, which include multispectral images and hyperspectral images. The prior noise image generation module is used to perform band difference processing on the multispectral images in the test set to form a multiband image, and to connect the multiband image and the noise image in parallel in the spectral dimension to form a prior noise image. The conditional prior construction module is used to perform singular value decomposition on the spectral response function and then construct a conditional prior containing the singular value pseudo-inverse matrix and the conditional prior variance. The prior diffusion module is used to constrain the back diffusion iteration process using the conditional prior, perform back diffusion iteration on the prior noise image to generate an intermediate hyperspectral image, and optimize the intermediate hyperspectral image using a loss function composed of the root mean square error function and the total variation loss and an optimizer to obtain the optimized intermediate hyperspectral image. The spectral correction module is used to perform spectral correction on the optimized intermediate hyperspectral image. During the spectral correction process, downsampling is used to obtain aggregated feature maps of three different spectral dimensions. The three aggregated feature maps are connected in the spectral dimension to obtain a fused feature map. The fused feature map is then subjected to convolution and activation processing to obtain the reconstructed hyperspectral image.

[0018] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: This disclosure presents a two-stage diffusion-guided hyperspectral reconstruction method for disaster scenarios. It employs a priori-guided diffusion approach, utilizing prior spectral information to achieve accurate hyperspectral reconstruction from a single sample. This method effectively generates hyperspectral data even in scenarios with scarce hyperspectral data. The spectral correction process uses a progressive design, dividing the input image into multiple levels and gradually eliminating noise introduced during diffusion at different resolution levels, thereby achieving high-quality hyperspectral image generation. This invention, on the one hand, enables spectral super-resolution generation based on a single sample; on the other hand, it improves the structural accuracy of the generated hyperspectral image, significantly enhancing the hyperspectral data generation effect. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1 A flowchart illustrating a two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios in an exemplary embodiment of this disclosure is shown. Figure 2 This diagram illustrates a flowchart of generating a priori noise image in an exemplary embodiment of this disclosure; Figure 3 This diagram illustrates the process of generating a priori noise image in an exemplary embodiment of the present disclosure. Figure 4 This diagram illustrates a flowchart of a process in an exemplary embodiment of the present disclosure, in which a singular value decomposition is performed on the spectral response function, and then a conditional prior is constructed, which includes a singular value pseudo-inverse matrix and a conditional prior variance. Figure 5 This diagram illustrates a flowchart of generating an intermediate hyperspectral image in an exemplary embodiment of this disclosure; Figure 6 This diagram illustrates the model architecture of the reconstruction system in an exemplary embodiment of this disclosure. Figure 7 This diagram illustrates a flowchart of generating an aggregated feature map in an exemplary embodiment of this disclosure; Figure 8 This diagram illustrates the image processing flowchart in the reconstruction method of an exemplary embodiment of the present disclosure; Figure 9 This diagram illustrates the modular structure of a two-stage diffusion remote sensing hyperspectral reconstruction system for disaster scenarios, as shown in an exemplary embodiment of this disclosure. Detailed Implementation

[0021] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0022] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0023] This example implementation first provides a two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios. Please refer to [reference needed]. Figure 1 The method may include: S101-S105. Specifically: S101, acquire raw multispectral-hyperspectral data of multiple bands, and divide the raw multispectral-hyperspectral data into a test set, a training set, and a validation set, which include multispectral images and hyperspectral images. The training set is used for data training, and the validation set is used to verify the performance of the reconstruction method.

[0024] S102, perform band difference processing on the multispectral images in the test set to form a multiband image, and connect the multiband image and the noise image in parallel in the spectral dimension to form a priori noise image.

[0025] S103, perform singular value decomposition on the spectral response function, and then use the singular value pseudo-inverse matrix, spectral singular vector and spatial singular vector obtained by decomposition to construct conditional priors.

[0026] S104, constrain the back diffusion iteration process using the aforementioned prior conditions, perform back diffusion iteration on the prior noise image to generate an intermediate hyperspectral image, and optimize the intermediate hyperspectral image using a loss function composed of the root mean square error function and the total variation loss, and an optimizer, to obtain the optimized intermediate hyperspectral image.

[0027] S105, the optimized intermediate hyperspectral image is spectrally corrected. During the spectral correction process, downsampling is used to obtain three aggregated feature maps of different spectral dimensions. The three aggregated feature maps are connected in the spectral dimension to obtain a fused feature map. The fused feature map is then convolved and activated to obtain the reconstructed hyperspectral image.

[0028] In this embodiment, a priori-guided diffusion method is employed to achieve accurate hyperspectral construction using spectral prior information, enabling effective generation of hyperspectral data even in scenarios where hyperspectral data is scarce. The spectral correction process uses a progressive design, dividing the input image into multiple levels and gradually eliminating noise introduced during diffusion at different resolution levels, thereby achieving high-quality generation of the hyperspectral image. This invention, on the one hand, enables spectral super-resolution generation based on a single sample, and on the other hand, improves the structural accuracy of the hyperspectral image generated by the algorithm, significantly improving the hyperspectral data generation effect.

[0029] The specific process of each step in the above embodiments will be described below.

[0030] In S101, the dataset used in this invention is the Orbita hyperspectral satellite dataset launched by China. The dataset has a spectral resolution of 10 nanometers, a band distribution ranging from 400 to 1000 nanometers, and a spatial resolution of 10 meters. During actual training, atmospheric and geometric corrections were performed on the original L1B-level data, and the original data was cropped, ultimately yielding 9025 images with a spatial resolution of 256 nm. 256 hyperspectral data points were used to generate corresponding RGB images using the band selection recommended in the official manual. The final multispectral-hyperspectral dataset consists of 9025 pairs of multispectral-hyperspectral datasets, of which 7581 pairs were randomly selected for training the comparative model, 722 pairs for validation, and 722 pairs for testing.

[0031] Please refer to Figure 2 S102 includes S201~S204, as detailed below: S201, the first difference image B1 is obtained by subtracting the blue band and green band of the multispectral images in the test set.

[0032] S202, the second difference image B2 is obtained by subtracting the green band and the red band of the multispectral image in the test set.

[0033] S203, a five-band image F1 is formed in the band order of blue band, first difference image B1, green band, second difference image B2 and red band.

[0034] S204, the five-band image F1 and the noise image F2 are connected in parallel in the spectral dimension to form a priori noise image F3.

[0035] In this embodiment, combined with Figure 3 The initial input is 256. 256 The multispectral image of 3, after difference processing and sorting, forms a five-band image F1 with a size of 256. 256 5. The noisy image F2 consists of pure Gaussian noise, and its specific image size is 256. 256 26. Then, the five-band image F1 and the noisy image F2 are concatenated in parallel along the spectral dimension to form a pattern of size 256. 256 The prior noisy image F3 is 31. The prior noisy image F3 is the input data for subsequent diffusion processing.

[0036] Please refer to Figure 4 S103 includes S301 to S303, as detailed below: S301, perform singular value decomposition on the spectral response function to obtain the spatial singular vector, the spectral singular vector and the singular value matrix, and obtain the singular value pseudo-inverse matrix based on the singular value matrix; S302, constructs a conditional prior using a prior noisy image, spatial singular vectors, and singular value pseudo-inverse matrix; S303, obtain the conditional prior variance based on the conditional prior.

[0037] In this embodiment, the spectral response function , Singular Value Decomposition (SVD) is performed to obtain spatial singular vectors. Spectral singular vectors and singular value matrix The input data is characterized by singular value decomposition. Then, the singular value matrix is ​​inverted to obtain the singular value pseudo-inverse matrix. Based on the prior noisy image F3, and Obtaining conditional priors Intermediate variables in the iteration ,in, For the prior noisy image F3, This is a noisy image at time step t.

[0038] Next, please refer to Figure 5 The process of generating the intermediate hyperspectral image in S104 includes S401~S406, as follows: S401, The prior conditions are embedded into the reverse diffusion iteration process for constraint; S402, using the prior noise image as the initial input for back diffusion, perform the following multiple rounds of back diffusion iterations: S403, For each time step, the current variable is input into the noise prediction network. The noise prediction network is a U-shaped structure without skip connections and includes a channel-level multi-head attention mechanism to focus on spectral dimension features and estimate the noise component of the current time step. S404, Based on the inverse recursive formula, the prior conditions and estimated noise components are used to eliminate some noise in the current variable to obtain the variable for the next time step; S405, During the iteration process, the spatial structure prior information of the multi-band image is retained through the aforementioned conditional prior, while noise in the noisy image is removed, so that the variable gradually shifts from noise-dominated to hyperspectral feature-dominated. S406, when iterating to time step 0, the output variable is the intermediate hyperspectral image.

[0039] In this process, the prior noisy image F3 is the input data for the diffusion process. The inverse recursive formula for the diffusion process is as follows:

[0040] in, Let be the variance of the variable at time step t. Depend on Decide; For conditional prior variance; and All of these are hyperparameters for controlling the noise level in each step; Singular value matrix The i-th singular value in These are the intermediate variables obtained through the sampling network during the iteration. Here are the network parameters at time step t; It follows a normal distribution.

[0041] Please refer to Figure 6 In stage 1, the sampling network has a U-shaped structure, and its specific structural components are as follows:

[0042] The sampling network does not contain any skip connection modules, which maximizes the prediction and elimination of noise. In addition, the Multi-Head Attention used calculates channel-level attention, which can better reconstruct spectral information and reduce over-resolution error.

[0043] The iteration starting variable of the diffusion process can be obtained by using the reverse recursive formula of the above diffusion process. That is, processed data and The intermediate hyperspectral image H1, with a size of 256, was obtained through recursion. 256 31.

[0044] This application employs a two-stage diffusion network; please refer to... Figure 6 and Figure 8 The first stage involves inputting a priori noisy image, constructed from noise and multispectral data, into a priori diffusion module. After multiple iterations, this is the back-diffusion process of the diffusion model. During this process, prior information from the noise image is stored as useful information to impose spatial constraints on the generated hyperspectral image, while the noise image is removed during iteration. At each time step, a noise prediction network is used to estimate and eliminate image noise. Simultaneously, the prior noise information transmits structural information to the feature images generated at different time steps, ensuring that the generated feature images possess spatial features similar to the input multispectral image. The second stage involves inputting the feature images into a spectral correction module to recover spectral and spatial features at different granularities. This process is implemented through a hierarchical network architecture. In this architecture, convolution and attention mechanisms are simultaneously applied to feature maps at different scales to extract local and global information, thereby achieving progressive generation of the hyperspectral image across all scales.

[0045] In this application, the root mean square error (MSE) and total variation loss (TV Loss) are jointly used as the loss function of the model during the diffusion process. The Adam optimizer is employed (e.g., Diederik et al. in their paper DP Kingma and J. Ba, “Adam: A Method for Stochastic Optimization”). arXiv.org The Adam optimizer, proposed in [Journal Name], Dec. 22, 2014, optimizes the diffusion process. It performs 1000 iterations of optimization, and the loss from the first 500 iterations is not included in gradient backpropagation.

[0046] Next, please refer to Figure 7 In S105, the steps for obtaining the aggregation feature map through spectral correction are as follows: S501, downsampling the intermediate hyperspectral image to obtain three first feature maps with different spectral and spatial dimensions; S502, the three first feature maps are convolved to obtain the corresponding three second feature maps; S503, perform multi-head attention processing on the first feature map to obtain the third feature map; S504, sum the second feature map and the third feature map to obtain aggregated feature maps of three different spectral dimensions.

[0047] The spectral correction network used in the spectral correction process is a three-level network structure. First, the input intermediate hyperspectral image H1 is downsampled to obtain a size of 256. 256 The first feature map D1 of 31 has a size of 128. 128 The first feature map D2 of 124 has a size of 64. 64 The first feature map D3 is 496 in size. In the first level, D1 is first input to 1... 1 and 3 In the convolutional module of 3, the feature map D1 is then concatenated along the spectral dimension and input into 3. The second feature map D1' is obtained from the convolutional module of layer 3; simultaneously, D1 is input into the multi-head attention module, and then D1 is used as input to the LayerNorm module to obtain the third feature map D1''. Finally, D1' and D1'' are summed to obtain the size of the first layer, which is 256. 256 The aggregated feature map D11 is 31. Similarly, the output size of the second level is 256. 256 The aggregated feature map D21 of 124 has an output size of 256 compared to the third level. 256 The aggregation feature map D31 of 496.

[0048] Finally, D11, D21, and D31 are connected in the spectral dimension to form a structure of size 256. 256 651 fused feature map D4, respectively processed by 3 A 3x3 convolution with the LeakyReLU activation function yields a size of 256. 256 The final output of 31 is the reconstructed hyperspectral image.

[0049] Spectral correction is performed to recover fine-grained spectral information from the hyperspectral feature map generated after diffusion, referring to... Figure 6 Phase 2, the spectral correction module is structured as follows: Feature map H1 → Feature maps D1, D2, and D3 with decreasing PixelShuffle space size and increasing spectral dimension → Conv11 and Conv33 in parallel → Concat → Conv3 3 → LeakyReLU → Feature maps D11, D21, D31 → D1, D2, D3 are input into Multi-Head Attention → LayerNorm feature maps → D12, D22, D32 → Feature maps D11, D21, D31 are added to feature maps D12, D22, D32 to obtain E1, E2, E3 → Upsample → Concat → Output image size is 256 256 The final result F is 31. The spectral correction module designed in this application can divide the original feature map into different granularities to extract spatial and spectral features respectively. By aggregating the two across dimensions, a high-fidelity fine-grained hyperspectral image can be generated.

[0050] This application also optimizes the spectral correction process by using the root mean square error (MSE) as the model's loss function. The Adam optimizer was used to conduct self-supervised learning and the spectral correction process was iterated and optimized for 500 rounds.

[0051] In summary, this invention proposes a two-stage diffusion model for spectral super-resolution tasks, which mainly consists of a priori-guided diffusion module and a spectral correction module. This invention also proposes a training-free priori diffusion module that utilizes prior noise information, enabling high-quality conversion from multispectral to hyperspectral data with limited data volume. The proposed spectral correction network effectively reduces the impact of noise introduced by the diffusion module, and its multi-level structure enhances spectral fidelity at different scales.

[0052] The reverse diffusion process of this invention is implemented through a single-sample sampling network. This sampling network mainly consists of convolution operations, deconvolution operations, MultiHeadAttention, LayerNorm function, GELU activation function, and LeakyReLU activation function. The input is a three-channel multispectral image and a noisy image, and the output is a multi-channel hyperspectral image. The spectral correction network mainly consists of PixelShuffle and Conv3. 3, Conv1 1. It consists of the LeakyReLU activation function, the Upsample function, and the Multi-Head Attention function. The input is the feature image output from the first stage, and the output is the reconstructed hyperspectral image.

[0053] This application also provides a two-stage diffusion remote sensing hyperspectral reconstruction system for disaster scenarios. Please refer to [link / reference]. Figure 9The system includes: a dataset construction and processing module 101, a priori noisy image generation module 102, a conditional priori construction module 103, a priori diffusion module 104, and a spectral correction module 105.

[0054] Specifically, the dataset construction and processing module 101 is used to acquire raw multispectral-hyperspectral data of multiple bands and divide the raw multispectral-hyperspectral data into a test set, a training set and a validation set including multispectral images and hyperspectral images; The prior noise image generation module 102 is used to perform band difference processing on the multispectral images in the test set to form a multiband image, and to connect the multiband image and the noise image in parallel in the spectral dimension to form a prior noise image. The conditional prior construction module 103 is used to perform singular value decomposition on the spectral response function, and then use the singular value pseudo-inverse matrix, spectral singular vector and spatial singular vector obtained by decomposition to construct the conditional prior. The prior diffusion module 104 is used to constrain the back diffusion iteration process using the conditional prior, perform back diffusion iteration on the prior noise image to generate an intermediate hyperspectral image, and optimize the intermediate hyperspectral image using a loss function composed of the root mean square error function and the total variation loss and an optimizer to obtain the optimized intermediate hyperspectral image. The spectral correction module 105 is used to perform spectral correction on the optimized intermediate hyperspectral image. During the spectral correction process, downsampling is used to obtain aggregated feature maps of three different spectral dimensions. The three aggregated feature maps are connected in the spectral dimension to obtain a fused feature map. The fused feature map is then subjected to convolution and activation processing to obtain the reconstructed hyperspectral image.

[0055] In this embodiment, the functions of each module in the system and the image processing procedure are the same as those of the aforementioned reconstruction method, and will not be repeated here.

[0056] The following specific experimental examples further illustrate the beneficial effects of the two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios proposed in this application.

[0057] 1. Experimental Environment and Setup The software framework for this invention is PyTorch 1.6, and the hardware platform is an RTX 3090 graphics card. To verify the effectiveness of this invention, experiments were conducted on the Zhuhai-1 hyperspectral satellite dataset proposed by He et al. in the literature J. He, J. Li, Q. Yuan, H. Shen, and L. Zhang, “Spectral Response Function-Guided Deep Optimization-Driven Network for Spectral Super-Resolution,” IEEE Trans. Neural Netw. Learning ySst., pp. 1–15, 2021. The performance comparison results between this invention and other reconstruction methods are shown in Table 1.

[0058] Table 1. Experimental results of performance indicators of the implementation method of the present invention and existing spectral reconstruction methods.

[0059] 2. Analysis of Experimental Results As shown in Table 1, the proposed method outperforms existing spectral reconstruction methods in terms of root mean square error, mean relative absolute error, and relative root mean square error. This is because the proposed method fully utilizes the sparse spectral characteristics of remote sensing hyperspectral images to establish an effective spectral attention mechanism for effectively modeling the correlation between different spectral bands. Furthermore, the multi-level feature extraction method used in this invention also demonstrates good performance in removing image degradation, improving the model's expressive power and robustness under complex conditions.

[0060] As can be seen from the above, the prior diffusion module proposed in this invention can effectively reconstruct and generate multispectral images by means of a diffusion model, thereby realizing the construction of high-quality hyperspectral images. At the same time, the prior-guided prior diffusion module and the spectral correction module form a complementary coupling relationship, which can accurately address the shortcomings of the diffusion model in generating fine-grained structural information, realize the reconstruction of hyperspectral details at different granularities, and ultimately achieve efficient and accurate generation of hyperspectral images.

[0061] It should be noted that although several modules of the system for executing actions are mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules for embodiment. Components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0062] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios, characterized in that, include: Acquire raw multispectral-hyperspectral data in multiple bands, and divide the raw multispectral-hyperspectral data into a test set, a training set, and a validation set, which include multispectral images and hyperspectral images; The multispectral images in the test set are processed by band difference to form a multiband image, and the multiband image is concatenated with the noise image in the spectral dimension to form a priori noise image. Singular value decomposition is performed on the spectral response function, and then the conditional prior is constructed using the obtained singular value pseudo-inverse matrix, spectral singular vector, and spatial singular vector. The back diffusion iteration process is constrained by the aforementioned prior conditions. The back diffusion iteration is performed on the prior noise image to generate an intermediate hyperspectral image. The intermediate hyperspectral image is optimized using a loss function consisting of the root mean square error function and the total variation loss, as well as an optimizer, to obtain the optimized intermediate hyperspectral image. The optimized intermediate hyperspectral image is spectrally corrected. During the spectral correction process, downsampling is used to obtain aggregated feature maps of three different spectral dimensions. The three aggregated feature maps are connected in the spectral dimension to obtain a fused feature map. The fused feature map is then convolved and activated to obtain the reconstructed hyperspectral image. The step of constraining the back-diffusion iteration process using the conditional prior and performing back-diffusion iteration on the prior noise image to generate an intermediate hyperspectral image includes: The aforementioned prior conditions are embedded into the reverse diffusion iteration process for constraint; Using the prior noisy image as the initial input for back diffusion, the following multiple rounds of back diffusion iterations are performed: For each time step, the current variable is input into the noise prediction network, which is a U-shaped structure without skip connections and includes a channel-level multi-head attention mechanism to focus on spectral dimension features and estimate the noise component of the current time step. Based on the inverse recursive formula, the aforementioned prior conditions and estimated noise components are used to eliminate some noise in the current variable, thus obtaining the variable for the next time step. During the iteration process, the spatial structure prior information of the multi-band image is preserved through the aforementioned conditional prior, while noise in the noisy image is removed, so that the variables gradually shift from noise-dominated to hyperspectral feature-dominated. When the iteration reaches time step 0, the output variable is the intermediate hyperspectral image.

2. The two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios according to claim 1, characterized in that, The step of performing band interpolation processing on the multispectral images in the test set to form a multiband image, and concatenating the multiband image and the noise image in parallel along the spectral dimension to form a priori noise image includes: The first difference image is obtained by subtracting the blue band and the green band of the multispectral images in the test set; The second difference image is obtained by subtracting the green band and the red band of the multispectral images in the test set; The five-band image is formed in the band order of blue band, first difference image, green band, second difference image, and red band; The five-band image and the noise image are concatenated in parallel along the spectral dimension to form a priori noise image.

3. The two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios according to claim 2, characterized in that, The step of performing singular value decomposition on the spectral response function and then constructing a conditional prior including the singular value pseudo-inverse matrix and the conditional prior variance includes: Singular value decomposition is performed on the spectral response function to obtain the spatial singular vector, the spectral singular vector, and the singular value matrix. The singular value pseudo-inverse matrix is ​​then obtained from the singular value matrix. Conditional priors are constructed using prior noisy images, spatial singular vectors, and singular value pseudo-inverse matrices. The conditional prior variance is obtained based on the stated conditional prior.

4. The two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios according to claim 1, characterized in that, The intermediate hyperspectral image is optimized using a loss function consisting of the root mean square error function and the total variation loss, along with an optimizer. The optimization process involves 1000 iterations, and the loss in the first 500 iterations does not participate in gradient backpropagation.

5. The two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios according to claim 1, characterized in that, The step of performing spectral correction on the optimized intermediate hyperspectral image, and obtaining aggregated feature maps of three different spectral dimensions by downsampling during the spectral correction process, includes: Downsampling the intermediate hyperspectral image yields three first feature maps with different spectral and spatial dimensions. The three types of first feature maps are convolved to obtain the corresponding three types of second feature maps; The first feature map is subjected to multi-head attention processing to obtain the third feature map; The second feature map and the third feature map are summed to obtain aggregated feature maps of three different spectral dimensions.

6. The two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios according to claim 5, characterized in that, The reconstruction method further includes: By using the root mean square error function as the loss function and employing the Adam optimizer for self-supervised learning, the spectral correction process is optimized.

7. The two-stage diffusion remote sensing hyperspectral reconstruction method for disaster scenarios according to claim 6, characterized in that, Using the root mean square error function as the loss function, the Adam optimizer is employed to conduct self-supervised learning and perform 500 rounds of iterative optimization on the spectral correction process.

8. A two-stage diffusion remote sensing hyperspectral reconstruction system for disaster scenarios, characterized in that, The system performs hyperspectral reconstruction using the reconstruction method described in any one of claims 1 to 7, and the system comprises: The dataset construction and processing module is used to acquire raw multispectral-hyperspectral data of multiple bands and divide the raw multispectral-hyperspectral data into a test set, a training set, and a validation set, which include multispectral images and hyperspectral images. The prior noise image generation module is used to perform band difference processing on the multispectral images in the test set to form a multiband image, and to connect the multiband image and the noise image in parallel in the spectral dimension to form a prior noise image. The conditional prior construction module is used to perform singular value decomposition on the spectral response function, and then use the singular value pseudo-inverse matrix, spectral singular vector and spatial singular vector obtained by decomposition to construct the conditional prior; The prior diffusion module is used to constrain the back diffusion iteration process using the conditional prior, perform back diffusion iteration on the prior noise image to generate an intermediate hyperspectral image, and optimize the intermediate hyperspectral image using a loss function composed of the root mean square error function and the total variation loss and an optimizer to obtain the optimized intermediate hyperspectral image. The spectral correction module is used to perform spectral correction on the optimized intermediate hyperspectral image. During the spectral correction process, downsampling is used to obtain aggregated feature maps of three different spectral dimensions. The three aggregated feature maps are connected in the spectral dimension to obtain a fused feature map. The fused feature map is then subjected to convolution and activation processing to obtain the reconstructed hyperspectral image.

Images