Unsupervised diffusion model hyperspectral image super-resolution reconstruction method and system fused with spectrum compensation mechanism

Abstract

The invention discloses an unsupervised diffusion model hyperspectral image super-resolution reconstruction method and system fused with a spectrum compensation mechanism. The method comprises the following steps: acquiring an end member matrix containing spectral information from a low-resolution hyperspectral image, and further generating a spectrum enhanced end member matrix; inputting the low-resolution hyperspectral image, the high-resolution multispectral image and the spectrum enhancement end member matrix into a spectrum enhancement diffusion model to obtain a high-resolution abundance image; and multiplying the high-resolution abundance image by the spectrum enhancement end member matrix, and reconstructing a high-resolution hyperspectral image through a second adaptive spectrum enhancement module. The self-adaptive spectrum enhancement module is introduced into the diffusion model, spectrum information difference introduced by matrix decomposition under different spatial resolutions can be compensated under the condition that spatial information is not changed, the diffusion model is assisted to generate an accurate abundance image, the spatial-spectral correlation of the hyperspectral image is fully utilized, and the resolution of the hyperspectral image is improved. And the quality of the reconstructed hyperspectral image is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and deep learning, and particularly relates to a hyperspectral image super-resolution reconstruction method and system based on an unsupervised diffusion model fused with a spectral compensation mechanism. BACKGROUND

[0002] Hyperspectral imaging technology forms a unique spectral "fingerprint" by capturing the reflection or radiation information of an object in hundreds of continuous narrow bands, providing an irreplaceable data basis for fine material identification (such as precision agriculture crop classification, environmental pollutant monitoring, and mineral component analysis). However, due to the physical nature of imaging sensors (such as detector sensitivity, spectral element characteristics, and signal-to-noise ratio constraints), hyperspectral imaging systems generally face a fundamental contradiction: it is difficult to simultaneously achieve high spatial resolution and high spectral resolution. Generally, increasing spatial resolution requires reducing pixel size or increasing integration time, but this will result in a decrease in the photon energy received by a single pixel, thereby forcing the system to make compromises between reducing spectral resolution (merging bands) or sacrificing signal-to-noise ratio. The resulting low spatial resolution hyperspectral image seriously restricts the application performance in scenarios requiring fine spatial details (such as small-scale farmland plot identification and rock and mineral microstructure analysis). Hyperspectral image super-resolution (HSI-SR) technology is designed to break through this physical limitation, aiming to use algorithms to restore high spatial resolution hyperspectral images (HR-HSI) from easily obtainable low spatial resolution hyperspectral images (LR-HSI).

[0003] Early HSI-SR methods were mainly based on physical models and optimization theory, such as Bayesian inference-based probabilistic models, non-negative matrix factorization, CP decomposition based on tensor algebra, and sparse representation-based methods. Although the theoretical basis is clear, these methods highly depend on manually designed prior models and feature extraction rules. The modeling of the imaging degradation process is often overly idealized, resulting in poor robustness, generalization ability, and computational efficiency in complex real-world scenarios.

[0004] The rise of deep learning provides a powerful data-driven solution for HSI-SR, and deep learning methods are mainly divided into two categories: supervised learning and unsupervised learning. Under supervised learning, an end-to-end mapping function is often constructed by minimizing the spectral-spatial difference between the reconstructed hyperspectral image and the true value, but its reconstruction quality is highly dependent on the size and quality of the paired training set. Since high-resolution hyperspectral images need to rely on professional imaging equipment to realize, the high cost of acquisition leads to a lack of effective training samples, which not only increases the risk of model overfitting, but also has significant limitations in cross-band adaptability and scene migration robustness. Unsupervised learning only relies on the spectral and spatial characteristics of the input image to drive the update of the neural network parameters, although it solves the dependence on paired data, but the lack of explicit supervision signal will make the network fall into a local optimal solution.

[0005] In recent years, although the diffusion model shows potential, the existing methods generally have a key defect: they fail to fully model and compensate for the spectral distortion caused by the spatial resolution difference (i.e., between LR and HR) and the spectral response difference of different modal images (such as HSI and MSI). Therefore, there is an urgent need to develop a new type of super-resolution reconstruction technology that can effectively model and compensate for this cross-resolution spectral bias under an unsupervised (or weakly supervised) framework. SUMMARY

[0006] The purpose of the present application is to propose a hyperspectral image super-resolution reconstruction method and system of an unsupervised diffusion model with a spectral compensation mechanism, which ignores the spectral bias in the prior art, by introducing an adaptive spectral enhancement module in the iteration of the diffusion model to improve the reconstruction quality.

[0007] To achieve the above purpose, in a first aspect, the present application provides a hyperspectral image super-resolution reconstruction method of an unsupervised diffusion model with a spectral compensation mechanism, the method comprising:

[0008] obtaining a low-resolution hyperspectral image and a corresponding high-resolution multispectral image ;

[0009] performing matrix decomposition on the low-resolution hyperspectral image to obtain an endmember matrix E containing spectral information;

[0010] selecting a feature band combination with representative and information robustness from the endmember matrix E containing spectral information to generate a spectral enhancement endmember matrix ;

[0011] inputting the low-resolution hyperspectral image , the high-resolution multispectral image , and the spectral enhancement endmember matrix The high-resolution abundance image output by the spectral enhancement diffusion model is multiplied by a spectral enhancement endmember matrix to reconstruct a high-resolution hyperspectral image by a second adaptive spectral enhancement module .

[0012] The high-resolution abundance image output by the spectral enhancement diffusion model is multiplied by a spectral enhancement endmember matrix to reconstruct a high-resolution hyperspectral image by a second adaptive spectral enhancement module . . .

[0013] Preferably, the matrix decomposition employs a non-negative matrix factorization.

[0014] Preferably, the selecting of a representative and information-robust combination of characteristic wavebands from the endmember matrix E containing spectral information generates a spectral enhancement endmember matrix ; specifically:

[0015] The plurality of candidate wavebands are selected from the endmember matrix containing spectral information by a greedy optimal waveband selection method.

[0016] The transformation matrix T is composed of candidate wavebands. The endmember matrix E is then multiplied by the transformation matrix T to obtain a spectral enhancement endmember matrix E . . .

[0017] The spectral enhancement abundance image is defined according to the abundance image A and the transformation matrix T . . .

[0018] More preferably, the plurality of candidate wavebands are selected from the endmember matrix containing spectral information by a greedy optimal waveband selection method specifically as follows:

[0019] The spectral angle distance between each two different wavebands in the endmember matrix E containing spectral information and the energy of each waveband are calculated . . . .

[0020] The waveband with the maximum energy is selected from the endmember matrix E containing spectral information according to the energy of each waveband . . The remaining wavebands are taken as pre-candidate wavebands.

[0021] The first score is calculated according to the spectral angle distance between the waveband with the maximum energy and each pre-candidate waveband, and the energy of the corresponding pre-candidate waveband . .

[0022] The second score is calculated according to the first score and the energy of the waveband with the maximum energy All pre-candidate bands are sorted in descending order, and then the top s-2 pre-candidate bands are extracted; these s-2 pre-candidate bands and the band with the highest energy are then analyzed. This forms the s-1 candidate bands after reordering;

[0023] Calculate the second score for all remaining pre-candidate bands. Choose the second rating The pre-candidate band is used as the s-th candidate band. .

[0024] Preferably, the spectral enhancement diffusion model includes a pre-trained diffusion model and a first adaptive spectral enhancement module;

[0025] The inverse Markov chain process of the diffusion model includes multiple stages;

[0026] The spectral enhancement diffusion model includes a pre-trained diffusion model and a first adaptive spectral enhancement module;

[0027] The inverse Markov chain process of the diffusion model includes multiple stages;

[0028] At each stage, the gradient term incorporates the observation constraints into the generated image within the current time step, and then performs gradient correction on the generated image; at the same time, it estimates the high-quality abundance image of the last step for the generated abundance image; and performs adaptive spectral enhancement on the estimated high-quality abundance image of the last step through the first adaptive spectral enhancement module.

[0029] Preferably, the first adaptive spectral enhancement module and the second adaptive spectral enhancement module have the same structure, each containing two convolutional layers connected in series.

[0030] In a second aspect, the present invention provides a hyperspectral image super-resolution reconstruction system for implementing the method, comprising:

[0031] The data acquisition module is responsible for acquiring low-resolution hyperspectral images and high-resolution multispectral images;

[0032] The data processing module is responsible for performing matrix decomposition on low-resolution hyperspectral images to obtain endmember matrices containing spectral information; and selecting representative and robust feature band combinations from the endmember matrices containing spectral information to generate spectrally enhanced endmember matrices.

[0033] The image reconstruction module is responsible for inputting low-resolution hyperspectral images, high-resolution multispectral images, and spectral enhancement endmember matrices into the spectral enhancement diffusion model to obtain high-resolution abundance images; multiplying the high-resolution abundance images with the spectral enhancement endmember matrices, and reconstructing the high-resolution hyperspectral image through the second adaptive spectral enhancement module.

[0034] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method.

[0035] Fourthly, the present invention provides a computing device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method.

[0036] The beneficial effects of this invention are at least as follows:

[0037] (1) This invention creatively integrates an adaptive spectral enhancement module (ASEM) into the inverse denoising chain iteration process of the diffusion model. This module compensates in real time for spectral distortion accumulated due to spatial degradation and differences in spectral responses at different resolutions / modes. Combined with a spectral enhancement endmember matrix (Es) based on greedy optimal band selection, it maximizes the retention of characteristic band combinations with high information content and representational power, and significant differences between them (low spectral angular distance redundancy). This provides a purer and more robust spectral foundation (Es) for subsequent diffusion reconstruction and ASEM correction. s This significantly reduces the interference of noise and invalid bands on the reconstructed spectral curve, breaks through the bottleneck of spectral deviation, and significantly improves reconstruction accuracy and spectral fidelity.

[0038] (2) This invention is based on the unsupervised learning paradigm. The core relies on the optimization of the diffusion model driven by the constraints of the observed image. Through the powerful generation prior and observation constraints of the diffusion model, high-performance reconstruction under the unsupervised framework is achieved, getting rid of the strong dependence on paired data and solving the data bottleneck and overfitting risk problems inherent in supervised learning methods.

[0039] (3) The adaptive spectral enhancement module (ASEM) proposed in this invention learns and compensates for the inherent spectral bias patterns of a specific dataset or imaging system through training, rather than relying on fixed artificial assumptions. This data-adaptive compensation mechanism gives the model stronger robustness to different sensors, different scenes, and different degradation types, and can be more reliably applied to diverse real-world scenes and different imaging platforms. It reduces the risk of performance degradation due to scene migration or equipment replacement and expands the practical boundaries of hyperspectral super-resolution technology. Attached Figure Description

[0040] Figure 1 This is a flowchart of the super-resolution reconstruction method for hyperspectral images using an unsupervised diffusion model with a fused spectral compensation mechanism disclosed in this invention.

[0041] Figure 2 This is a schematic diagram illustrating the principle of nonnegative matrix decomposition and band selection to generate spectral enhancement endmember matrices in this invention.

[0042] Figure 3 This is a schematic diagram of the first adaptive spectral enhancement module and the second adaptive spectral enhancement module in this invention.

[0043] Figure 4 This is a schematic diagram of the spectral enhancement diffusion model of the present invention.

[0044] Figure 5 These are application effect diagrams of the present invention on the ICVL dataset, Pavia dataset, and Houston dataset. Detailed Implementation

[0045] To better illustrate the invention and advantages of this project, the invention will be further explained below with reference to the accompanying drawings and examples.

[0046] This invention enables unified processing of low-resolution hyperspectral images from different datasets to generate diverse high-resolution hyperspectral images, expanding the application scope of hyperspectral image super-resolution. This invention can be applied to multiple fields such as precision agriculture and environmental monitoring. This invention utilizes non-negative matrix factorization and a greedy optimization band selection method based on band spectral angular distance and spectral energy to obtain a spectral enhancement endmember matrix containing spectral information; it trains an adaptive spectral enhancement module using the product of the non-negative matrix factorization endmember matrix and the abundance image, along with the low-resolution hyperspectral image; it then uses the high-resolution multispectral image, the low-resolution hyperspectral image, and the spectral enhancement module to constrain the denoising of a pre-trained diffusion model to generate an abundance image; finally, it multiplies the abundance image and the spectral enhancement endmember matrix to obtain the high-resolution hyperspectral image, fully utilizing the spatial-spectral correlation of the hyperspectral image and improving the quality of the reconstructed hyperspectral image.

[0047] Specifically, such as Figure 1 As shown, this embodiment provides an unsupervised diffusion model hyperspectral image super-resolution reconstruction method with fused spectral compensation mechanism, including the following steps:

[0048] Step S1: Generate a test low-resolution hyperspectral image from a real high-resolution hyperspectral image. and high-resolution multispectral images .

[0049] Step S2: Process low-resolution hyperspectral images Perform matrix decomposition to obtain the endmember matrix E containing spectral information.

[0050] Step S3: Select representative and robust feature band combinations from the endmember matrix E containing spectral information to generate a spectral enhancement endmember matrix. .

[0051] Step S4: Convert the low-resolution hyperspectral image High-resolution multispectral images Spectral enhancement endmember matrix Inputting the data into a spectral enhancement diffusion model yields a high-resolution abundance image. .

[0052] Step S5: Output the high-resolution abundance image from the spectral enhancement diffusion model. With spectral enhancement endmember matrix Multiplication is performed to reconstruct a high-resolution hyperspectral image using a second adaptive spectral enhancement module. .

[0053] In one implementation method, step S1 specifically involves:

[0054] This embodiment uses hyperspectral images from the publicly available ICVL, Houston, and Pavia datasets. The number of channels in each hyperspectral image is preserved, and the images are cropped and normalized in the spatial dimension to obtain a 256×256 spatial resolution hyperspectral image. (Low-resolution hyperspectral images are also mentioned.) It is a high-resolution multispectral image obtained by applying 9×9 Lanczos blur and 8x downsampling to the cropped hyperspectral image. It is obtained by multiplying the cropped image with the spectral response function.

[0055] One implementation method, such as Figure 2 As shown, step S2 specifically involves:

[0056] The matrix decomposition uses nonnegative matrix decomposition to obtain the abundance matrix A and the endmember matrix E; specifically, it is expressed as follows:

[0057]

[0058]

[0059] in Representative low-resolution hyperspectral images The two-dimensional tensor after the modulus 3 expansion, and These represent the abundance matrix and endmember matrix after nonnegative matrix factorization, respectively. These represent the height, width, and number of bands of a low-resolution hyperspectral image, respectively. This indicates the number of endmembers in the nonnegative matrix decomposition; the superscript T indicates transpose. This represents the Frobenius norm function.

[0060] Nonnegative matrix factorization employs an iterative method with an alternating minimization strategy to progressively optimize. and That is, fixed renew Then fix renew In order to reduce and The difference between them. This process is repeated while ensuring that both are non-negative, until the convergence condition is met.

[0061] The matrix decomposition method, through strict nonnegativity constraints, conforms to the physical characteristics of hyperspectral reflectance while effectively suppressing negative artifacts caused by noise. Its sparse representation mechanism, during endmember extraction, captures the sparsity of high-dimensional space while adhering to the principles of linear mixture models. Furthermore, by representing data as a linear combination of basic components to effectively model the spectral mixing process, the generated coefficient matrix possesses richer spectral information.

[0062] One implementation method, such as Figure 2 As shown, step S3 includes:

[0063] 3-1 A greedy optimal band selection method is used to select representative and robust characteristic band combinations from the endmember matrix containing spectral information, resulting in... One candidate band;

[0064] 3-2 by The candidate bands constitute the transformation matrix. Then the endmember matrix and Multiplication yields the spectral enhancement endmember matrix. ;

[0065] 3-3 Based on abundance images Transformation matrix Define spectral enhancement abundance image ;

[0066] Specifically, step 3-1 is as follows:

[0067] 3-1-1 Calculating the endmember matrix containing spectral information Spectral angular distance between every two different bands and the energy of each band The process is represented as follows:

[0068]

[0069]

[0070] in, and This represents two different bands in E, where i ≠ j. This indicates that the Frobenius norm is used in the calculation;

[0071] 3-1-2 Based on the energy of each band From the endmember matrix The highest energy band was selected from the middle. The remaining bands are designated as pre-candidate bands;

[0072] 3-1-3 Based on the waveband of maximum energy The first score is calculated by considering the spectral angular distances between each pre-candidate band and the energy of the corresponding pre-candidate band. ;

[0073]

[0074] in Indicates the band with the highest energy. and pre-candidate bands Spectral angular distance between them;

[0075] 3-1-4 Based on the first score All pre-candidate bands are sorted in descending order, and then the top s-2 pre-candidate bands and the band with the highest energy are extracted. This constitutes the s-1 candidate bands after reordering. ;S>s≥2;

[0076] 3-1-5 Obtaining the s-th candidate band Specifically:

[0077] The second score for all remaining pre-candidate bands is calculated as follows:

[0078]

[0079] Choose the second rating The highest pre-candidate band is selected as the s-th candidate band. ;

[0080] For each candidate band, select the smallest... This is to force the new band to maintain sufficient difference from all the selected bands to avoid redundancy, and then, while satisfying the difference, retain the high-energy frequency band to balance the difference and the amount of information.

[0081] The present invention proposes a greedy optimal band selection method that combines spectral angular distance measurement and band energy assessment. This method can select a combination of characteristic bands that are representative and information robust to generate a spectral enhancement endmember matrix.

[0082] Specifically, step 3-2 is as follows:

[0083] Select The transformation matrix of each candidate band can be expressed as: Then the endmember matrix and Multiplication yields the spectral enhancement endmember matrix. The process is represented as:

[0084]

[0085] Specifically, step 3-3 is as follows:

[0086] Due to low-resolution hyperspectral images From abundance images Endmember matrix and transformation matrix The composition is shown in the following formula;

[0087]

[0088] Then, Substitute the formula into the above formula, and Redefining as a spectrally enhanced abundance image ;

[0089]

[0090] Specifically, step 4 is as follows:

[0091] like Figure 4 As shown, the spectral enhancement diffusion model includes a pre-trained diffusion model and a first adaptive spectral enhancement module. During the inverse Markov chain process of the pre-trained diffusion model, spectral information from the low-resolution hyperspectral image and spatial information from the high-resolution multispectral image are introduced through gradient terms, and the first adaptive spectral enhancement module is used to perform spectral enhancement at each time step.

[0092] The pre-trained diffusion model is a DDPM-CD diffusion model trained using 3-band remote sensing images taken by Sentinel-2.

[0093] The reverse Markov chain process includes multiple stages;

[0094] The gradient term in stage t, during the process of incorporating observation constraints into the generated image within the current time step, can be represented as:

[0095]

[0096] in This represents a pre-trained noise prediction network. Represents a standard normal distribution. Represents the cumulative noise attenuation coefficient Represents a time step. The input image represents stage t, where t∈[1,T]. Use random noise images; Hyperparameters representing noise intensity;

[0097] Since the diffusion model was not trained on the target data distribution, the generated image distribution deviates from the target image distribution. Therefore, the generated abundance images need to be modified. Perform gradient correction; first estimate the first... High-quality abundance images of the steps Specifically:

[0098]

[0099] right Adaptive spectral enhancement is performed through the first adaptive spectral enhancement module, and the process is as follows:

[0100]

[0101] in, This indicates the first adaptive spectral enhancement module. Representing the High-resolution hyperspectral images at the stage, Represents the product modulo 3;

[0102] for Gradient correction can be represented as follows:

[0103]

[0104] in and It is the step size of gradient descent, which controls the magnitude of the update. represents the spectral response function, and Y represents a low-resolution hyperspectral image; Indicates a downsampling operation; Indicates fuzzy operation; Represents high-resolution multispectral images; It is the consistency loss function for low-resolution hyperspectral images. It is the consistency loss function for high-resolution multispectral images.

[0105] The first adaptive spectral enhancement module and the second adaptive spectral enhancement module have the same structure, such as... Figure 3 As shown, it can correct the spectrum as much as possible without changing the spatial resolution, and the number of parameters per layer is only S×2S×1×1, making it very suitable for fast processing. Specifically, it contains two concatenated convolutional layers. The first convolutional layer uses LeakyReLU to introduce nonlinearity, allowing the network to learn a nonlinear mapping of spectral features. In the second layer, a linear layer is used for subtle linear adjustments. The process is represented as follows:

[0106]

[0107] in , These represent the operations of the first 1×1 convolutional layer and the second 1×1 convolutional layer, respectively; This represents the LeakyReLU activation function; This represents the output of the first adaptive spectral enhancement module and the second adaptive spectral enhancement module; This represents the input to the first adaptive spectral enhancement module and the second adaptive spectral enhancement module.

[0108]

[0109] in This represents the loss of the first adaptive spectral enhancement module and the second adaptive spectral enhancement module.

[0110] The adaptive spectral enhancement module introduced in this invention can compensate for the differences in spectral information introduced by matrix decomposition at different spatial resolutions without changing the spatial information, and assist the diffusion model in generating accurate abundance images.

[0111] This embodiment also provides a hyperspectral image super-resolution reconstruction system, including:

[0112] The data acquisition module is responsible for acquiring low-resolution hyperspectral images and high-resolution multispectral images;

[0113] The data processing module is responsible for performing matrix decomposition on low-resolution hyperspectral images to obtain endmember matrices containing spectral information; and selecting representative and robust feature band combinations from the endmember matrices containing spectral information to generate spectrally enhanced endmember matrices.

[0114] The image reconstruction module is responsible for inputting low-resolution hyperspectral images, high-resolution multispectral images, and spectral enhancement endmember matrices into the spectral enhancement diffusion model to obtain high-resolution abundance images; multiplying the high-resolution abundance images with the spectral enhancement endmember matrices, and reconstructing the high-resolution hyperspectral image through the second adaptive spectral enhancement module.

[0115] This embodiment provides an unsupervised diffusion model hyperspectral image super-resolution reconstruction method that integrates a spectral compensation mechanism. This method can process different low-resolution hyperspectral images and reconstruct high-resolution hyperspectral images. (Appendix) Figure 5 The super-resolution reconstruction results of the present invention on the ICVL dataset, Pavia dataset, and Houston dataset are shown, proving that the present invention can reconstruct high-quality, high-resolution hyperspectral images.

[0116] To further illustrate the effects of the present invention, this embodiment compares multiple methods under identical experimental conditions. The experimental results are shown in the table below:

[0117]

[0118] The results in the table show that the method disclosed in this invention achieves excellent super-resolution results, with PSNR, SSIM, and ERGAS metrics significantly higher than the comparative methods on multiple datasets. PSNR and SSIM primarily measure the spatial quality of the super-resolution hyperspectral image, while ERGAS primarily measures the spectral quality. Therefore, the method disclosed in this invention obtains hyperspectral images with smaller spatial and spectral errors, outperforming other methods in both spatial quality and spectral fidelity.

[0119] This invention also discloses a data processing device. Specifically, the device includes a storage component and a computing unit. The storage component contains an executable instruction set. When the computing unit runs the instruction set, it can execute various operation processes described in the embodiments.

[0120] The storage component may be configured with high-speed random access memory (RAM) and may also be expanded with non-volatile storage units (such as at least one disk storage device). Data channels between device nodes and other nodes are established through at least one transmission interface (supporting wired / wireless communication modes), adapting to multi-level network architectures (including the Internet, wide area networks, local area networks, and metropolitan area networks).

[0121] The bus system can be of various types, such as the Industry Standard Architecture (ISA) bus, the Peripheral Interconnect (PCI) bus, or the Extended Industry Standard Architecture (EISA) bus. Its internal architecture is divided into three functional modules: address transmission channel, data transmission channel, and control signal channel.

[0122] The storage component is used to store executable code. When the computing unit receives an execution instruction, it implements the method flow described in the various embodiments of the present invention by parsing and executing the code set. The computing unit is the carrier for running the method flow.

[0123] The computing unit can adopt a semiconductor integrated circuit architecture, possessing digital signal parsing and computation capabilities. Its implementation involves a dual path: direct hardware decoding via physical layer transistor logic circuits, or hardware-software collaborative computation based on a programmable instruction set. Specifically, the computing unit can adapt to multiple computing architectures: general-purpose computing units (including CPUs / NPs suitable for scalar instruction processing); heterogeneous computing units (including DSPs / ASICs / FPGAs) supporting customized algorithm acceleration; and reconfigurable computing units (including discrete gate circuits / transistor arrays) providing hardware-level dynamic configuration capabilities. The instruction execution carrier is implemented through a storage hierarchy: the software instruction set resides in a non-volatile memory matrix (including NOR Flash / EEPROM) or a high-speed temporary storage area (SRAM / register file) of the storage component, and is accessed and scheduled by the computing unit through a bus architecture. The hardware acceleration module can be directly mapped to on-chip storage media (Cache / BRAM) to achieve zero-latency instruction response.

[0124] This invention provides a readable storage medium in the form of a computer program product, which includes a computer-readable storage carrier with program code storage function. The program code carried on the carrier includes executable instructions for implementing the technical solutions in the foregoing method embodiments. The specific implementation details of the relevant method steps have been fully described in the foregoing method embodiments, and will not be repeated in this embodiment.

[0125] When the functional modules are implemented as independent software units and commercially operated as commodities, their operational data can be stored in digital information storage media. Based on this technical principle, the core innovative value of this patent, or the feature modules that distinguish it from existing technologies, can be presented in the form of application package. Such program suites typically reside in data carriers and contain multiple built-in operation commands to guide electronic devices (covering hardware facilities such as personal terminals, cloud servers, and networked devices) to execute all or core operation steps of the process detailed in the patent embodiments. The information carriers include physical media with program encoding and storage capabilities such as portable flash drives, external storage devices, firmware memory (ROM), dynamic access memory (RAM), magnetic storage disks, or optical recording media.

[0126] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for super-resolution reconstruction of hyperspectral images using an unsupervised diffusion model incorporating spectral compensation mechanisms, characterized in that: The method includes: Acquiring low-resolution hyperspectral images and corresponding high-resolution multispectral images ; For low-resolution hyperspectral images Perform matrix decomposition to obtain the endmember matrix E containing spectral information; From the endmember matrix E containing spectral information, select representative and robust combinations of characteristic bands to generate a spectrally enhanced endmember matrix. ; Low-resolution hyperspectral images High-resolution multispectral images Spectral enhancement endmember matrix Inputting the data into a spectral enhancement diffusion model yields a high-resolution abundance image. ; High-resolution abundance images output by the spectral enhancement diffusion model With spectral enhancement endmember matrix Multiplication is performed to reconstruct a high-resolution hyperspectral image using a second adaptive spectral enhancement module. .

2. The method according to claim 1, characterized in that, The matrix decomposition uses nonnegative matrix decomposition.

3. The method according to claim 1, characterized in that, The process involves selecting representative and robust feature band combinations from the endmember matrix E containing spectral information to generate a spectrally enhanced endmember matrix. ; Specifically: A greedy optimal band selection method is used to select multiple candidate bands from an endmember matrix containing spectral information; Depend on The candidate bands constitute the transformation matrix. Then the endmember matrix and transformation matrix Multiplication yields the spectral enhancement endmember matrix. ; Based on abundance image Transformation matrix Define spectral enhancement abundance image .

4. The method according to claim 3, characterized in that, The greedy optimal band selection method for selecting multiple candidate bands from an endmember matrix containing spectral information specifically involves: Calculate the endmember matrix containing spectral information Two different bands in the middle spectral angular distance and the energy of each band ; Based on the energy of each band From the endmember matrix The highest energy band was selected from the middle. The remaining bands are designated as pre-candidate bands; Based on the highest energy band The first score is calculated by considering the spectral angular distances between each pre-candidate band and the energy of the corresponding pre-candidate band. ; Based on the first rating All pre-candidate bands are sorted in descending order, and then the top s-2 pre-candidate bands are extracted; these s-2 pre-candidate bands and the band with the highest energy are then analyzed. This forms the s-1 candidate bands after reordering; Calculate the second score for all remaining pre-candidate bands. Choose the second rating The pre-candidate band is used as the s-th candidate band. .

5. The method according to claim 1, characterized in that, The spectral enhancement diffusion model includes a pre-trained diffusion model and a first adaptive spectral enhancement module; The inverse Markov chain process of the diffusion model includes multiple stages; At each stage, the gradient term incorporates the observation constraints into the generated image within the current time step, and then performs gradient correction on the generated image; at the same time, it estimates the high-quality abundance image of the last step for the generated abundance image; and performs adaptive spectral enhancement on the estimated high-quality abundance image of the last step through the first adaptive spectral enhancement module.

6. The method according to claim 5, characterized in that, The first adaptive spectral enhancement module and the second adaptive spectral enhancement module have the same structure, both containing two convolutional layers connected in series.

7. A hyperspectral image super-resolution reconstruction system implementing the method of any one of claims 1-6, characterized in that... include: The data acquisition module is responsible for acquiring low-resolution hyperspectral images and high-resolution multispectral images; The data processing module is responsible for performing matrix decomposition on low-resolution hyperspectral images to obtain endmember matrices containing spectral information. A spectrally enhanced endmember matrix is ​​generated by selecting representative and robust feature band combinations from an endmember matrix containing spectral information. The image reconstruction module is responsible for inputting low-resolution hyperspectral images, high-resolution multispectral images, and spectral enhancement endmember matrices into the spectral enhancement diffusion model to obtain high-resolution abundance images; multiplying the high-resolution abundance images with the spectral enhancement endmember matrices, and reconstructing the high-resolution hyperspectral image through the second adaptive spectral enhancement module.

8. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-6.

9. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-6.

Images