Hyperspectral image super-resolution method with alternating diffusion band selection and degradation adaptation

Abstract

The application relates to a hyperspectral image super-resolution method based on alternating diffusion band selection and degradation adaptation. The method comprises the following steps: constructing a blind image restoration target function and decomposing the function into a high-quality image and a degradation matrix solving subproblem. Then, a hyperspectral image super-resolution model containing three modules is constructed: a band selection and Gaussian dimension reduction module selects bands with large information quantity according to a variance distribution and a Gaussian attenuation model; a sampling priori and degradation kernel estimation module generates a random degradation matrix by Monte Carlo sampling, and combines a data fidelity loss function to optimize parameters to dynamically estimate the degradation matrix; a frequency band projection and diffusion guided image super-resolution module solves a high-quality image by means of a pre-trained diffusion model, band selection and degradation matrix solving, and optimizes an intermediate high-quality image through a data fidelity loss function and total variation regularization. Finally, the three modules are iterated to a preset number of times, and the obtained intermediate high-quality image is the result after restoration. The method can improve the image restoration performance and efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a hyperspectral image super-resolution method with alternating diffusion band selection and degradation adaptation. BACKGROUND

[0002] Hyperspectral image super-resolution (HSI-SR) technology aims to reconstruct high-resolution (HR) hyperspectral images from low-resolution (LR) ones. This process not only significantly improves the spatial resolution of images but also preserves the rich spectral information of hyperspectral images, providing a foundation for more accurate image analysis and applications. In the field of remote sensing, HSI-SR technology enables images acquired from space or high-altitude platforms to more clearly reveal the details and characteristics of ground objects, thereby improving the accuracy of tasks such as land use classification, crop growth monitoring, and mineral resource exploration. In environmental monitoring, hyperspectral images processed by super-resolution can more sensitively capture subtle differences in environmental pollution, water quality changes, and forest degradation, providing scientific basis for environmental assessment and protection measures. In the field of agriculture, this technology helps the development of precision agriculture by analyzing the health status of crops, the distribution of pests and diseases, and the soil nutrient content through high-resolution spectral information, optimizing agricultural production management decisions. In the field of geological exploration, HSI-SR technology can help geologists more accurately identify rock types, mineral distribution, and geological structures, providing key data support for the development of mineral resources and the prevention of geological disasters.

[0003] However, HSI-SR tasks face many severe challenges. First, the HSI-SR problem itself is ill-posed, meaning that there are multiple possible solutions for the mapping from LR images to HR images, making it extremely difficult to find a unique and accurate HR image. Second, the complexity of hyperspectral data further exacerbates this problem. Hyperspectral images usually have hundreds of bands, each corresponding to different spectral information. These bands not only have rich correlations but also contain unique detailed features. How to fully utilize these spectral information in the super-resolution reconstruction process while avoiding excessive computational cost and overfitting problems is a key difficulty that current research needs to solve. In addition, the degradation model in practical applications is often complex and variable, including blurring, noise, downsampling, and other factors. Accurate modeling and estimation of these degradation processes are crucial for effective super-resolution reconstruction. In summary, although HSI-SR technology has great application potential in multiple fields, to achieve efficient and accurate super-resolution reconstruction, many technical obstacles and challenges still need to be overcome.

[0004] Initially, hyperspectral image super-resolution (HSI-SR) techniques primarily relied on model-driven methods, but now most have shifted to learning-based approaches. Learning-based HSI-SR methods are mainly divided into supervised and unsupervised categories. Supervised HSI-SR techniques have achieved significant results using deep neural networks (DNNs) trained on large LR-HR paired datasets. Network architectures include spectral-spatial networks, attention mechanisms, and Transformer-based models. However, these methods typically require large amounts of labeled HSI data, which is extremely costly to obtain and often impractical in certain real-world scenarios. Some methods attempt to mitigate the data dependency problem by utilizing readily available RGB datasets as auxiliary information, but in testing scenarios, performance still significantly degrades when encountering conditions different from those during training (such as different HSI datasets or kernel settings).

[0005] Recently, unsupervised HSI-SR methods have gained increasing popularity. These methods typically design an objective function that uses a known degenerate kernel and observed LR HSI to guide the updates of an untrained network. Major unsupervised methods include Deep Image Prior (DIP) methods and diffusion model methods. However, existing unsupervised methods suffer from two main limitations: first, insufficient utilization of band information; the spectral redundancy of hyperspectral images leads to inefficient bandwidth utilization, as commonly addressed by methods such as full-band iteration or fixed-bandwidth compression (e.g., SVD-based methods); second, oversimplification of the degenerate model; current unsupervised methods use fixed degenerate priors (such as fixed blur kernels) to constrain the solution space of HSI-SR, while degenerate priors in real-world scenarios are often unknown and complex. Summary of the Invention

[0006] Therefore, it is necessary to provide a hyperspectral image super-resolution method with alternating diffusion band selection and degradation adaptation that can improve image restoration performance and efficiency, addressing the aforementioned technical problems.

[0007] A hyperspectral image super-resolution method with alternating diffusion band selection and degradation adaptation, the method comprising:

[0008] Obtain the target image to be restored; construct the objective function for the blind image restoration problem of the target image;

[0009] The objective function is decomposed into two sub-problems: solving for high-quality images and the degradation matrix. A hyperspectral image super-resolution model is constructed. The hyperspectral image super-resolution model includes a band selection and Gaussian dimensionality reduction module, a sampling prior and degradation kernel estimation module, and a frequency band projection and diffusion-guided image super-resolution module.

[0010] A band selection and Gaussian dimensionality reduction module is used to dynamically select bands with high information content through variance distribution and Gaussian attenuation model;

[0011] The sampling prior and degradation kernel estimation module is adopted, a random degradation matrix is generated by using Monte Carlo sampling, and the neural network parameters are optimized by combining a data fidelity loss function to realize dynamic estimation of the degradation matrix.

[0012] The band projection and diffusion guided image super-resolution module is adopted, a high-quality image is solved by means of a pre-trained diffusion model, band selection and a degradation matrix, and an intermediate high-quality image is optimized by a data fidelity loss function and total variation regularization.

[0013] According to the three-module iterative recovery in the hyperspectral image super-resolution model until the number of iterations meets a preset number, the intermediate high-quality image obtained by the current iteration is a high-quality image after the target image is recovered.

[0014] The above-mentioned alternating diffusion band selection and degradation adaptive hyperspectral image super-resolution method decomposes the target function into two sub-problems of solving the high-quality image and the degradation matrix after the target image is brought into the target function of the blind image restoration problem, solves the two sub-problems by using an alternating iterative solution, in each iteration, firstly calls the solving process of the degradation matrix solving sub-problem, updates and optimizes the estimated value of the degradation matrix by using a neural network combined with a random prior guidance and a data fidelity loss function. Then, the degradation matrix just updated and the pre-trained diffusion model are used to solve the high-quality image solving sub-problem, and a new intermediate high-quality image is obtained by optimizing and constraining the data fidelity loss function and the total variation regularization. This process is alternated between the two, and each iteration can further improve the quality and accuracy of image restoration based on the result of the previous iteration. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 FIG. 1 is a flowchart of an embodiment of an alternating diffusion band selection and degradation adaptive hyperspectral image super-resolution method. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0017] In one embodiment, as shown in FIG. 1, an alternating diffusion band selection and degradation adaptive hyperspectral image super-resolution method is provided, comprising the following steps: Figure 1

[0018] Step 102, obtaining a target image to be recovered; constructing a target function of a blind image restoration problem of the target image; decomposing the target function into two sub-problems of solving a high-quality image and a degradation matrix. ​

[0019] An image to be super-resolved is obtained, and the image is a low-resolution (LR) degraded image. The image is brought into an objective function of a blind image restoration problem.

[0020] For an HSI super-resolution problem, a degradation process of a low-resolution image is generally shown in the following formula:

[0021] (1)

[0022] In formula (1), denotes a degraded image, denotes a corresponding high-quality image, that is, an image after image restoration, denotes a degradation matrix, denotes additive noise.

[0023] In a non-blind image restoration problem, the high-quality image is unknown, and the degradation matrix is known. For example, in an image super-resolution problem, the degradation matrix is a downsampling matrix, in an image inpainting problem, the degradation matrix is a mask matrix, and in an image deblurring problem, the degradation matrix is a known blur matrix.

[0024] In a blind image restoration problem, the high-quality image is unknown, and related prior information of the degradation matrix is also unknown. For example, in a blind image deblurring problem, the degradation matrix is an unknown blur matrix, and in image deraining, dehazing, and de-shadowing problems, the degradation matrix is a corresponding rain, haze, or shadow matrix. In actual applications, real image degradation can not be only a single image degradation type, but can be a combination of multiple different degradation matrices. The present application is directed to image restoration in a blind image restoration problem.

[0025] Further, a non-blind image restoration problem can be regarded as a special form of a blind image restoration problem. Since the degradation matrix is known in a non-blind image restoration problem, only the high-quality image needs to be solved. Therefore, an objective function of a non-blind image restoration problem can be written as:

[0026] (2)

[0027] An objective function of a blind image restoration problem is shown as follows:

[0028] (3)

[0029] The objective function aims to recover the high-quality image by minimizing the difference between the high-quality image and the degraded image, while considering the regularization term of the image.

[0030] In formula (3), and denote the high-quality image and the degradation matrix , the prior term, and denote their corresponding weights. The non-blind image restoration problem can be regarded as a special form of the blind image restoration problem. Since the degradation matrix is known in the non-blind image restoration problem, for the convenience of expression, the subsequent image restoration problem is introduced in the form of the blind image restoration problem in formula (3).

[0031] Since there are two unknown variables to be solved in formula (3), the problem is further decomposed into two sub-problems to solve the high-quality image and the degradation matrix respectively:

[0032] (4)

[0033] In this embodiment, for the two unknown variables in formula (4), a blind hyperspectral image super-resolution method based on dynamic waveband selection and degradation adaptation of alternating diffusion is proposed to solve , that is, to solve the degradation matrix , a pre-trained, ready-to-use diffusion model is used to solve , that is, to solve the high-quality image .

[0034] Step 104, construct a hyperspectral image super-resolution model; the hyperspectral image super-resolution model includes a waveband selection and Gaussian dimension reduction module, a sampling prior and degradation kernel estimation module, and a frequency band projection and diffusion guided image super-resolution module.

[0035] Step 106, use the waveband selection and Gaussian dimension reduction module to dynamically select wavebands with large amounts of information through variance distribution and Gaussian attenuation models.

[0036] The core of the band selection and Gaussian dimension reduction module is to dynamically select bands with large amounts of information through variance distribution and Gaussian attenuation model, thereby avoiding high computational cost brought by full-band processing, including first calculating the variance of each band, and bands with large variance usually contain more information. A Gaussian-based attenuation model is adopted to adjust the band selection according to the differences between bands, ensuring that the selected bands with the largest amount of information are flexible and sensitive to changes, thereby avoiding the high computational cost of current methods for full-band hyperspectral data analysis. The diversity of the selected bands is ensured, and the computational efficiency is also improved. In each iteration process, the most suitable band combination is dynamically selected according to the current image features and recovery requirements, providing strong support for subsequent image restoration.

[0037] Specifically, given the observed degraded image , the goal of the present application is to select a subset of bands from the bands , where , such that the posterior probability of the selected bands is maximized given . This can be formulated as a maximum a posteriori (MAP) problem, as shown in equation (5):

[0038] (5)

[0039] However, direct calculation of this problem becomes infeasible due to the combinatorial explosion of possible band combinations. Therefore, the present application adopts an efficient approximation method based on sampling to approximately solve the MAP solution. According to Bayes' theorem, the posterior distribution can be sampled by the ratio of the likelihood term and the prior term , as shown in equation (6):

[0040] (6)

[0041] For the likelihood term , the present application assumes that bands with higher variance contain more information, and therefore can better explain the observed data . The present application approximates the likelihood as the product of the variances of the selected bands, as shown in equation (7):

[0042] (7)

[0043] where is the variance of band , and the calculation formula (8) is:

[0044] (8)

[0045] Here is the mean of the waveband .

[0046] For the prior term , the present application aims to select diverse and non-adjacent wavebands to avoid redundancy, since adjacent wavebands in hyperspectral images are usually highly correlated. The present application employs a Gaussian function to reduce the probability of wavebands near the previously selected ones, as shown in equation (9):

[0047] (9)

[0048] This approach is conceptually similar to the use of Gaussian kernels in spectral clustering or the Gaussian function in non-maximum suppression (NMS) in object detection, where the Gaussian function models the similarity or influence between nearby entities. By combining the likelihood term and the prior term, the present application can obtain an approximate MAP solution by repeating the sampling process . Finally, the hyperspectral image is selected from the selected wavebands and used for subsequent kernel estimation and hyperspectral image restoration.

[0049] Step 108, using the sampling prior and degradation kernel estimation module, generates a random degradation matrix using Monte Carlo sampling, and optimizes the neural network parameters combined with the data fidelity loss function to realize dynamic estimation of the degradation matrix.

[0050] The main function of the sampling prior and degradation kernel estimation module is to generate a random degradation matrix through Monte Carlo sampling, and to optimize the parameters of the neural network combined with the data fidelity loss function, thereby realizing the dynamic estimation of the degradation matrix. This includes randomly sampling a series of possible degradation matrices using the Monte Carlo method, which are used as labels to guide the learning of the neural network. The predicted degradation matrix output by the neural network is compared with these labels, and the loss function is calculated. According to the predicted degradation matrix, the intermediate high-quality image obtained in the last iteration is degraded, and the degraded image is compared with the target image to calculate the data fidelity loss function. By considering these two loss information comprehensively, the sampling prior and degradation kernel estimation module can effectively update the parameters of the neural network, making the predicted degradation matrix more accurate, thereby providing a reliable basis for the restoration of high-quality images.

[0051] Specifically, first, the sampling kernel prior is performed. Based on the Gaussian approximation of the blur kernel, the present application constructs an unsupervised but effective pre-experience model. For any Gaussian kernel with size , it can be defined as equation (10):

[0052] ​(10)

[0053] where, is the precision matrix, is the spatial coordinate, when the size of the kernel is determined, the blur kernel is completely determined by the precision matrix , e.g., variance and rotation angle . The Gaussian prior can be represented by the following integral formula (11):

[0054] (11)

[0055] However, it is very difficult to collect all possible . Therefore, the Monte Carlo (MC) simulation is adopted in this application to approximate the above integral, as shown in formula (12):

[0056] (12)

[0057] where, denotes the sampling series, denotes the precision matrix of the -th random sampling, denotes the corresponding blur kernel, denotes the weight of the -th sampling, which can be obtained by calculating the posterior probability formula (13):

[0058] (13)

[0059] where, denotes the selected waveband from the high-resolution HSI.

[0060] For the estimation of the blur kernel, a lightweight network is adopted to estimate the kernel, which takes the fixed random noise as input, formula as (14):

[0061] (14)

[0062] where, is the network parameter. The update of the parameter includes two items: data consistency item and random prior item . Given the kernel randomly sampled by Monte Carlo simulation, the optimization of the network parameter can be represented as formula (15):

[0063] (15)

[0064] wherein,

[0065] (16)

[0066] (17)

[0067] The optimization is realized by the famous Adam algorithm . The optimized can be used to generate the kernel by formula (14).

[0068] In step 110, a band projection and diffusion guided image super-resolution module is adopted to solve the high-quality image by means of a pre-trained diffusion model, band selection and degradation matrix, and to optimize the intermediate high-quality image by a data fidelity loss function and total variation regularization.

[0069] The band projection and diffusion guided image super-resolution module optimizes the intermediate high-quality image by a data fidelity loss function and total variation regularization to ensure that the restored image is both faithful to the original image and has good visual quality. First, a pre-trained diffusion model is used, which can gradually recover the details and structure of the image from noise. Combined with the intermediate degradation matrix, the DHR module can better understand the degradation of the image and thus more targetedly perform image restoration.

[0070] In order to ensure that the restored image is consistent with the target image, the band projection and diffusion guided image super-resolution module will calculate the data fidelity loss function to measure the difference between the restored image and the target image. In order to maintain the edges and details of the image, the band projection and diffusion guided image super-resolution module also introduces a total variation regularization term to constrain the smoothness and details of the image. Through the joint action of the two mechanisms, the band projection and diffusion guided image super-resolution module can effectively optimize the intermediate high-quality image and finally obtain a clear and accurate restoration result.

[0071] Specifically, the denoising diffusion probability model (DDPM) defines a T-step forward process of gradually adding noise to data and a T-step reverse process of reconstructing the target data from noise. When a ready-made DDPM is used to solve the image restoration problem, the reverse process is used as the inference process to estimate the high-quality image, specifically as formula (18):

[0072] (18)

[0073] wherein, denotes the HR image estimated at the step is a hyperparameter. Given the estimated , observed LR HSI and estimated kernel The data consistency term can be expressed as equation (19):

[0074] (19)

[0075] To constrain the additional noise of estimated HSI , TV regularization is implemented to realize HSI prior, which can be expressed as equation (20):

[0076] (20)

[0077] Combining the data consistency term and the TV regularization term, the prediction of HR HSI can be obtained by solving the following optimization problem equation (21):

[0078] (21)

[0079] Based on the network, the full-band hyperspectral image projection adopts a lightweight network to learn the mapping between and , which can be expressed as equation (22):

[0080] (22) wherein, the network input

[0081] is a fixed random noise. The parameter is optimized by minimizing the data consistency loss equation (23):

[0082] (23)

[0083] wherein,

[0084] (24)

[0085] The optimized is used to generate the mapping matrix . The obtained is multiplied by the mapping matrix to obtain the full-band HR HSI :

[0086] (25)

[0087] It should be noted that the present application adopts a shared mapping matrix to represent to and to ​transformations.

[0088] In step 112, according to the three-module iterative restoration in the hyperspectral image super-resolution model, until the number of iterations meets the preset number, the intermediate high-quality image calculated in the current iteration is the high-quality image after the target image is restored.

[0089] In each iteration, first, the solution process of the degenerate matrix solution subproblem is called, and the estimate value of the degenerate matrix is updated and optimized by combining the random prior guidance and the data fidelity loss function of the neural network. Then, the just updated degenerate matrix and the pre-trained diffusion model are used to solve the high-quality image solution subproblem, and the new intermediate high-quality image is optimized and obtained through the joint constraint of the data fidelity loss function and the total variation regularization. This process is alternated between the two, and each iteration can further improve the quality and accuracy of image restoration based on the results of the previous iteration.

[0090] Specifically, the band selection and Gaussian dimension reduction module selects the spectral band with the most information from the posterior distribution by sampling , and uses formulas (6)-(9) to obtain the HSI of the band , ensuring that subsequent calculations are performed on the band , avoiding full-band calculation. The sampling prior and degenerate kernel estimation module obtains the kernel prior by randomly sampling from the Gaussian distribution, and the lightweight kernel estimator estimates the kernel in an unsupervised manner by combining the kernel prior and the data consistency term. In the band projection and diffusion guided image super module, the data consistency term and the total variation regularization term are used to guide the existing diffusion model to restore the high-resolution HSI. By alternately performing band selection, kernel estimation and HSI restoration, the HSI of the band is obtained. Finally, a lightweight network is learned to generate a mapping matrix , which projects to the full-band HSI . It is worth noting that the entire DASR algorithm runs in an unsupervised manner without the need for prior training, and is applied in a plug-and-play manner, significantly improving flexibility and generalization ability.

[0091] In the hyperspectral image super-resolution method of the alternating diffusion band selection and degradation adaptation, the target function of the blind image restoration problem is decomposed into two sub-problems of solving the high-quality image and the degradation matrix by bringing the target image into the target function, and the two sub-problems are solved by using an alternating iterative solution, in each iteration, the solving process of the degradation matrix solving sub-problem is called first, the estimate value of the degradation matrix is updated and optimized by combining the random prior guidance and the data fidelity loss function of the neural network. Then, the high-quality image solving sub-problem is solved by using the just updated degradation matrix and the pre-trained diffusion model, and the new intermediate high-quality image is obtained by optimizing and constraining the data fidelity loss function and the total variation regularization. The process is alternated between the two, and the quality and accuracy of image restoration are further improved based on the previous result in each iteration.

[0092] In one embodiment, the target function of the blind image restoration problem is:

[0093] ;

[0094] wherein, denotes the degraded image, denotes the high-quality image, i.e., the image after image restoration, denotes the degradation matrix, and denotes the high-quality image and the prior term of the degradation matrix , and denote the corresponding weights.

[0095] In one embodiment, the target function is decomposed into two sub-problems of solving the high-quality image and the degradation matrix, including:

[0096] The target function is decomposed into two sub-problems of solving the high-quality image and the degradation matrix:

[0097] ;

[0098] wherein, denotes the degraded image, denotes the high-quality image, i.e., the image after image restoration, denotes the degradation matrix, and denotes the high-quality image and the prior term of the degradation matrix , and denote the corresponding weights.

[0099] In another embodiment, when solving the degenerate matrix solving sub-problem by using a neural network, the optimization problem of the degenerate matrix is converted into a parameter optimization problem of the neural network, and the converted degenerate matrix solving sub-problem is expressed as:

[0100] ;

[0101] In the above formula, and are the data fidelity loss functions of the blur kernel and the image, respectively, for measuring the difference between the estimated degenerate matrix and the actual degeneration; is a random prior loss function, is a TV regularization loss function, based on the prior knowledge of the degenerate matrix, to help the network learn a more accurate degeneration model; and are hyperparameters for balancing the weights between the two; is the parameter of the neural network ; is the input fixed noise; is the estimated image output by the t-th step diffusion model.

[0102] In one embodiment, a waveband selection and Gaussian dimension reduction module is used to dynamically select wavebands with large amounts of information through variance distribution and Gaussian decay model, including:

[0103] Given the observed degenerated image , a subset of wavebands containing wavebands is selected from wavebands, which can be expressed as a maximum a posteriori problem, where denotes the image height, denotes the image width;

[0104] Based on the efficient approximation method of sampling, the maximum a posteriori problem is approximately solved, and according to Bayes' theorem, the posterior distribution is sampled by the ratio of the likelihood term and the prior term , the likelihood term is approximated as the product of the variances of the selected wavebands, and the prior term is designed by using a Gaussian function to reduce the probability of the wavebands near the previously selected wavebands;

[0105] According to the prior term and the likelihood term, the solution of the approximate maximum a posteriori problem is obtained by repeating the sampling process times;

[0106] From the selected wavebands , a hyperspectral image with channels is selected for subsequent kernel estimation and hyperspectral image restoration.

[0107] In one of the embodiments, the likelihood term is approximated as the product of the selected band variance, including:

[0108] The likelihood term is approximated as the product of the selected band variance is:

[0109] ;

[0110] wherein, is the variance of the band , and denotes the selected band.

[0111] In one of the embodiments, the prior term is designed by using a Gaussian function to reduce the probability of the bands near the previously selected band, including:

[0112] The prior term is designed by using a Gaussian function to reduce the probability of the bands near the previously selected band is:

[0113] ;

[0114] wherein, denotes the selected band, denotes the band serial number, and denote different bands.

[0115] In one of the embodiments, the sampling prior and the degradation kernel estimation module are used to generate a random degradation matrix by Monte Carlo sampling, and the dynamic estimation of the degradation matrix is realized by combining the data fidelity loss function to optimize the neural network parameters, further including:

[0116] The kernel prior is randomly sampled from the Gaussian distribution, that is, a random degradation matrix, which is used as a label to guide the learning of the neural network;

[0117] The data fidelity loss function and the loss function of the random prior term are designed by comparing the predicted degradation matrix output by the neural network with the label;

[0118] The network parameters of the lightweight network are optimized by using the data fidelity loss function and the loss function of the random prior term, and the kernel, that is, the degradation matrix, is estimated according to the optimized lightweight network.

[0119] In one of the embodiments, the network parameters of the lightweight network are optimized by using the data fidelity loss function and the loss function of the random prior term, including:

[0120] Given the kernel randomly sampled by Monte Carlo simulation, the optimization of the network parameters is represented as:

[0121] ;

[0122] wherein,

[0123] ;

[0124] ;

[0125] a data fidelity loss function representing network parameters, a loss function representing a random prior term, represents a high-quality image, i.e. an image recovered after image restoration, represents a light-weight network, represents a fixed random noise, represents a degraded image, represents a corresponding blur kernel, represents a weight of the th sampling.

[0126] In one embodiment, a band projection and diffusion guided image super-resolution module is employed to solve a high-quality image by means of a pre-trained diffusion model, band selection and degradation matrix, and to optimize an intermediate high-quality image by a data fidelity loss function and total variation regularization, including:

[0127] A pre-trained diffusion model and an intermediate degradation matrix are utilized to solve a high-quality image, gradually recovering details and structures of an image from noise, represented as:

[0128] ;

[0129] wherein, represents a high-quality image estimated at the th step, is a hyperparameter;

[0130] A data fidelity loss function and total variation regularization of a high-quality image are calculated, and an intermediate high-quality image is optimized by the data fidelity loss function and total variation regularization.

[0131] In one embodiment, a data fidelity loss function and total variation regularization of a high-quality image are calculated, and an intermediate high-quality image is optimized by the data fidelity loss function and total variation regularization, including:

[0132] Given an estimated , an observed degraded image and an estimated kernel K, a data fidelity loss function of the

[0133] ;

[0134] The total variation regularization is:

[0135] ;

[0136] The prediction of high-quality image is obtained by solving the following optimization problem:

[0137] ;

[0138] The network-based full-band hyperspectral image projection adopts a lightweight network to learn the mapping between and , denoted as:

[0139] ;

[0140] wherein the network input is a fixed random noise;

[0141] The parameters are optimized by minimizing the data consistency loss formula:

[0142] ;

[0143] wherein, ;

[0144] The optimized is used to generate the mapping matrix ;

[0145] The obtained is multiplied by the mapping matrix to obtain the full-band high-quality image :

[0146] .

[0147] In one embodiment, the pseudo code of the blind hyperspectral image super-resolution method based on the above-mentioned alternating diffusion-based dynamic band selection and degradation adaptation is given as follows:

[0148] Input: low-quality image , pre-trained diffusion model ;

[0149] Initialization: ;

[0150] for to0do;

[0151] % DSB module; ​

[0152] Calculate the likelihood term according to formulas (7) and (8). ;

[0153] Calculate the prior terms according to formula (9) ;

[0154] According to formula (6), from the posterior distribution Mid-sampling ;

[0155] according to ,get ;

[0156] % DSK module;

[0157] calculate ;

[0158] Sampling using the Monte Carlo method ;

[0159] calculate ;

[0160] ;

[0161] % DHR module;

[0162] ;

[0163] ;

[0164] ;

[0165] ;

[0166] ;

[0167] ;

[0168] Output: Estimated high-resolution HSI Fuzzy kernel .

[0169] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1At least one of the steps in the above embodiments can include a plurality of sub-steps or a plurality of stages, which are not necessarily executed at the same time, but can be executed at different times, and the order of execution of the sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or at least a part of the sub-steps or stages of other steps.

[0170] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered to be within the scope of the present disclosure.

[0171] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the present application. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these are within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A hyperspectral image super-resolution method with alternating diffusion band selection and degradation adaptation, characterized in that, The method includes: Obtain the target image to be restored; construct the objective function for the blind image restoration problem of the target image; decompose the objective function into two sub-problems: solving for the high-quality image and the degradation matrix; A hyperspectral image super-resolution model is constructed; the hyperspectral image super-resolution model includes a band selection and Gaussian dimensionality reduction module, a sampling prior and degradation kernel estimation module, and a frequency band projection and diffusion-guided image super-resolution module; A band selection and Gaussian dimensionality reduction module is used to dynamically select bands with high information content through variance distribution and Gaussian attenuation model; A sampling prior and degradation kernel estimation module is adopted. Monte Carlo sampling is used to generate a random degradation matrix. The neural network parameters are optimized by combining the data fidelity loss function to achieve dynamic estimation of the degradation matrix. A frequency band projection and diffusion-guided image super-resolution module is adopted. High-quality images are solved by using a pre-trained diffusion model, band selection and degradation matrix. Intermediate high-quality images are optimized by data fidelity loss function and total variation regularization. The high-resolution model of hyperspectral images is based on the three modules of iterative recovery until the number of iterations meets the preset number. The intermediate high-quality image obtained in the current iteration is the high-quality image after the target image is recovered.

2. The method according to claim 1, characterized in that, The objective function for the blind image restoration problem is: in, Indicates a degraded image. This refers to a high-quality image, that is, an image after restoration. Represents the degeneracy matrix, and Indicates high-quality image and degeneracy matrix Priors, and This indicates its corresponding weight.

3. The method according to claim 1, characterized in that, The objective function is decomposed into two sub-problems: solving for the high-quality image and the degradation matrix. The objective function is decomposed into two subproblems: solving for the high-quality image and the degradation matrix. in, Indicates a degraded image. This refers to a high-quality image, that is, an image after restoration. Represents the degeneracy matrix, and Indicates high-quality image and degeneracy matrix Priors, and This indicates its corresponding weight.

4. The method according to claim 1, characterized in that, A band selection and Gaussian dimensionality reduction module is used to dynamically select bands with high information content through variance distribution and Gaussian attenuation models, including: Given the observed degraded image Later from Select one band from the available bands. The subset representation of each band is a maximum a posteriori probability problem, where, Indicates the image height. Indicates the image width; Efficient approximation methods based on sampling are used to approximate the maximum a posteriori probability problem. According to Bayes' theorem, the posterior distribution... Through the likelihood term and prior terms The proportional relationship is sampled, the likelihood term is approximated as the product of the variances of the selected bands, and the Gaussian function is used to reduce the probability of bands near the previously selected bands to design the prior term; Based on the prior terms and likelihood terms, through a repeated sampling process This will allow us to obtain an approximate solution to the maximum a posteriori probability problem. ; From the selected band Select hyperspectral images of the channel It is used for subsequent kernel estimation and hyperspectral image restoration.

5. The method according to claim 4, characterized in that, The likelihood term is approximated as the product of the variances of the selected bands, including: The likelihood term is approximated as the product of the variances of the selected bands as follows: in, It is a band variance Indicates the selected band.

6. The method according to claim 1, characterized in that, A Gaussian function is used to reduce the probability of bands near the previously selected band in the design of prior terms, including: The prior term is designed by using a Gaussian function to reduce the probability of bands near the previously selected band: in, Indicates the selected band. Indicates the band number. and Indicates different frequency bands.

7. The method according to claim 1, characterized in that, The system employs a sampling prior and degradation kernel estimation module, utilizing Monte Carlo sampling to generate a random degradation matrix. It then combines this with a data-fidelity loss function to optimize neural network parameters, achieving dynamic estimation of the degradation matrix. Other features include: The kernel prior, i.e., the random degeneracy matrix, is obtained by randomly sampling from a Gaussian distribution. This random degeneracy matrix is ​​used as a label to guide the learning of the neural network. The predicted degradation matrix output by the neural network is compared with the label to design a data fidelity loss function and a loss function for random prior terms; The network parameters of the lightweight network are optimized using the data fidelity loss function and the loss function of the random prior term. The kernel, i.e. the degradation matrix, is estimated based on the optimized lightweight network.

8. The method according to claim 7, characterized in that, The network parameters of lightweight networks are optimized using a data fidelity loss function and a loss function with random prior terms, including: Given a kernel randomly sampled by Monte Carlo simulation Network parameters The optimization is expressed as: in, The data fidelity loss function represents the network parameters. The loss function represents the random prior term. This refers to a high-quality image, that is, an image after restoration. Indicates a lightweight network. Indicates fixed random noise. Indicates a degraded image. Indicates the corresponding fuzzy kernel. Indicates the first Weights for each sample.

9. The method according to claim 1, characterized in that, A frequency band projection and diffusion-guided image super-resolution module is employed. High-quality images are obtained using a pre-trained diffusion model, band selection, and degradation matrix. Intermediate high-quality images are optimized through a data fidelity loss function and total variation regularization, including: Using a pre-trained diffusion model and intermediate degradation matrix to solve for high-quality images, the details and structure of the image are gradually recovered from the noise, as shown below: in, Indicates the first High-quality images estimated step by step, It's a hyperparameter; Calculate the data fidelity loss function and total variation regularization for high-quality images, and optimize intermediate high-quality images using the data fidelity loss function and total variation regularization.

10. The method according to claim 9, characterized in that, Calculate the data fidelity loss function and total variation regularization for high-quality images, and optimize intermediate high-quality images using the data fidelity loss function and total variation regularization, including: Given an estimate Degraded images observed And the estimated kernel K, The data fidelity loss function is expressed as: The total variation is regularized as follows: Combining data consistency terms and TV regularization terms, high-quality images The prediction is obtained by solving the following optimization problem: Network-based full-band hyperspectral image projection employs a lightweight network. study and Mapping between , represented as: ； Among them, network input It is fixed random noise; parameter Optimize by minimizing the formula for data consistency loss: in, ; Optimized Used to generate mapping matrix ; Obtained With mapping matrix Multiplying yields a high-quality image across the entire band. : 。

Images