Depth expansion spatial spectrum sparse memory hyperspectral sharpening fidelity method and system

Abstract

The invention relates to the technical field of hyperspectral image processing, and provides a deep-expanded spatial spectrum sparse memory hyperspectral sharpening fidelity method and system, and the method comprises the steps: carrying out the priori knowledge coding of a high-resolution hyperspectral image through a regularization technology, solving a target function of a hyperspectral panchromatic sharpening task through a semi-quadratic splitting method, and carrying out the processing of a hyperspectral image. A residual module is adopted to simulate a degradation operator, spectral fidelity and spatial fidelity sub-problems are solved through gradient descent iteration, and a sparse prior sub-problem is solved based on an iterative shrinkage threshold algorithm; the high-fidelity and high-resolution hyperspectral images are corrected through the spatial spectrum prior features, and the multi-stage reconstructed high-resolution hyperspectral images are fused through the cross-stage memory fusion network to obtain the high-resolution hyperspectral images. According to the method, the reconstruction precision of the image is improved, the calculation complexity is reduced, the extraction of the spatial spectrum combined prior features of the original image is realized, and lossless information transmission is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image processing, and in particular to a method and system for hyperspectral sharpening and fidelity with deep unfolding of spatial spectral sparse memory. Background Art

[0002] Hyperspectral remote sensing imaging technology captures surface feature information through continuous narrow-band spectral sampling. Due to the physical limitations of sensors, this technology faces an inherent trade-off challenge between spatial resolution and spectral resolution: to achieve nanoscale spectral resolution, the spatial sampling interval needs to be increased accordingly, resulting in a significant phenomenon of mixed spectra within a single pixel. This sub-pixel level spectral mixing effect severely restricts the accuracy of target detection and classification based on spectral features. Therefore, multi-source remote sensing data fusion technology, especially the sharpening fusion method of hyperspectral images (HSIs) and panchromatic images (PANs), has become a key technical path to improve spatial resolution.

[0003] Existing research has significant theoretical limitations in complex spectral-spatial feature modeling. Traditional linear decomposition models are difficult to effectively represent the non-linear correlation structure between multi-spectral channels, resulting in spectral information loss; although the end-to-end method based on convolutional neural networks (CNNs) can improve the quality of spatial detail reconstruction, its single-branch network architecture fails to fully model the cross-modal feature correlation of HSI-PAN, resulting in a decrease in spectral fidelity. More critically, existing methods have insufficient generalization ability for multi-sensor and multi-scene data, severely restricting practical engineering applications.

[0004] Existing technologies still have significant deficiencies in spectral-spatial information mining, multi-dimensional feature expression, cross-scene adaptability, and computational efficiency. These problems directly restrict the popularization and performance improvement of hyperspectral image fusion super-resolution technology in practical applications. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the related technologies. Therefore, the present invention provides a method and system for hyperspectral sharpening and fidelity with deep unfolding of spatial spectral sparse memory, optimizing the deficiency of the existing technology in the utilization of spectral and spatial original prior feature information in the super-resolution fusion of remote sensing hyperspectral images, and aiming at the disadvantage that the network model is difficult to interpret. By deeply studying the correlation between the spectral and panchromatic sharpened image spatial information inside hyperspectral images, the spectral-spatial optimization reconstruction problem is decoupled, and by using the prior features in the original images, the network model becomes more interpretable; a fusion model with anti-noise and information loss characteristics is constructed to effectively cope with the interference caused by multi-source data differences; technical support is provided for the analysis and practical application of hyperspectral images.

[0006] The present invention provides a method for hyperspectral sharpening and fidelity with deep unfolding of spatial spectral sparse memory, including: S1: Encode the prior knowledge of the high - resolution hyperspectral image through regularization techniques to obtain the objective function of the hyperspectral pan - sharpening task; S2: Solve the objective function of the hyperspectral pan - sharpening task through the semi - quadratic splitting method, decouple the problem of reconstructing the high - resolution hyperspectral image from the panchromatic image and the low - spatial - resolution hyperspectral image into spectral fidelity, spatial fidelity sub - problems, and sparse prior sub - problems; S3: Use a residual module to simulate the degradation operator and iteratively solve the spectral fidelity and spatial fidelity sub - problems through the gradient descent algorithm to obtain a high - fidelity high - resolution hyperspectral image; S4: Solve the sparse prior sub - problem through the iterative shrinkage threshold algorithm to obtain the spatial - spectral prior features; S5: Correct the high - fidelity high - resolution hyperspectral image through the spatial - spectral prior features to obtain the first - stage reconstructed high - resolution hyperspectral image; S6: Repeat steps S1 to S5 to obtain a multi - stage reconstructed high - resolution hyperspectral image; S7: Fuse the multi - stage reconstructed high - resolution hyperspectral images through a cross - stage memory fusion network to obtain a high - resolution hyperspectral image.

[0007] Furthermore, in step S1, the objective function of the hyperspectral pan - sharpening task is: where, is the high - resolution hyperspectral image with bands and pixels, , is the image set with dimension , is the observed low - resolution hyperspectral image with bands and pixels, , is the image set with dimension , is the spatial response operator, , is the matrix with dimension , is the observed panchromatic image with 1 band and pixels, , is the image set with dimension , is the spectral response operator, , is the matrix with dimension , is the trade - off parameter, is the prior information of the high-resolution hyperspectral image, is the Frobenius matrix norm, is the value that makes the function obtain the minimum value. value.

[0008] Furthermore, in step S2, the calculation expressions of the spectral fidelity and spatial fidelity sub-problems are: where, is the spectral fidelity and spatial fidelity sub-problem, is the auxiliary variable, is the penalty parameter; The calculation expression of the sparse prior sub-problem is: where, is the sparse prior sub-problem.

[0009] Furthermore, in step S3, expand the Frobenius matrix norm of the spectral fidelity and spatial fidelity sub-problems and apply the gradient descent algorithm to reconstruct the high-resolution hyperspectral image; Adopt a residual module to simulate the degradation operator to iteratively update the spectral difference between the low-resolution hyperspectral image and the reconstructed high-resolution hyperspectral image and the spatial difference between the panchromatic image and the reconstructed high-resolution hyperspectral image, and obtain a high-fidelity high-resolution hyperspectral image.

[0010] Furthermore, in step S3, the degradation operator includes the spatial response operator matrix and the transposed spatial response operator matrix in the spatial degradation process, and the spectral response operator matrix and the transposed spectral response operator matrix in the spectral degradation process.

[0011] Furthermore, step S4 includes: S411: Decompose the panchromatic image into multiple image patches through a sliding window; there are an over-complete dictionary and sparse coefficients in the image patches; S412: Obtain the spatial optimization function by learning the spatial mapping relationship covered by the over-complete dictionary for the image; S413: Generate an initial over-complete dictionary through the Xavier initialization strategy; iteratively optimize the initial over-complete dictionary through gradient descent to obtain the optimal spatial dictionary; S414: Iteratively solve the spatial optimization function through the iterative shrinkage threshold algorithm to obtain the optimal spatial sparse coefficients; S415: Reconstruct the prior features of the image patches by linearly combining the optimal spatial dictionary and the optimal spatial sparse coefficients, and integrate the prior features of the image patches to obtain the spatial prior features.

[0012] Furthermore, step S4 further includes: S421: Reshape the dimensions of the low-resolution hyperspectral image; S422: Obtain a spectral optimization function by learning the spectral mapping relationship of the image covered by the spectral dictionary; S423: Iteratively solve the spectral optimization function based on the iterative shrinkage threshold algorithm to obtain the optimal spectral dictionary and the optimal spectral sparse coefficients; S424: Obtain spectral prior features by linearly combining the optimal spectral dictionary and the optimal spectral sparse coefficients.

[0013] Furthermore, integrate the spatial prior features and the spectral prior features to obtain spatial-spectral prior features.

[0014] Furthermore, in step S7, the cross-stage memory fusion network adds the features of the previous stage embedded into the features of the current stage element by element.

[0015] The present invention also provides a deep unfolding spatial-spectral sparse memory hyperspectral sharpening and fidelity system for performing the above-mentioned deep unfolding spatial-spectral sparse memory hyperspectral sharpening and fidelity method, including: A construction module, which encodes prior knowledge of the high-resolution hyperspectral image through a regularization technique to obtain an objective function for the hyperspectral pan-sharpening task; A decoupling module, which solves the objective function of the hyperspectral pan-sharpening task through the semi-quadratic splitting method, and decouples the problem of reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image into a spectral fidelity and a spatial fidelity sub-problem and a sparse prior sub-problem; A first reconstruction module, which uses a residual module to simulate a degradation operator and an iterative gradient descent algorithm to solve the spectral fidelity and spatial fidelity sub-problems to obtain a high-fidelity high-resolution hyperspectral image; A second reconstruction module, which solves the sparse prior sub-problem through an iterative shrinkage threshold algorithm to obtain spatial-spectral prior features; A correction module, which corrects the high-fidelity high-resolution hyperspectral image through the spatial-spectral prior features to obtain a first-stage reconstructed high-resolution hyperspectral image; A cross-stage fusion module, which fuses the multi-stage reconstructed high-resolution hyperspectral images through a cross-stage memory fusion network to obtain a high-resolution hyperspectral image.

[0016] One or more of the above technical solutions in the embodiments of the present invention have at least one of the following technical effects: To address the insufficient utilization of spectral and spatial original prior feature information in existing technologies for hyperspectral image super-resolution fusion and the drawback of the network model being difficult to interpret, by deeply studying the correlation between the spectral information of hyperspectral images and the spatial information of panchromatic sharpened images, the spectral-spatial optimization and reconstruction problem is decoupled, and by leveraging the prior features in the original images, the network model becomes more interpretable; by constructing a fusion model with anti-noise and information-loss characteristics, it effectively copes with the interference caused by multi-source data differences; it provides technical support for the analysis and practical applications of hyperspectral images. The present invention improves the reconstruction accuracy of the image, reduces the computational complexity, realizes the extraction of the joint prior features of the spatial spectrum of the original image, and realizes lossless information transmission.

[0017] Additional aspects and advantages of the present invention will be given in part in the following description, will become apparent in part from the following description, or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the present invention or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a schematic flowchart of a depth-unrolled spatial-spectral sparse memory hyperspectral sharpening and fidelity method provided by the present invention.

[0020] Figure 2 It is a schematic diagram of the network structure of a depth-unrolled spatial-spectral sparse memory hyperspectral sharpening and fidelity method provided by the present invention.

[0021] Figure 3 It is a schematic diagram of the structure of a depth-unrolled spatial-spectral sparse memory hyperspectral sharpening and fidelity system provided by the present invention.

[0022] Reference Signs: 101, construction module; 102, decoupling module; 103, first reconstruction module; 104, second reconstruction module; 105, correction module; 106, cross-stage fusion module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention. The following embodiments are used to illustrate the present invention but cannot be used to limit the scope of the present invention.

[0024] In the description of the embodiments of the present invention, it should be noted that the terms "first", "second", and "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. In the description of this specification, the description referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0025] Next, in conjunction with Figures 1 to 3 a method and system for high - spectral sharpening and fidelity with deep - unfolded spatial - spectral sparse memory will be described for the present invention.

[0026] As Figure 1 shown, a method for high - spectral sharpening and fidelity with deep - unfolded spatial - spectral sparse memory includes: S1: Encoding prior knowledge of a high - resolution hyperspectral image through a regularization technique to obtain an objective function for the hyperspectral pan - sharpening task; For the hyperspectral pan - sharpening task, its observation model can be expressed as follows: where, is a high - resolution hyperspectral image with bands and pixels, , is an image set with a dimension of , is the observed low - resolution hyperspectral image with bands and pixels, , is an image set with a dimension of , is a panchromatic image with 1 band and pixels, , is an image set with a dimension of , is a spatial response operator, , is a matrix with a dimension of , is a spectral response operator, , is a matrix with a dimension of , is the noise existing in the low-resolution hyperspectral image during the observation process, is the noise existing in the panchromatic image during the observation process.

[0027] It can be seen from the above observation model that obtaining a high-resolution hyperspectral image from a low-resolution hyperspectral image and a panchromatic image is an ill-posed inverse problem. Therefore, regularization techniques are usually used to encode prior knowledge for the desired high-resolution hyperspectral image. The calculation expression of the objective function for the hyperspectral panchromatic sharpening task is: where, is the spectral fidelity term, is the spatial fidelity term, is the regularization term, is the trade-off parameter, is the prior feature of the high-resolution hyperspectral image, is the Frobenius matrix norm, is the value that makes the function reach the minimum.

[0028] S2: Solve the objective function of the hyperspectral panchromatic sharpening task by the Half-Quadratic Splitting (HQS) method, decoupling the problem of reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image into a spectral fidelity and a spatial fidelity sub-problem and a sparse prior sub-problem; As Figure 2 shown, when reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image in the present invention, upsampling and convolution operations are performed on the low-spatial-resolution hyperspectral image, and convolution operations are performed on the panchromatic image.

[0029] The Half-Quadratic Splitting method is used to solve the objective function of the hyperspectral panchromatic sharpening task, and auxiliary variables are introduced to decompose the objective function of the hyperspectral panchromatic sharpening task: Among them, is a constraint, is an auxiliary variable; Transform the optimization problem with equality constraints into an unconstrained optimization problem, and the calculation expression is: Among them, is the unconstrained optimization objective function, is the penalty parameter, is the prior information of, is the and values that make the function obtain the minimum value.

[0030] Decompose the unconstrained optimization problem into two sub-problems for iterative solution, decouple the fidelity term and the regularization term into two separate sub-problems, and the calculation expressions for the spectral fidelity and spatial fidelity sub-problems are: Among them, is the spectral fidelity and spatial fidelity sub-problem; The calculation expression for the sparse prior sub-problem is: Among them, is the sparse prior sub-problem, is the value that makes the function obtain the minimum value.

[0031] S3: Use the residual module to simulate the degradation operator and the gradient descent algorithm to iteratively solve the spectral fidelity and spatial fidelity sub-problems to obtain a high-fidelity high-resolution hyperspectral image; Expand the Frobenius matrix norm of the spectral fidelity and spatial fidelity sub-problems and apply the gradient descent algorithm to reconstruct the high-resolution hyperspectral image; By expanding the Frobenius matrix norm and applying the gradient descent algorithm to solve the gradient of, the calculation expression is: Among them, is the gradient of, is the th iteration of the high-resolution hyperspectral image, is the th iteration of the auxiliary variable, is the transpose of the matrix; The residual module is used to simulate the spectral differences between the low-resolution hyperspectral image and the reconstructed high-resolution hyperspectral image and the spatial differences between the panchromatic image and the reconstructed high-resolution hyperspectral image during the iterative update of the degradation operator, so as to obtain a high-fidelity high-resolution hyperspectral image; The degradation operator includes the spatial response operator matrix and its transpose matrix during the spatial degradation process, and the spectral response operator matrix and its transpose matrix during the spectral degradation process; Since 、 、 、 are all unknown terms, the residual network is used in the present invention to simulate the degradation operator, The calculation expression for the update is: where is the high-resolution hyperspectral image at the -th iteration, is the first step size; is the spectral difference between the low-resolution hyperspectral image and the -th iteration reconstruction result, is the spatial difference between the panchromatic image and the -th iteration reconstruction result. After multiple iterations, a high-fidelity high-resolution hyperspectral image is reconstructed.

[0032] S4: Solve the sparse prior sub-problem through the iterative shrinkage threshold algorithm to obtain the spatial-spectral prior features; The core of sparse dictionary learning lies in representing signals through the linear combination of a small number of atoms. Its effectiveness depends on the balance between the completeness of the dictionary and the sparse constraint. Traditional methods (such as K-SVD) rely on the training of fixed input data. The present invention adopts an end-to-end learnable dictionary structure, enabling the dictionary to adaptively train the data distribution.

[0033] The panchromatic image has high spatial resolution, and its spatial detail features (such as edges and textures) can be regarded as a kind of prior knowledge for improving the spatial resolution of the hyperspectral image. The sparse representation theory provides a mathematical basis for prior modeling. Its core theory holds that the linear representation of local image patches of an image under appropriate basis functions is usually sparse. Assume that the image patch , , is a -dimensional set of image blocks, there exists an over-complete dictionary and sparse coefficients , , is a -dimensional set of dictionaries, , For -dimensional sparse coefficients, such that: Wherein, is the number of non-zero elements, and the image structure can be efficiently represented by a linear combination of a small number of atoms.

[0034] When the dimension of the image patch is , the number of atoms in the dictionary satisfies , the learned over-complete dictionary can cover the spatial mapping relationship of the image, and the following optimization function is adopted: Wherein, is the over-complete dictionary, , is the dimension of the image patch, is the sparse coefficient of the -th image patch, is the balance parameter, is the function to take the minimum value, is the -th image patch, is the Manhattan norm, is the square of the Euclidean norm.

[0035] Obtaining spatial prior features includes: S411: Decompose the panchromatic image into multiple image patches through a sliding window; there are an over-complete dictionary and sparse coefficients for the image patches; For the input panchromatic image , through a sliding window, where is the height of the image, is the width of the image, and the image is decomposed into many image patches , , is the -th image patch, is the number of image patches; S412: Obtain a spatial optimization function by learning the over-complete dictionary to cover the spatial mapping relationship of the image; the calculation expression is: .

[0036] Generate an initial over-complete dictionary through the Xavier initialization strategy; iteratively optimize the initial over-complete dictionary through gradient descent to obtain the optimal spatial dictionary; The initial over-complete dictionary satisfies: Wherein, is the initial over-complete dictionary for the th image patch; During the iteration process, the over-complete dictionary is updated by the following formula: where is the over-complete dictionary for the th image patch at the th iteration, is the second step size, is the gradient of the over-complete dictionary, is the over-complete dictionary for the th image patch at the th iteration, is the sparse coefficient matrix for the th image patch at the th iteration, is the th iteration of the th image patch.

[0037] S414: Obtain the optimal spatial sparse coefficients by iteratively solving the spatial optimization function based on the iterative shrinkage threshold algorithm; the calculation expression is: where is the sparse coefficient matrix for the th iteration, is the soft threshold function, , is the sign function, is the threshold parameter of the soft threshold function, is the maximum value function.

[0038] S415: Reconstruct the prior features of the image patch by linearly combining the optimal spatial dictionary and the optimal spatial sparse coefficients, and integrate the prior features of the image patch to obtain the spatial prior features; The prior features of the image patch are reconstructed by linearly combining the optimal spatial dictionary and the optimal spatial sparse coefficients , and finally the prior features of the image patch are integrated back into the complete image prior features , and the calculation expression is: where is the operation of integrating the image patches.

[0039] Hyperspectral images have characteristics such as high spectral resolution and strong correlation between spectral bands. Traditional methods are difficult to directly model such complex mapping relationships. Directly processing hyperspectral images often faces problems such as high computational complexity, easy propagation of sensor noise in the original spectral space, and high correlation between adjacent bands that requires compact representation. Sparse dictionary learning constructs adaptive dictionary atoms in a data-driven manner, providing an efficient solution for modeling spectral prior features. Its core assumption is that spectral information can be composed of a linear combination of a small number of dictionary atoms: where, is the spectral dictionary, , is the spectral dictionary set with dimension , is the spectral sparse coefficient, , is the spectral dictionary set with dimension .

[0040] Obtaining spectral prior features includes: S421: Reshape the dimension of the low-resolution hyperspectral image; Given the hyperspectral image , , reshape into , is the reshaped image, , S422: Obtain the spectral optimization function by learning the spectral mapping relationship of the image covered by the spectral dictionary; The calculation expression of the spectral optimization function is: where, is the balance parameter.

[0041] S423: Iteratively solve the spectral optimization function based on the iterative shrinkage threshold algorithm to obtain the optimal spectral dictionary and the optimal spectral sparse coefficient. The calculation expression is: where, is the spectral dictionary of the th iteration, is the spectral dictionary of the th iteration, is the gradient of the spectral dictionary, is the th iteration of the reshaped image, is the soft threshold function, is the The spectral sparse coefficient of the th iteration is .

[0042] S424: The spectral prior feature is obtained by linearly combining the optimal spectral dictionary and the optimal spectral sparse coefficient. The calculation expression is: where is the spectral prior feature, is the optimal spectral sparse coefficient, and is the optimal spectral dictionary.

[0043] Next, the prior features after two sparse codings are integrated to obtain the spatio-spectral prior feature. The calculation expression is: where is the spatio-spectral prior feature.

[0044] S5: The high-fidelity high-resolution hyperspectral image is corrected by the spatio-spectral prior feature to obtain the first-stage reconstructed high-resolution hyperspectral image; For the sub-problem , it is regarded as correcting the reconstructed preliminarily. The traditional method may cause excessive smoothing of details in complex texture regions. Sparse dictionary learning can generate more adaptable basis functions according to the edges, details, and spectral features in the data, so as to more accurately estimate the image content. In addition, the noise in the image data is suppressed due to the sparsity constraint. Therefore, sparse representation can significantly separate signals and noise. In the fusion reconstruction process, the upsampling operation of the LRHS will cause distortion of spectral information, and sparse dictionary learning can effectively restore spectral features.

[0045] S6: Repeat steps S1 to S5 to obtain the multi-stage reconstructed high-resolution hyperspectral image; The reconstructed image is continuously optimized through the multi-stage reconstructed high-resolution hyperspectral image, and the spatial resolution and spectral fidelity of the hyperspectral image are gradually improved. By fusing multi-stage features, a high-quality and high-resolution reconstructed image is finally output, making the processed image more accurate in terms of structure and spectral information.

[0046] S7: The multi-stage reconstructed high-resolution hyperspectral image is fused through the cross-stage memory fusion network to obtain the high-resolution hyperspectral image; In the deep unfolding network, it is often faced with the situation that the spectral-spatial information reconstructed in the previous stage is lost as the number of reconstruction stages increases. A cross-stage memory fusion network is designed in each stage to transmit lossless information to the next stage The cross-stage memory fusion network embeds the features of the previous stage into the features of the current stage and performs element-wise addition. The computational expression of the cross-stage fusion memory fusion network is as follows: where, is the high-resolution hyperspectral image of the th stage, is the reconstructed high-resolution hyperspectral image of the th stage, is the first learnable affine parameter, is the second learnable affine parameter; and are both obtained by convolutional processing of the convolutional features of the th stage, and are used to control the fusion ratio between the previous stage and the current stage. is the high-resolution hyperspectral image of the th stage obtained through convolution with a convolution kernel size of . The computational expression is as follows: where, is the convolution with a convolution kernel size of .

[0047] In the above fusion process, and are not vectors but tensors with the same dimension as the stage features. By simultaneously processing the refined memory features of the previous stage and the reconstructed features of the current stage, the spectral-spatial information is preserved intact, and proximal mapping enhancement is achieved.

[0048] In the present invention, the spectral information of the hyperspectral image and the high-resolution spatial information of the panchromatic image are used to obtain the prior information in the original image through sparse dictionary learning; then, fusion and reconstruction are performed through an interpretable deep unfolding network, and finally a fusion image with both hyperspectral fidelity and high spatial resolution is generated.

[0049] Solving spectral fidelity, solving spatial fidelity, solving sparse priors, and the cross-stage fusion network in the present invention constitute an interpretable deep unfolding network.

[0050] The present invention conducts quantitative index result comparison experiments and subjective comparison experiments with a variety of internationally advanced algorithms on the public dataset Pavia Center. To better observe the differences between the fusion results, the root mean square error (RMSE) between the fusion results and the ground truth (GT) is calculated and shown. The internationally advanced algorithms are FPFNet(2023), HyperRefiner(2023), TreeNet(2024), and DFCFN(2025). FPFNet(2023) is the 2023 version of the feature pyramid fusion network, HyperRefiner(2023) is the 2023 version of the refined hyperspectral panspectral network, TreeNet(2024) is the 2024 version of the tree-structured neural network, and DFCFN(2025) is the 2025 version of the two-stage feature correction fusion network. It can also be seen from the quantitative index results that the quantitative result indicators of the algorithm of the present invention are better than those of the internationally advanced algorithms in recent years. The experimental results are shown in Table 1.

[0051] Table 1 Comparative experiments on quantitative index results of Pavia center dataset Among them, SCC is the structure correlation coefficient, SAM is the spectral angle mapping, RMSE is the root mean square error, ERGAS is the dimensionless global relative error, PSNR is the peak signal-to-noise ratio, "↑" indicates that the higher the value, the better, and "↓" indicates that the lower the value, the better.

[0052] As Figure 3 shown, a deep unfolding spatial-spectral sparse memory hyperspectral sharpening and fidelity system is used to execute a deep unfolding spatial-spectral sparse memory hyperspectral sharpening and fidelity method, including: The construction module 101 encodes the prior knowledge of the high-resolution hyperspectral image through regularization technology to obtain the objective function of the hyperspectral panchromatic sharpening task; The decoupling module 102 solves the objective function of the hyperspectral panchromatic sharpening task through the semi-quadratic splitting method, decoupling the problem of reconstructing the high-resolution hyperspectral image from the panchromatic image and the low-spatial-resolution hyperspectral image into spectral fidelity, spatial fidelity sub-problems, and sparse prior sub-problems; The first reconstruction module 103 uses the residual module to simulate the degradation operator and the gradient descent algorithm to iteratively solve the spectral fidelity and spatial fidelity sub-problems to obtain a high-fidelity high-resolution hyperspectral image; The second reconstruction module 104 solves the sparse prior sub-problem through the iterative shrinkage threshold algorithm to obtain the spatial-spectral prior features; The correction module 105 corrects the high-fidelity high-resolution hyperspectral image through the spatial-spectral prior features to obtain the first-stage reconstructed high-resolution hyperspectral image; The cross-stage fusion module 106 fuses the multi-stage reconstructed high-resolution hyperspectral images through a cross-stage memory fusion network to obtain high-resolution hyperspectral images.

[0053] Through the collaborative work of the above-mentioned modules, the deficiencies in the utilization of spectral and spatial original prior feature information in the super-resolution fusion of remote sensing hyperspectral images in the prior art are optimized. Aiming at the shortcoming that the network model is difficult to interpret, by deeply studying the correlation between the spectral information of hyperspectral images and the spatial information of panchromatic sharpened images, the spectral-spatial optimization reconstruction problem is decoupled, and by utilizing the prior features in the original images, the network model becomes more interpretable; by constructing a fusion model with anti-noise and information loss characteristics, it effectively copes with the interference caused by the differences in multi-source data; it provides technical support for the analysis and practical application of hyperspectral images. The present invention improves the reconstruction accuracy of the images, reduces the computational complexity, realizes the extraction of the joint prior features of the spatial spectrum of the original images, and realizes lossless information transmission.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for hyperspectral sharpening fidelity with deep unfolded spatial spectral sparse memory, characterized in that Including: S1: Encoding the prior knowledge of the high - resolution hyperspectral image through regularization technology to obtain the objective function of the hyperspectral pan - sharpening task; S2: Solving the objective function of the hyperspectral pan - sharpening task by the semi - quadratic splitting method, decoupling the problem of reconstructing the high - resolution hyperspectral image from the panchromatic image and the low - spatial - resolution hyperspectral image into spectral fidelity, spatial fidelity sub - problems and sparse prior sub - problems; S3: Using a residual module to simulate the degradation operator and iteratively solving the spectral fidelity and spatial fidelity sub - problems through the gradient descent algorithm to obtain a high - fidelity high - resolution hyperspectral image; S4: Solving the sparse prior sub - problem through the iterative shrinkage threshold algorithm to obtain the spatial - spectral prior features; S5: Correcting the high - fidelity high - resolution hyperspectral image through the spatial - spectral prior features to obtain the first - stage reconstructed high - resolution hyperspectral image; S6: Repeating steps S1 to S5 to obtain the multi - stage reconstructed high - resolution hyperspectral image; S7: Fusing the multi - stage reconstructed high - resolution hyperspectral images through a cross - stage memory fusion network to obtain the high - resolution hyperspectral image.

2. A method for high - spectral sharpening and fidelity with deep - unfolded spatial - spectral sparse memory according to claim 1, characterized in that In step S1, the objective function of the hyperspectral pan - sharpening task is: Among them, is a high-resolution hyperspectral image with bands and pixels, , is an image set with a dimension of , is a low-resolution hyperspectral image with bands and pixels observed, , is an image set with a dimension of , is a spatial response operator, , is a matrix with a dimension of , is a panchromatic image with 1 band and pixels observed, , is an image set with a dimension of , is a spectral response operator, , is a matrix with a dimension of , is a trade-off parameter, is the prior information of the high-resolution hyperspectral image, is the Frobenius matrix norm, is the value that makes the function obtain the minimum value.

3. A method for hyperspectral sharpening and fidelity with deep-expanded spatial spectral sparse memory according to claim 2, characterized in that In step S2, the calculation expressions of the spectral fidelity and spatial fidelity sub - problems are: wherein, is the spectral fidelity and spatial fidelity sub-problem, is the auxiliary variable, is the penalty parameter; The calculation expression of the sparse prior sub - problem is: Among them, is a sparse prior sub-problem.

4. A method for hyperspectral sharpening and fidelity with deep unfolded spatial spectral sparse memory according to claim 1, characterized in that In step S3, Expanding the Frobenius matrix norm of the spectral fidelity and spatial fidelity sub - problems and applying the gradient descent algorithm to reconstruct the high - resolution hyperspectral image; Using a residual module to simulate the degradation operator to iteratively update the spectral difference between the low - resolution hyperspectral image and the reconstructed high - resolution hyperspectral image, and the spatial difference between the panchromatic image and the reconstructed high - resolution hyperspectral image to obtain a high - fidelity high - resolution hyperspectral image.

5. A method for high - spectral sharpness and fidelity of deep - unfolded spatial - spectral sparse memory, according to claim 4, wherein In step S3, the degradation operator includes the spatial response operator matrix and its transpose matrix in the spatial degradation process, and the spectral response operator matrix and its transpose matrix in the spectral degradation process.

6. A method for high - spectral sharpening and fidelity of deep - unfolded spatial - spectral sparse memory, according to claim 1, wherein Step S4 includes: S411: Decomposing the panchromatic image into multiple image patches through a sliding window; there are over - complete dictionaries and sparse coefficients for the image patches; S412: Obtaining the spatial optimization function by learning the spatial mapping relationship of the over - complete dictionary covering the image; S413: Generating an initial over - complete dictionary through the Xavier initialization strategy; iteratively optimizing the initial over - complete dictionary through the gradient descent to obtain the optimal spatial dictionary; S414: Iteratively solving the spatial optimization function through the iterative shrinkage threshold algorithm to obtain the optimal spatial sparse coefficients; S415: Reconstructing the prior features of the image patches by linearly combining the optimal spatial dictionary and the optimal spatial sparse coefficients, and integrating the prior features of the image patches to obtain the spatial prior features.

7. A method for high - spectral sharpening and fidelity of spatially - spectral sparse memory with deep unfolding according to claim 6, characterized in that, Step S4 also includes: S421: Reshaping the dimensions of the low - resolution hyperspectral image, S422: Obtaining the spectral optimization function by learning the spectral mapping relationship of the spectral dictionary covering the image; S423: Iteratively solving the spectral optimization function based on the iterative shrinkage threshold algorithm to obtain the optimal spectral dictionary and the optimal spectral sparse coefficients; S424: Obtain spectral prior features by linearly combining the optimal spectral dictionary and the optimal spectral sparse coefficients.

8. A method for high - spectral - sharpness fidelity with deep - unfolded spatial - spectral sparse memory according to claim 7, characterized in that, Integrate the spatial prior features and the spectral prior features. Obtain spatial-spectral prior features.

9. A method for hyperspectral sharpening and fidelity with deep - unfolded spatial - spectral sparse memory, according to claim 1, characterized in that In step S7, the cross-stage memory fusion network adds the features of the previous stage embedded into the features of the current stage element by element.

10. A deeply unfolded spatial spectral sparse memory hyperspectral sharpening fidelity system, characterized in that, For implementing a depth-unrolled spatial-spectral sparse memory hyperspectral sharpening fidelity method according to any one of claims 1 to 9, comprising: A construction module, which encodes prior knowledge of a high-resolution hyperspectral image through a regularization technique to obtain an objective function for the hyperspectral pan-sharpening task; A decoupling module, which solves the objective function of the hyperspectral pan-sharpening task through a semi-quadratic splitting method, decoupling the problem of reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image into a spectral fidelity and a spatial fidelity sub-problem and a sparse prior sub-problem; A first reconstruction module, which uses a residual module to simulate a degradation operator and an iterative gradient descent algorithm to solve the spectral fidelity and spatial fidelity sub-problems to obtain a high-fidelity high-resolution hyperspectral image; A second reconstruction module, which solves the sparse prior sub-problem through an iterative shrinkage threshold algorithm to obtain spatial-spectral prior features; A correction module, which corrects the high-fidelity high-resolution hyperspectral image through the spatial-spectral prior features to obtain a first-stage reconstructed high-resolution hyperspectral image; A cross-stage fusion module, which fuses multi-stage reconstructed high-resolution hyperspectral images through a cross-stage memory fusion network to obtain a high-resolution hyperspectral image.

Images