Deeply expanded spatial spectrum sparse memory hyperspectral sharpening fidelity preservation method and system

Abstract

The present invention relates to the technical field of hyperspectral image processing, and provides a method and system for deeply expanded spatial spectrum sparse memory hyperspectral sharpening fidelity preservation. The method includes encoding prior knowledge of high-resolution hyperspectral images through regularization technology, solving the objective function of the hyperspectral panchromatic sharpening task through a semi-quadratic splitting method, using a residual module to simulate a degradation operator, solving the spectral fidelity and spatial fidelity sub-problems through gradient descent iteration, and solving the sparse prior sub-problem through an iterative shrinkage threshold algorithm; correcting high-fidelity high-resolution hyperspectral images through spatial spectrum prior features, and fusing multi-stage reconstructed high-resolution hyperspectral images through a cross-stage memory fusion network to obtain a high-resolution hyperspectral image. The present invention improves the reconstruction accuracy of the image, reduces the computational complexity, realizes the extraction of the spatial spectrum joint prior features of the original image, and realizes lossless information transmission.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image processing, and in particular to a method and system for preserving the fidelity of a deeply expanded spatial spectrum sparse memory hyperspectral sharpening. Background Art

[0002] Hyperspectral remote sensing imaging captures surface features through continuous, narrow-band spectral sampling. However, due to the physical limitations of sensors, this technology faces an inherent trade-off between spatial and spectral resolution. To achieve nanometer-level spectral resolution, the spatial sampling interval must be increased, leading to significant spectral mixing within a single pixel. This sub-pixel spectral mixing severely limits the accuracy of spectral-based target detection and classification. Therefore, multi-source remote sensing data fusion, particularly the sharpening fusion of hyperspectral imagery (HSI) and panchromatic imagery (PAN), has become a key technology for improving spatial resolution.

[0003] Existing research faces significant theoretical limitations in modeling complex spectral-spatial features. Traditional linear decomposition models struggle to effectively represent the nonlinear correlation structure between multispectral channels, resulting in loss of spectral information. While end-to-end methods based on convolutional neural networks (CNNs) can improve the quality of spatial detail reconstruction, their single-branch network architecture fails to fully model the cross-modal feature correlations between HSI and PAN, leading to reduced spectral fidelity. More critically, existing methods lack generalizability to multi-sensor and multi-scenario data, severely restricting their practical engineering applications.

[0004] Existing technologies still have significant shortcomings in spectral-spatial information mining, multi-dimensional feature expression, cross-scene adaptability, and computational efficiency. These issues directly restrict the promotion and performance improvement of hyperspectral image fusion super-resolution technology in practical applications. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a deep-expanded spatial spectrum sparse memory hyperspectral sharpening fidelity method and system, which optimizes the shortcomings of the existing technology in the super-resolution fusion of remote sensing hyperspectral images in terms of the use of spectral and spatial original prior feature information, as well as the shortcomings of the network model that are difficult to interpret. By deeply studying the correlation between the hyperspectral image spectrum and the spatial information of the full-color sharpened image, the spectral-spatial optimization reconstruction problem is decoupled, and by utilizing the prior features in the original image, the network model is made more interpretable; a fusion model with anti-noise and information loss characteristics is constructed to effectively deal with the interference caused by differences in multi-source data; and technical support is provided for the analysis and practical application of hyperspectral images.

[0006] The present invention provides a method for sharpening and preserving the fidelity of a deep-expanded spatial spectrum sparse memory hyperspectral, comprising:

[0007] S1: The prior knowledge of high-resolution hyperspectral images is encoded by regularization techniques to obtain the objective function of the hyperspectral pan-sharpening task;

[0008] S2: Solve the objective function of the hyperspectral pan-sharpening task by using the semi-quadratic splitting method, decoupling the problem of reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image into spectral fidelity and spatial fidelity sub-problems and a sparse prior sub-problem;

[0009] S3: A residual module is used to simulate the degradation operator and the gradient descent algorithm is used to iteratively solve the spectral fidelity and spatial fidelity sub-problems to obtain high-fidelity and high-resolution hyperspectral images.

[0010] S4: Obtain spatial spectral prior features by solving the sparse prior subproblem based on the iterative shrinkage threshold algorithm;

[0011] S5: Modify the high-fidelity high-resolution hyperspectral image by using spatial spectral prior features to obtain the first-stage reconstructed high-resolution hyperspectral image;

[0012] S6: Repeat steps S1 to S5 to obtain a multi-stage reconstructed high-resolution hyperspectral image;

[0013] S7: The multi-stage reconstructed high-resolution hyperspectral images are fused through a cross-stage memory fusion network to obtain a high-resolution hyperspectral image.

[0014] Furthermore, in step S1, the objective function of the hyperspectral pan-sharpening task is:

[0015]

[0016] in, To have bands and high-resolution hyperspectral images with pixels, , The dimension is Image set, For the observed bands and pixels of low-resolution hyperspectral image, , The dimension is Image set, is the spatial response operator, , The dimension is The matrix, is the observed waveband with 1 band and pixels of full-color image, , The dimension is Image set, is the spectral response operator, , The dimension is The matrix, To balance the parameters, is the prior information of high-resolution hyperspectral images, is the Frobenius matrix norm, To minimize the function The value of .

[0017] Furthermore, in step S2, the computational expressions for the spectral fidelity and spatial fidelity subproblems are:

[0018]

[0019] in, are the spectral fidelity and spatial fidelity subproblems, is an auxiliary variable, is the penalty parameter;

[0020] The computational expression of the sparse prior subproblem is:

[0021]

[0022] in, is the sparse prior subproblem.

[0023] Furthermore, in step S3,

[0024] The Frobenius matrix norm expansion of the spectral fidelity and spatial fidelity subproblems is used to reconstruct high-resolution hyperspectral images using the gradient descent algorithm.

[0025] The residual module is used to simulate the degradation operator to iteratively update the spectral difference between the low-resolution hyperspectral image and the reconstructed high-resolution hyperspectral image, as well as the spatial difference between the panchromatic image and the reconstructed high-resolution hyperspectral image, to obtain a high-fidelity high-resolution hyperspectral image.

[0026] Furthermore, in step S3, the degradation operator includes a spatial response operator matrix and a spatial response operator transposed matrix in a spatial degradation process, and a spectral response operator matrix and a spectral response operator transposed matrix in a spectral degradation process.

[0027] Furthermore, step S4 includes:

[0028] S411: Decomposing the full-color image into a plurality of image blocks by a sliding window; the image blocks have an overcomplete dictionary and sparse coefficients;

[0029] S412: Obtaining a spatial optimization function by learning a spatial mapping relationship of an overcomplete dictionary covering the image;

[0030] S413: Generate an initialized overcomplete dictionary through the Xavier initialization strategy; and obtain an optimal spatial dictionary by iteratively optimizing the initialized overcomplete dictionary through gradient descent.

[0031] S414: Obtaining an optimal spatial sparse coefficient by iteratively solving a spatial optimization function based on an iterative shrinkage threshold algorithm;

[0032] S415: reconstructing the prior features of the image block by linearly combining the optimal spatial dictionary and the optimal spatial sparse coefficient, and integrating the prior features of the image block to obtain the spatial prior features.

[0033] Furthermore, step S4 further includes:

[0034] S421: Reshaping low-resolution hyperspectral images;

[0035] S422: Obtaining a spectral optimization function by learning a spectral mapping relationship of a spectral dictionary covering an image;

[0036] S423: Iteratively solving the spectral optimization function based on the iterative shrinkage threshold algorithm to obtain the optimal spectral dictionary and the optimal spectral sparse coefficient;

[0037] S424: Obtaining spectral prior features by linearly combining the optimal spectral dictionary and the optimal spectral sparse coefficient.

[0038] Furthermore, the spatial prior features and the spectral prior features are integrated to obtain the spatial spectral prior features.

[0039] Furthermore, in step S7, the cross-stage memory fusion network embeds the features of the previous stage into the features of the current stage and adds them element by element.

[0040] The present invention also provides a deep-expanded spatial spectrum sparse memory hyperspectral sharpening and fidelity preservation system for executing the above-mentioned deep-expanded spatial spectrum sparse memory hyperspectral sharpening and fidelity preservation method, comprising:

[0041] A construction module, wherein the construction module encodes prior knowledge of the high-resolution hyperspectral image through a regularization technique to obtain an objective function of the hyperspectral pan-sharpening task;

[0042] a decoupling module, wherein the decoupling module solves the objective function of the hyperspectral pan-sharpening task by using a semi-quadratic splitting method, and decouples the problem of reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image into spectral fidelity and spatial fidelity subproblems and a sparse prior subproblem;

[0043] A first reconstruction module, wherein the first reconstruction module uses a residual module to simulate a degradation operator and a gradient descent algorithm to iteratively solve spectral fidelity and spatial fidelity sub-problems to obtain a high-fidelity and high-resolution hyperspectral image;

[0044] The second reconstruction module obtains spatial spectral prior features by solving the sparse prior sub-problem based on the iterative shrinkage threshold algorithm;

[0045] A correction module, wherein the correction module corrects the high-fidelity and high-resolution hyperspectral image by using spatial spectral prior features to obtain a first-stage reconstructed high-resolution hyperspectral image;

[0046] A cross-stage fusion module is used to fuse the multi-stage reconstructed high-resolution hyperspectral images through a cross-stage memory fusion network to obtain a high-resolution hyperspectral image.

[0047] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0048] Optimize the existing technology for insufficient utilization of spectral and spatial original prior feature information in remote sensing hyperspectral image super-resolution fusion. Aiming at the shortcoming that the network model is difficult to interpret, the spectral space optimization reconstruction problem is decoupled by in-depth research on the correlation between the spectrum of the hyperspectral image and the spatial information of the pan-sharpened image, and the network model is made more interpretable by utilizing the prior features in the original image; by constructing a fusion model with anti-noise and information loss characteristics, it effectively copes with the interference caused by the differences in multi-source data; and provides technical support for the analysis and practical application of hyperspectral images. The present invention improves the reconstruction accuracy of the image, reduces the computational complexity, realizes the extraction of the joint prior features of the spatial spectrum of the original image, and realizes lossless information transmission.

[0049] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 It is a flow chart of a method for sharpening and preserving the fidelity of a deep-expanded spatial spectrum sparse memory hyperspectral provided by the present invention.

[0052] Figure 2This is a schematic diagram of the network structure of a deeply expanded spatial spectrum sparse memory hyperspectral sharpening and fidelity preservation method provided by the present invention.

[0053] Figure 3 It is a structural schematic diagram of a deeply expanded spatial spectrum sparse memory hyperspectral sharpening and fidelity preservation system provided by the present invention.

[0054] Reference numerals:

[0055] 101. Construction module; 102. Decoupling module; 103. First reconstruction module; 104. Second reconstruction module; 105. Correction module; 106. Cross-stage fusion module. DETAILED DESCRIPTION

[0056] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0057] In the description of the embodiments of the present invention, it should be noted that the terms "first", "second" and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0058] The following combination Figures 1 to 3 The present invention describes a method and system for deep expansion of spatial spectrum sparse memory hyperspectral sharpening and fidelity preservation.

[0059] like Figure 1 As shown, a deep unfolded spatial spectrum sparse memory hyperspectral sharpening fidelity preservation method includes:

[0060] S1: The prior knowledge of high-resolution hyperspectral images is encoded by regularization techniques to obtain the objective function of the hyperspectral pan-sharpening task;

[0061] For the hyperspectral pan-sharpening task, the observation model can be expressed as follows:

[0062]

[0063]

[0064] in, To have bands and high-resolution hyperspectral images with pixels, , The dimension is Image set, For the observed bands and pixels of low-resolution hyperspectral image, , The dimension is Image set, is the observed frequency band with 1 pixels of full-color image, , The dimension is Image set, is the spatial response operator, , The dimension is The matrix, is the spectral response operator, , The dimension is The matrix, is the noise existing in the low-resolution hyperspectral image during the observation process, It is the noise existing in the full-color image during the observation process.

[0065] From the above observation model, it can be seen that obtaining a high-resolution hyperspectral image from a low-resolution hyperspectral image and a panchromatic image is an ill-posed inverse problem. Therefore, regularization technology is usually used to encode the prior knowledge of the desired high-resolution hyperspectral image. The calculation expression of the objective function of the hyperspectral panchromatic sharpening task is:

[0066]

[0067] in, is the spectral fidelity term, is the spatial fidelity term, is the regularization term, To balance the parameters, is the prior feature of high-resolution hyperspectral image, is the Frobenius matrix norm, To minimize the function value.

[0068] S2: Solve the objective function of the hyperspectral pan-sharpening task by using the half-quadratic splitting (HQS) method, decoupling the problem of reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image into spectral fidelity and spatial fidelity sub-problems and a sparse prior sub-problem;

[0069] like Figure 2 As shown, when the present invention reconstructs a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image, the low-spatial-resolution hyperspectral image is upsampled and The convolution operation is performed on the full-color image The convolution operation.

[0070] The semi-quadratic splitting method is used to solve the objective function of the hyperspectral pan-sharpening task, and auxiliary variables are introduced to decompose the objective function of the hyperspectral pan-sharpening task:

[0071]

[0072] in, For constraints, is an auxiliary variable;

[0073] Convert the optimization problem with equality constraints into an unconstrained optimization problem, and the calculation expression is:

[0074]

[0075] in, is the unconstrained optimization objective function, is the penalty parameter, for Prior information, To minimize the function and value.

[0076] The unconstrained optimization problem is decomposed into two sub-problems for iterative solution. The fidelity term and regularization term are decoupled into two separate sub-problems. The calculation expressions of the spectral fidelity and spatial fidelity sub-problems are:

[0077]

[0078] in, These are the spectral fidelity and spatial fidelity subproblems;

[0079] The computational expression of the sparse prior subproblem is:

[0080]

[0081] in, is the sparse prior subproblem, To minimize the function value.

[0082] S3: Using the residual module to simulate the degradation operator and the gradient descent algorithm to iteratively solve the spectral fidelity and spatial fidelity sub-problems to obtain high-fidelity and high-resolution hyperspectral images;

[0083] The Frobenius matrix norm expansion of the spectral fidelity and spatial fidelity subproblems is used to reconstruct high-resolution hyperspectral images using the gradient descent algorithm.

[0084] By expanding the Frobenius matrix norm and applying the gradient descent algorithm to it, The gradient of , is calculated as:

[0085]

[0086] in, for The gradient, For the Iterations of high-resolution hyperspectral images, For the Auxiliary variables for iterations, is the transpose of the matrix;

[0087] A residual module is used to simulate the degradation operator to iteratively update the spectral difference between the low-resolution hyperspectral image and the reconstructed high-resolution hyperspectral image, as well as the spatial difference between the panchromatic image and the reconstructed high-resolution hyperspectral image, to obtain a high-fidelity high-resolution hyperspectral image.

[0088] The degradation operators include a spatial response operator matrix and a spatial response operator transposed matrix in a spatial degradation process, and a spectral response operator matrix and a spectral response operator transposed matrix in a spectral degradation process;

[0089] because 、 、 、 are all unknown items, so the present invention uses residual network to simulate the degradation operator. The updated calculation expression is:

[0090]

[0091] in, For the Iterations of high-resolution hyperspectral images, For the first step;

[0092] For low-resolution high-spectral and The spectral difference between the iterative reconstruction results is For full-color images The spatial differences between the iterative reconstruction results are calculated, and a high-fidelity and high-resolution hyperspectral image is reconstructed after multiple iterations.

[0093] S4: Obtain spatial spectral prior features by solving the sparse prior subproblem based on the iterative shrinkage threshold algorithm;

[0094] The core of sparse dictionary learning is to represent signals through linear combinations of a small number of atoms. Its effectiveness depends on the balance between the completeness of the dictionary and the sparsity constraints. Traditional methods (such as K-SVD) rely on training with fixed input data. The present invention adopts an end-to-end learnable dictionary structure, which enables the dictionary to adapt to the distribution of training data.

[0095] Panchromatic images have high spatial resolution, and their spatial detail features (such as edges and textures) can be regarded as a priori knowledge to improve the spatial resolution of hyperspectral images. Sparse representation theory provides a mathematical basis for prior modeling. Its core theory believes that the linear representation of local image blocks under appropriate basis functions is usually sparse. Assuming that the image blocks , , for dimensional graphic block set, there is an overcomplete dictionary and sparse coefficients , , for Dimensional dictionary set, , for The sparse coefficient of dimension is such that:

[0096]

[0097] in, is the number of non-zero elements, the image structure can be efficiently represented by a linear combination of a small number of atoms.

[0098] When the image block dimension is , the number of atoms in the dictionary satisfy When , the learned overcomplete dictionary can cover the spatial mapping relationship of the image, and the following optimization function is used:

[0099]

[0100] in, To complete the dictionary, , is the image block dimension, For the The sparse coefficients of image blocks, is the balance parameter, To obtain the minimum function, For the image blocks, is the Manhattan norm, is the square of the Euclidean norm.

[0101] Obtaining spatial prior features includes:

[0102] S411: Decomposing the full-color image into a plurality of image blocks by a sliding window; the image blocks have an overcomplete dictionary and sparse coefficients;

[0103] For the input full-color image , Through the sliding window, where is the height of the image, is the width of the image, decompose the image into many image blocks , , For the image blocks, is the number of image blocks;

[0104] S412: Obtain a spatial optimization function by learning the spatial mapping relationship of the overcomplete dictionary covering the image; the calculation expression is:

[0105] .

[0106] Generate an initialized overcomplete dictionary through the Xavier initialization strategy; obtain the optimal spatial dictionary by iteratively optimizing the initialized overcomplete dictionary through gradient descent;

[0107] The initial overcomplete dictionary satisfies:

[0108]

[0109] in, For the An initial overcomplete dictionary of image patches;

[0110] During the iteration process, the overcomplete dictionary is updated as follows:

[0111]

[0112] in, For the Image block The overcomplete dictionary of iterations, is the second step length, is the gradient of the overcomplete dictionary, For the Image block The overcomplete dictionary of iterations, For the Image block The sparse coefficient matrix of the iteration, For the The first iteration image blocks.

[0113] S414: Obtain the optimal spatial sparse coefficient by iteratively solving the spatial optimization function based on the iterative shrinkage threshold algorithm; the calculation expression is:

[0114]

[0115] in, For the The sparse coefficient matrix of the iteration, is the soft threshold function, , is a symbolic function, is the threshold parameter of the soft threshold function, is the maximum value function.

[0116] S415: reconstructing a priori features of the image block by linearly combining the optimal spatial dictionary and the optimal spatial sparse coefficient, and integrating the a priori features of the image block to obtain a spatial priori feature;

[0117] Through the optimal spatial dictionary and the optimal spatial sparse coefficient Linear combination reconstruction obtains the prior features of the image block Finally, the prior features of the image block are integrated back into the complete image prior features , the calculation expression is:

[0118]

[0119]

[0120] in, An operation to integrate image blocks.

[0121] Hyperspectral images have the characteristics of high spectral resolution and strong correlation between spectral bands. Traditional methods have difficulty directly modeling such complex mapping relationships. Direct processing of hyperspectral images often faces problems such as high computational complexity, the easy propagation of sensor noise in the original spectral space, and the high correlation between adjacent bands requiring compact representation. Sparse dictionary learning constructs adaptive dictionary atoms in a data-driven manner, providing an efficient solution for modeling spectral prior features. Its core assumption is that spectral information can be constructed as a linear combination of a small number of dictionary atoms:

[0122]

[0123] in, is the spectral dictionary, , The dimension is The spectral dictionary set, is the spectral sparsity coefficient, , The dimension is A collection of spectral dictionaries.

[0124] Obtaining spectral prior features includes:

[0125] S421: Reshaping low-resolution hyperspectral images;

[0126] Given a hyperspectral image , ,Will Reshape into , To reshape the image, ,

[0127] S422: Obtain a spectral optimization function by learning the spectral mapping relationship of the spectral dictionary covering the image; the calculation expression of the spectral optimization function is:

[0128]

[0129] in, is the balance parameter.

[0130] S423: Iteratively solve the spectrum optimization function based on the iterative shrinkage threshold algorithm to obtain the optimal spectrum dictionary and the optimal spectrum sparse coefficient. The calculation expression is:

[0131]

[0132]

[0133] in, For the The spectral dictionary of the iteration, For the The spectral dictionary of the iteration, is the gradient of the spectral dictionary, For the The reshaped image of the iterations, is the soft threshold function, For the The spectral sparsity coefficient of the iteration, For the The spectral sparsity coefficient of the iteration, .

[0134] S424: Obtain spectral prior features by linearly combining the optimal spectral dictionary and the optimal spectral sparse coefficient. The calculation expression is:

[0135]

[0136] in, is the spectral prior feature, is the optimal spectral sparsity coefficient, is the optimal spectral dictionary.

[0137] Next, the two sparsely coded prior features are integrated to obtain the spatial spectrum prior features. The calculation expression is:

[0138]

[0139] in, is the spatial spectrum prior feature.

[0140] S5: Modify the high-fidelity high-resolution hyperspectral image by using spatial spectral prior features to obtain the first-stage reconstructed high-resolution hyperspectral image;

[0141] For the sub-problem , as the initial reconstruction of Traditional methods can oversmooth details in complex textured areas. Sparse dictionary learning can generate more adaptable basis functions based on the edges, details, and spectral features in the data, enabling more accurate estimation of image content. Furthermore, noise in the image data is suppressed due to the sparsity constraint, so sparse representation can significantly separate signal from noise. During the fusion reconstruction process, the upsampling operation of LRHS can cause distortion of spectral information, but sparse dictionary learning can effectively restore spectral features.

[0142] S6: Repeat steps S1 to S5 to obtain a multi-stage reconstructed high-resolution hyperspectral image;

[0143] Through multi-stage reconstruction of high-resolution hyperspectral images, the reconstructed images are continuously optimized, the spatial resolution and spectral fidelity of the hyperspectral images are gradually improved, and the multi-stage features are integrated to finally output high-quality, high-resolution reconstructed images, making the processed images more accurate in structure and spectral information.

[0144] S7: The multi-stage reconstructed high-resolution hyperspectral images are fused through a cross-stage memory fusion network to obtain a high-resolution hyperspectral image;

[0145] In a deep unfolding network, it is often faced with the situation that the spectral spatial information reconstructed in the previous stage will be lost as the reconstruction stage processing increases. A cross-stage memory fusion network is designed in each stage to pass lossless information to the next stage.

[0146] The cross-stage memory fusion network embeds the features of the previous stage into the features of the current stage and adds them element by element. The calculation expression of the cross-stage fusion memory fusion network is:

[0147]

[0148] in, For the High-resolution hyperspectral images of the phase, For the The reconstruction of high-resolution hyperspectral images in the stage is the first learnable affine parameter, is the second learnable affine parameter;

[0149] and All passed Convolutional features of the stage It is obtained through convolution processing to control the fusion ratio of the previous stage and the current stage. For the High-resolution hyperspectral images of the phase The convolution kernel size is The convolution calculation is obtained, and the calculation expression is:

[0150]

[0151] in, is the convolution kernel size convolution.

[0152] In the above fusion process, as well as Not a vector but with The stage features have tensors of the same dimension, and the refined memory features of the previous stage and the reconstructed features of the current stage are processed simultaneously, which maintains the integrity of the spectral-spatial information and enhances the proximal mapping.

[0153] The present invention combines the spectral information of the hyperspectral image and the high-resolution spatial information of the panchromatic image through sparse dictionary learning to obtain the prior information in the original image; then fuses and reconstructs them through an interpretable deep unfolding network, and finally generates a fused image with both high spectral fidelity and high spatial resolution.

[0154] The present invention solves spectral fidelity, spatial fidelity, sparse prior and cross-stage fusion networks to form an interpretable deep expansion network.

[0155] This method was compared with several state-of-the-art algorithms on the public Pavia Center dataset using quantitative and subjective metrics. To better visualize the differences between the fusion results, the root mean square error (RMSE) between the fusion results and the ground truth (GT) was calculated and displayed. The state-of-the-art algorithms used were FPFNet (2023), HyperRefiner (2023), TreeNet (2024), and DFCFN (2025). FPFNet (2023) is the 2023 version of the Feature Pyramid Fusion Network, HyperRefiner (2023) is the 2023 version of the Refined Hyperspectral Pan-spectral Network, TreeNet (2024) is the 2024 version of the Tree-structured Neural Network, and DFCFN (2025) is the 2025 version of the Dual-stage Feature Correction Fusion Network. The quantitative metrics of the proposed algorithm demonstrate that they outperform those of recent state-of-the-art algorithms. The experimental results are shown in Table 1.

[0156] Table 1 Comparative experiment results of quantitative indicators of Pavia center dataset

[0157] Among them, SCC is the structural correlation coefficient, SAM is the spectral angle mapping, RMSE is the root mean square error, ERGAS is the dimensionless global relative error, PSNR is the peak signal-to-noise ratio, "↑" means the higher the value, the better, and "↓" means the lower the value, the better.

[0158] like Figure 3 As shown, a deep-expanded spatial spectrum sparse memory hyperspectral sharpening and fidelity preservation system is used to perform a deep-expanded spatial spectrum sparse memory hyperspectral sharpening and fidelity preservation method, comprising:

[0159] The construction module 101 encodes prior knowledge of the high-resolution hyperspectral image through regularization technology to obtain the objective function of the hyperspectral pan-sharpening task;

[0160] The decoupling module 102 solves the objective function of the hyperspectral pan-sharpening task by using a semi-quadratic splitting method, and decouples the problem of reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image into spectral fidelity and spatial fidelity sub-problems and a sparse prior sub-problem;

[0161] The first reconstruction module 103 uses a residual module to simulate a degradation operator and a gradient descent algorithm to iteratively solve the spectral fidelity and spatial fidelity sub-problems to obtain a high-fidelity and high-resolution hyperspectral image;

[0162] The second reconstruction module 104 obtains spatial spectrum prior features by solving the sparse prior sub-problem based on an iterative shrinkage threshold algorithm;

[0163] The correction module 105 corrects the high-fidelity and high-resolution hyperspectral image by using the spatial spectrum prior features to obtain a first-stage reconstructed high-resolution hyperspectral image;

[0164] The cross-stage fusion module 106 fuses the multi-stage reconstructed high-resolution hyperspectral images through a cross-stage memory fusion network to obtain a high-resolution hyperspectral image.

[0165] Through the collaborative work of the above modules, the existing technology is optimized for the insufficient utilization of spectral and spatial original prior feature information in the super-resolution fusion of remote sensing hyperspectral images. In view of the shortcomings of the network model being difficult to interpret, the spectral space optimization reconstruction problem is decoupled by in-depth research on the correlation between the hyperspectral image spectrum and the spatial information of the full-color sharpened image, and the network model is made more interpretable by utilizing the prior features in the original image; by constructing a fusion model with anti-noise and information loss characteristics, it effectively copes with the interference caused by differences in multi-source data; and provides technical support for the analysis and practical application of hyperspectral images. The present invention improves the reconstruction accuracy of the image, reduces the computational complexity, realizes the extraction of the joint prior features of the spatial spectrum of the original image, and realizes lossless information transmission.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A deep unfolded spatial spectrum sparse memory hyperspectral sharpening and fidelity preservation method, characterized by: include: S1: The prior knowledge of high-resolution hyperspectral images is encoded by regularization techniques to obtain the objective function of the hyperspectral pan-sharpening task; S2: Solve the objective function of the hyperspectral pan-sharpening task by using the semi-quadratic splitting method, decoupling the problem of reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image into spectral fidelity and spatial fidelity sub-problems and a sparse prior sub-problem; S3: A residual module is used to simulate the degradation operator and the gradient descent algorithm is used to iteratively solve the spectral fidelity and spatial fidelity sub-problems to obtain high-fidelity and high-resolution hyperspectral images. S4: Obtain spatial spectral prior features by solving the sparse prior subproblem based on the iterative shrinkage threshold algorithm; S5: Modify the high-fidelity high-resolution hyperspectral image by using spatial spectral prior features to obtain the first-stage reconstructed high-resolution hyperspectral image; S6: Repeat steps S1 to S5 to obtain a multi-stage reconstructed high-resolution hyperspectral image; S7: The multi-stage reconstructed high-resolution hyperspectral images are fused through a cross-stage memory fusion network to obtain a high-resolution hyperspectral image.

2. The method for sharpening and preserving the fidelity of a deep-expanded spatial spectrum with sparse memory according to claim 1, characterized in that: In step S1, the objective function of the hyperspectral pan-sharpening task is: in, To have bands and high-resolution hyperspectral images with pixels, , The dimension is Image set, For the observed bands and pixels of low-resolution hyperspectral image, , The dimension is Image set, is the spatial response operator, , The dimension is The matrix, is the observed waveband with 1 band and pixels of full-color image, , The dimension is Image set, is the spectral response operator, , The dimension is The matrix, To balance the parameters, is the prior information of high-resolution hyperspectral images, is the Frobenius matrix norm, To minimize the function The value of .

3. The method for sharpening and preserving the fidelity of a deep-expanded spatial spectrum with sparse memory according to claim 2, characterized in that: In step S2, the calculation expressions of the spectral fidelity and spatial fidelity sub-problems are: in, are the spectral fidelity and spatial fidelity subproblems, is an auxiliary variable, is the penalty parameter; The computational expression of the sparse prior subproblem is: in, is the sparse prior subproblem.

4. The method for sharpening and preserving the fidelity of a deep unfolded spatial spectrum with sparse memory according to claim 1, characterized in that: In step S3, The Frobenius matrix norm expansion of the spectral fidelity and spatial fidelity subproblems is used to reconstruct high-resolution hyperspectral images using the gradient descent algorithm. The residual module is used to simulate the degradation operator to iteratively update the spectral difference between the low-resolution hyperspectral image and the reconstructed high-resolution hyperspectral image, as well as the spatial difference between the panchromatic image and the reconstructed high-resolution hyperspectral image, to obtain a high-fidelity high-resolution hyperspectral image.

5. The method for sharpening and preserving the fidelity of a deep-expanded spatial spectrum with sparse memory according to claim 4, characterized in that: In step S3, the degradation operator includes a spatial response operator matrix and a spatial response operator transposed matrix in a spatial degradation process, and a spectral response operator matrix and a spectral response operator transposed matrix in a spectral degradation process.

6. The method for sharpening and preserving the fidelity of a deep unfolded spatial spectrum with sparse memory according to claim 1, characterized in that: The S4 step includes: S411: Decomposing the full-color image into a plurality of image blocks by a sliding window; the image blocks have an overcomplete dictionary and sparse coefficients; S412: Obtaining a spatial optimization function by learning a spatial mapping relationship of an overcomplete dictionary covering the image; S413: Generate an initialized overcomplete dictionary through the Xavier initialization strategy; and obtain an optimal spatial dictionary by iteratively optimizing the initialized overcomplete dictionary through gradient descent. S414: Obtaining an optimal spatial sparse coefficient by iteratively solving a spatial optimization function based on an iterative shrinkage threshold algorithm; S415: reconstructing the prior features of the image block by linearly combining the optimal spatial dictionary and the optimal spatial sparse coefficient, and integrating the prior features of the image block to obtain the spatial prior features.

7. The method for sharpening and preserving the fidelity of a deep-expanded spatial spectrum with sparse memory according to claim 6, characterized in that: Step S4 also includes: S421: Reshaping low-resolution hyperspectral images, S422: Obtaining a spectral optimization function by learning a spectral mapping relationship of a spectral dictionary covering an image; S423: Iteratively solving the spectral optimization function based on the iterative shrinkage threshold algorithm to obtain the optimal spectral dictionary and the optimal spectral sparse coefficient; S424: Obtaining spectral prior features by linearly combining the optimal spectral dictionary and the optimal spectral sparse coefficient.

8. The method for sharpening and preserving the fidelity of a deep unfolded spatial spectrum with sparse memory according to claim 7, characterized in that: Integrate the spatial prior features and spectral prior features. Obtain spatial spectral prior features.

9. The method for sharpening and preserving the fidelity of a deep unfolded spatial spectrum with sparse memory hyperspectral according to claim 1, characterized in that: In step S7, the cross-stage memory fusion network embeds the features of the previous stage into the features of the current stage and adds them element by element.

10. A deep unfolded spatial spectrum sparse memory hyperspectral sharpening fidelity system, characterized by: The method for performing a deep unfolded spatial spectrum sparse memory hyperspectral sharpening and fidelity preservation method according to any one of claims 1 to 9 comprises: A construction module, wherein the construction module encodes prior knowledge of the high-resolution hyperspectral image through a regularization technique to obtain an objective function of the hyperspectral pan-sharpening task; a decoupling module, wherein the decoupling module solves the objective function of the hyperspectral pan-sharpening task by using a semi-quadratic splitting method, and decouples the problem of reconstructing a high-resolution hyperspectral image from a panchromatic image and a low-spatial-resolution hyperspectral image into spectral fidelity and spatial fidelity subproblems and a sparse prior subproblem; A first reconstruction module, wherein the first reconstruction module uses a residual module to simulate a degradation operator and a gradient descent algorithm to iteratively solve spectral fidelity and spatial fidelity sub-problems to obtain a high-fidelity and high-resolution hyperspectral image; The second reconstruction module obtains spatial spectral prior features by solving the sparse prior sub-problem based on the iterative shrinkage threshold algorithm; A correction module, wherein the correction module corrects the high-fidelity and high-resolution hyperspectral image by using spatial spectral prior features to obtain a first-stage reconstructed high-resolution hyperspectral image; A cross-stage fusion module is used to fuse the multi-stage reconstructed high-resolution hyperspectral images through a cross-stage memory fusion network to obtain a high-resolution hyperspectral image.

Images