A Multimodal Denoising Method for Hyperspectral Images

Abstract

This invention discloses a multimodal denoising method for hyperspectral images, characterized by the following steps: S1, using Tucker decomposition to perform low-rank tensor representation on both the hyperspectral image and the registered multispectral image, extracting the core tensors of both; S2, establishing a correlation model between the hyperspectral and multispectral core tensors through model-driven linear mapping or data-driven multilayer perceptron networks; S3, iteratively solving for the core tensors, factor matrices, and correlation model parameters using the alternating direction multiplier method; S4, reconstructing the denoised hyperspectral image using the optimized hyperspectral core tensor and factor matrix. Compared with existing technologies, this method fully explores and utilizes the spectral details of the hyperspectral image and the high signal-to-noise ratio spatial information of the multispectral image. Through a dual Tucker decomposition framework and a core tensor correlation strategy, it effectively improves the restoration accuracy in mixed noise scenarios, obtaining high-quality hyperspectral images.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology and relates to a multimodal denoising method for hyperspectral images. Background Technology

[0002] Hyperspectral imaging technology can acquire rich spatial and spectral information about ground features and has important application value in many fields. However, in actual imaging processes, the quality of hyperspectral images inevitably degrades due to complex imaging conditions and various noise sources, which severely restricts subsequent advanced processing, such as super-resolution, spectral unmixing, and target detection.

[0003] Currently, hyperspectral image denoising methods are mainly divided into two categories: model-driven and data-driven. Model-driven methods typically utilize prior information about the image, such as low-rank characteristics, to construct regularization models. However, these methods rely solely on the information inherent in the hyperspectral image itself, and their performance becomes bottlenecked when noise is severe. Data-driven methods, especially those based on deep learning, have made significant progress, but they usually require large amounts of labeled training data, or their performance is limited in unsupervised modes due to the limited availability of information.

[0004] Existing technologies generally neglect important reference information. Multispectral images acquired concurrently with hyperspectral images typically have higher signal-to-noise ratios and richer spatial information. How to effectively utilize this high-quality multimodal auxiliary information to enhance the denoising effect of hyperspectral images is a pressing technical challenge that needs to be addressed. Summary of the Invention

[0005] This invention aims to address the performance limitations of existing hyperspectral image denoising methods that rely solely on single-modal data. It proposes a multimodal denoising method that can effectively fuse hyperspectral image and high signal-to-noise ratio multispectral image information, thereby significantly improving the denoising performance and detail preservation of hyperspectral images.

[0006] The method of the present invention is achieved through the following technical solution:

[0007] A multimodal denoising method for hyperspectral images includes the following steps:

[0008] S1, using Tucker decomposition to perform low-rank tensor representations on the hyperspectral image and the registered multispectral image, respectively.

[0009] Extract the core tensors of both;

[0010] S2 establishes a correlation model between hyperspectral and multispectral core tensors through model-driven linear transformation or data-driven multilayer perceptron networks.

[0011] S3. Based on the core tensor of the hyperspectral image, the core tensor of the multispectral image, and the correlation model, an optimization objective function is constructed. The core tensor, factor matrix, and correlation model parameters in the optimization objective function are solved iteratively using the alternating direction multiplier method.

[0012] S4 reconstructs the denoised hyperspectral image using the optimized hyperspectral core tensor and factor matrix.

[0013] Furthermore, in step S2, the model-driven linear transformation constraint association model is as follows: a direct mapping function is established between the core tensor of the noisy hyperspectral image and the core tensor of the registered multispectral image through the linear transformation matrix, wherein the direct mapping function is the regularization term of the linear transformation matrix.

[0014] Furthermore, in step S2, the association model obtained by the data-driven multilayer perceptron network is as follows: the multilayer perceptron network is used to learn and approximate the nonlinear mapping function between the core tensor of the noisy hyperspectral image and the core tensor of the registered multispectral image. The multilayer perceptron network contains at least one linear layer and a LeakyReLu activation function, and is trained and solved using the stochastic gradient descent algorithm.

[0015] Furthermore, in step S3, the optimization objective function is expressed as minimizing the sum of the reconstruction error of the hyperspectral image, the L1 norm of the sparse noise term, the L1 norm of the core tensor of the noisy hyperspectral image, the L1 norm of the core tensor of the registered multispectral image, and the core tensor correlation modeling error. Specifically, the optimization objective function is:

[0016] The optimization objective function constructed through model-driven linear transformation is shown in the following equation:

[0017]

[0018] The optimization objective function constructed using a data-driven multilayer perceptron network is shown in the following equation:

[0019]

[0020] In the above declaration, X represents the reconstructed noise-free hyperspectral image, S represents the sparse noise component, C1 and C2 are the core tensors of the noise-free spectral image and the registered multispectral image, respectively, and U i V i denoted as the factor matrices for the noise-free hyperspectral image and the registered multispectral image, respectively; P is the linear transformation matrix in the model-driven strategy; Φ(·;Θ) is the multilayer perceptron network in the data-driven strategy; Θ represents the network parameters; st represents the constraints of the optimization objective function; ×1, ×2, and ×3 represent multiplication along modulo 1, modulo 2, and modulo 3, respectively; and ‖·‖1 represents L1 norm computation. This means stretching all elements of the tensor into a long vector and then calculating the Euclidean length of that long vector, where λ,μ1,μ2,γ,β are regularization parameters.

[0021] Furthermore, the objective function is iteratively solved using the alternating direction multiplier method, specifically including the following steps:

[0022] Input the image pair to be processed, γ and M, and the input parameters, where γ is a noisy hyperspectral remote sensing image with m rows, n columns, and z band, and M is a registered multispectral remote sensing image with m rows, n columns, and p band; the input parameters include: the maximum number of iterations t. max The preset threshold tol, the Tucker decomposition rank [r1,r2,R3] of the hyperspectral image, the Tucker decomposition rank [r1,r2,r3] of the multispectral image, and the model regularization parameters λ,μ1,μ2,γ,β, and the penalty parameter η;

[0023] Initialize each variable: sparse noise S (0) =0, core tensor auxiliary variable Lagrange multipliers

[0024] After introducing auxiliary variables Q1 and Q2, the two optimization objective functions are rewritten as follows:

[0025]

[0026] The core tensors C1 and C2 are updated sequentially, along with the associated modeling parameters, including the linear transformation matrix P in the model-driven strategy or the multilayer perceptron network parameters Θ and the multispectral factor matrix V in the data-driven strategy. i Auxiliary variables Q1 and Q2 of core tensor C1 and core tensor C2, sparse noise S, and hyperspectral factor matrix U. i The process involves reconstructing a noise-free hyperspectral image X, and using the dual variables Lagrange multipliers Λ1 and Λ2 corresponding to the linear constraint C1 = Q1, until the maximum number of iterations is reached or the change in X between two adjacent iterations is less than a preset threshold tol.

[0027] The specific process for updating each parameter is as follows:

[0028] (1) Update the core tensors C1 and C2 as follows:

[0029] (1-1) Under the model-driven strategy: Update C1: With other variables fixed, C1 is solved using the following formula:

[0030]

[0031] The solution in C1 matrix form obtained from the above equation is: Where, A1=ηIA +βP T P+I A I A This represents the identity matrix of size R3×R3. Indicates the inverse of A1. `unfold` means expanding the tensor according to mode 3.

[0032] Update C2: Keeping other variables fixed, we get:

[0033]

[0034] Solve the subproblem concerning C2, the solution of which has the form C2 = B2 / η + β + 1.

[0035] (1-2) Under the data-driven strategy: Update C1: With other variables fixed, solve using the following formula:

[0036]

[0037] Since MLP networks are differentiable, this problem also has a closed-form solution: Where, A1=ηI A +βΦ T Φ+I A I A This represents the identity matrix of size R3×R3. Indicates the inverse of A1.

[0038] Update C2: With other variables fixed, solve the subproblem of C2 using the following formula:

[0039]

[0040] Solve the subproblem concerning C2, the solution of which has the form C2 = B2 / η + β + 1, where,

[0041] (2) Update the association modeling parameters as follows:

[0042] (2-1) Under the model-driven strategy: Update P: With other variables fixed, solve for the transformation matrix P using the following formula:

[0043]

[0044] The solution is in the form of Among them, I p This represents an identity matrix of size r3×R3.

[0045] (2-2) Under the data-driven strategy: Updating network parameters Θ: With other variables fixed, the stochastic gradient descent (SGD) algorithm is used to solve the minimization problem of Θ using the following formula to update the multilayer perceptron network:

[0046]

[0047] (3) Update the multispectral image factor matrix V i and noiseless hyperspectral image factor matrix U i Specifically, the multispectral image factor matrix V is updated as follows. i With other variables fixed, solve using the following formula:

[0048]

[0049] The standard high-order orthogonal iterative algorithm is used to solve for V. i The optimization problem.

[0050] (4) Update the auxiliary variables Q1, Q2 and sparse noise S as follows:

[0051] When updating auxiliary variables Q1 and Q2, keep other variables fixed, and solve the subproblems about Q1 and Q2 using the following formula:

[0052]

[0053] With sparse noise S fixed, other variables are fixed, and the subproblem concerning S is solved using the following formula:

[0054]

[0055] By solving the above variables using the soft threshold shrinkage operator, we obtain the following results: S = Δ λ [YX], where Δ ε [q] = sgn(q)·max(|q|-ε,0), where · represents element-wise multiplication, sgn(·) represents the sign function, q is the variable to be substituted, and ε represents the constant term substituted into the formula.

[0056] (5) Update the noiseless hyperspectral image factor matrix U i With other variables fixed, the solution is obtained using the following formula:

[0057]

[0058] The standard high-order orthogonal iterative algorithm is used to solve for U. i The optimization problem.

[0059] (6) The updated and reconstructed noiseless hyperspectral image X is as follows:

[0060] X = C1×1U1×2U2×3U3.

[0061] (7) Update the Lagrange multipliers Λ1 and Λ2. According to the standard update rules of the alternating direction multiplier algorithm framework, the update formula of the Lagrange multipliers is: Λ1=Λ1+η(C1-Q1), Λ2=Λ2+η(C2-Q2).

[0062] Compared with the prior art, the beneficial effects of this invention are as follows:

[0063] This invention is the first to introduce high signal-to-noise ratio (SNR) multispectral images as auxiliary information into hyperspectral image denoising tasks. It proposes a dual Tucker decomposition framework that effectively separates and utilizes the spectral characteristics of hyperspectral images and the spatial characteristics of multispectral images, achieving effective fusion of cross-modal information. Simultaneously, it provides both model-driven and data-driven strategies to explore the intrinsic relationships of the core tensors, enhancing the model's adaptability and performance. By referencing the rich spatial details of multispectral images, this invention can fully utilize the multimodal information from both hyperspectral and multispectral sources to perform more thorough denoising of noisy hyperspectral images. It can simultaneously remove mixed noise of Gaussian noise and sparse noise, including salt-and-pepper noise, stripes, and dead lines, while better preserving the image's fine texture and edge structure. Attached Figure Description

[0064] Figure 1 This is a flowchart of a multimodal denoising method for hyperspectral images according to an embodiment of the present invention;

[0065] Figure 2 This is an operation flowchart of an embodiment of the present invention;

[0066] Figure 3 This is an overall flowchart of the dual Tucker framework proposed in this invention;

[0067] Figure 4 This is a qualitative comparison of experimental results for different denoising methods on the Pavia University dataset. Detailed Implementation

[0068] Exemplary embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the invention to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0069] This invention discloses a multimodal denoising method for hyperspectral images, such as... Figure 1 As shown, its basic implementation process is as follows:

[0070] First, Tucker decomposition is used to perform low-rank tensor representation on the hyperspectral image and the registered multispectral image respectively, and the core tensor of both is extracted. The high-dimensional data is represented as the product of a low-rank core tensor and a factor matrix, thereby effectively extracting the main features of each data mode.

[0071] Then, a mapping relationship is established from the core tensor of the noisy hyperspectral image to the core tensor of the registered multispectral image. This invention explores two strategies to achieve this cross-modal association: the first is a model-driven strategy, which assumes a linear transformation relationship between the two core tensors and explicitly models this relationship by introducing a linear transformation matrix; the second is a data-driven strategy, which utilizes a multilayer perceptron network to learn the potential, more complex nonlinear mapping relationship between the two core tensors.

[0072] Finally, the aforementioned dual-path Tucker decomposition and correlation modeling are integrated into a unified optimization objective function. This objective function comprehensively considers data fidelity, noise sparsity, and the low-rank characteristics of the clean image. The entire optimization problem is efficiently solved iteratively using the alternating direction multiplier method, sequentially updating the denoised hyperspectral image, noise components, two core tensors, each set of factor matrices, and correlation model parameters until the convergence condition or the maximum number of iterations is reached, ultimately reconstructing a high-quality denoised hyperspectral image.

[0073] The detailed calculation process of the denoising method proposed in this invention is as follows: Figure 2 As shown, the specific steps are as follows:

[0074] Step 1: Data and Parameter Initialization. Input the image pair to be processed, Y and M, and the input parameters, where Y is a noisy hyperspectral remote sensing image with m rows, n columns, and z band, and M is a registered multispectral remote sensing image with m rows, n columns, and p band; the input parameters include: maximum number of iterations t. max The Tucker decomposition rank [r1,r2,R3] of the hyperspectral image, the Tucker decomposition rank [r1,r2,r3] of the multispectral image, and the model regularization parameters λ,μ1,μ2,γ,β, and the penalty parameter η.

[0075] Initialize each variable: sparse noise S (0) =0, core tensor auxiliary variable Lagrange multipliers Maximum number of iterations t maxThe preset threshold is tol. Q1 is an auxiliary variable of the noiseless hyperspectral image core tensor C1, Q2 is an auxiliary variable of the registered multispectral image core tensor C2, and the Lagrange multipliers Λ1 and Λ2 represent the dual variables corresponding to the linear constraints C1=Q1 and C2=Q2, respectively.

[0076] Step 2: Construct the optimization objective function of the dual Tucker multimodal denoising model. The optimization objective function is specifically expressed as minimizing the reconstruction error of the hyperspectral image, the L1 norm of the sparse noise term, the L1 norm of the core tensor C1 of the noiseless hyperspectral image, the L1 norm of the core tensor C2 of the registered multispectral image, and the sum of the core tensor correlation modeling error.

[0077] The DTucker optimization objective function constructed through model-driven linear transformation is shown in the following equation:

[0078]

[0079] The DTuker+ optimization objective function constructed using a data-driven multilayer perceptron network is shown in the following equation:

[0080]

[0081] In the above declaration, X represents the reconstructed noise-free hyperspectral image, S represents the sparse noise component, C1 and C2 are the core tensors of the noise-free spectral image and the registered multispectral image, respectively, and U i V i represents the factor matrices of the noise-free hyperspectral image and the registered multispectral image, respectively; P is the linear transformation matrix in the model-driven strategy; Φ(·;Θ) is the multilayer perceptron network in the data-driven strategy; Θ is the network parameter; st represents the constraint condition of the optimization objective function; ×1, ×2, and ×3 represent multiplication along modulo 1, modulo 2, and modulo 3, respectively. ‖·‖1 represents L1 norm computation. This means stretching all elements of a tensor into a long vector and then calculating the Euclidean length of that long vector.

[0082] The above objective function is solved iteratively using the alternating direction multiplier method. Introducing auxiliary variables Q1 and Q2, the objective function can be rewritten as follows:

[0083]

[0084] The solution process specifically includes the following sub-steps:

[0085] 2.1. Update core tensors C1 and C2:

[0086] 2.2.1 Under the model-driven strategy: Updating C1: With other variables fixed, C1 is solved using the following formula:

[0087]

[0088] The solution in C1 matrix form obtained from the above equation is: Where, A1=ηI A +βP T P+I A I A This represents the identity matrix of size R3×R3. Indicates the inverse of A1. `unfold` means expanding a tensor according to mode 3. For example, tensor Y∈R m×n×Z Expanding yields matrix Y∈R Z×mn (mn=m×n).

[0089] Update C2: Keeping other variables fixed, we get:

[0090]

[0091] Solve the subproblem concerning C2, the solution of which has the form C2 = B2 / η + β + 1, where,

[0092] 2.2.2 Under the data-driven strategy: Update C1: With other variables fixed, solve using the following formula:

[0093]

[0094] Since MLP networks are differentiable, this problem also has a closed-form solution: Where, A1=ηI A +βΦ T Φ+I A I A This represents the identity matrix of size R3×R3. Indicates the inverse of A1.

[0095] Update C2: With other variables fixed, solve the subproblem of C2 using the following formula:

[0096]

[0097] Solve the subproblem concerning C2, the solution of which has the form C2 = B2 / η + β + 1, where,

[0098] 2.3 Update the association modeling parameters:

[0099] 2.3.1 Under the model-driven strategy: Updating P: With other variables fixed, solve for the transformation matrix P using the following formula:

[0100]

[0101] The solution is in the form of Among them, I p This represents an identity matrix of size r3×R3.

[0102] 2.3.2 Under the data-driven strategy: Updating network parameters Θ: With other variables fixed, the multilayer perceptron network is updated by minimizing the following equation using the stochastic gradient descent (SGD) algorithm:

[0103]

[0104] 2.4 Update the multispectral image factor matrix V i Similar to step 2.2, with other variables fixed, solve for V using the standard high-order orthogonal iterative algorithm with the following formula. i Optimization issues:

[0105]

[0106] 2.5 Update auxiliary variables Q1 and Q2, fix other variables, and solve the subproblems about Q1 and Q2 using the following formula:

[0107]

[0108] This problem can be solved efficiently using the soft threshold shrinkage operator, and the solutions are as follows: Where, Δ ε [q] = sgn(q)·max(|q|-ε,0), where · represents element-wise multiplication, sgn(·) represents the sign function, q is the variable to be substituted, and ε represents the constant term substituted into the formula.

[0109] 2.6 Update the sparse noise S, keeping other variables fixed, and solve the subproblem concerning S using the following formula:

[0110]

[0111] This problem can also be solved using the soft threshold shrinkage operator, with the solution in the form S = Δ λ [YX].

[0112] 2.7 Update the noise-free hyperspectral image factor matrix U i With other variables fixed, solve for U using the standard high-order orthogonal iterative algorithm with the following formula. i Optimization issues:

[0113]

[0114] 2.8 Update the noise-free hyperspectral image X. Using the core tensor and factor matrix of the updated noise-free hyperspectral image, the hyperspectral image is reconstructed through tensor product: X = C1×1U1×2U2×3U3, resulting in the updated X.

[0115] 2.9 Update Lagrange multipliers Λ1 and Λ2. The Lagrange multiplier update formula is updated according to the standard update rule of the alternating direction multiplier algorithm framework: Λ1=Λ1+η(C1-Q1), Λ2=Λ2+η(C2-Q2).

[0116] Step 3: Determine convergence and output. Repeat step 2 until the convergence condition is met: the change in X between two consecutive iterations is less than the preset threshold tol or the maximum number of iterations t is reached. max Based on the final parameters of the updated iteration, a noise-free hyperspectral image X is reconstructed and output.

[0117] This completes and realizes hyperspectral image denoising, improves the detail quality of the image, and enhances the usability of the shaking image.

[0118] This embodiment uses the PaviaUniversity dataset as an example to illustrate the method of the present invention. The association model between the hyperspectral and multispectral core tensors is established through model-driven linear transformation or data-driven multilayer perceptron networks using the following training image pairs: a noisy hyperspectral image Y with m = 200 rows, n = 200 columns, and z = 80 bands, and a noiseless ground truth hyperspectral image. The registered multispectral image M with m = 200 rows, n = 200 columns, and p = 80 bands.

[0119] When reconstructing hyperspectral images using the trained association model, the following image pairs are used: a noisy hyperspectral image Y with m=200 rows, n=200 columns, and Z=80 bands, and a registered multispectral image M with m=200 rows, n=200 columns, and p=4 bands.

[0120] When iteratively solving the objective function using the alternating direction multiplier method, the input parameters include: the Tucker decomposition rank of the hyperspectral image [r1,r2,R3] = [180,180,3], and the Tucker decomposition rank of the multispectral image [r1,r2,r3] = [180,180,3].

[0121] The table below presents a quantitative comparison of experimental results using different denoising methods on the Pavia University dataset, simulating different noise levels based on four evaluation metrics: mean peak signal-to-noise ratio (MPSNR), mean structural similarity index (SSIM), relative error global dimension synthesis (ERGAS), and mean spectral angular distance (MSAD). The best results are indicated in bold, while the second-best results are indicated by underline. DTucker represents the hyperspectral denoising model under a model-driven strategy, and DTucker+ represents the hyperspectral denoising model under a data-driven strategy. Compared to existing hyperspectral denoising methods, the method of this invention outperforms all metrics, exhibiting higher PSNR and SSIM values ​​and lower ERGAS and MSAD values, indicating improved denoising performance. This result highlights the practical value of the method of this invention. Figure 4 This paper presents a comparison of the spatial and spectral curves of different hyperspectral denoising methods on the Pavia University dataset and under Case 4 noise conditions. LRTA and PARAFAC fail to effectively handle severe mixed noise, still exhibiting significant stripe noise and fringe artifacts. BM4D outperforms LRTA and PARAFAC visually, but suffers from spatial overfitting. While LRMR can suppress stripe and fringe noise, it is insufficient in removing strong Gaussian noise. Figure 4 In the second row, rapid spectral fluctuations still exist in the results of BM4D, LRMR, and PARAFAC, indicating that residual mixing noise has not been completely removed. LRTA smooths the spectrum in the low-frequency band but distorts information in the high-frequency band. 3DTV-RPCA outperforms the first four methods in both spatial and spectral aspects. However, the proposed DTucker and DTucker+ methods exhibit the most outstanding performance, effectively preserving spatial details and achieving a high degree of agreement between their spectral curves and the original curves.

[0122] Table 1. Quantitative comparison of experimental results of different denoising methods on the Pavia University dataset.

[0123]

[0124] The present invention has been described in detail above through embodiments, but the content described is only an exemplary embodiment of the present invention and should not be considered as limiting the scope of the present invention. The scope of protection of the present invention is defined by the claims. Any technical solutions designed by those skilled in the art using the technical solutions described in the present invention, or similar technical solutions designed by those skilled in the art under the inspiration of the technical solutions of the present invention, within the substance and scope of protection of the present invention, to achieve the above-mentioned technical effects, or equivalent changes and improvements made to the scope of the application, should still fall within the patent protection scope of the present invention. It should be noted that, for clarity, descriptions of some components and processes that are not directly and obviously related to the scope of protection of the present invention but are known to those skilled in the art have been omitted in the description of the present invention.

Claims

1. A multimodal denoising method for hyperspectral images, characterized in that, Includes the following steps: S1. Using Tucker decomposition, low-rank tensor representations are performed on hyperspectral images and registered multispectral images respectively, and the core tensors of both are extracted. S2 establishes a correlation model between hyperspectral and multispectral core tensors through model-driven linear transformation or data-driven multilayer perceptron networks. S3. Based on the core tensor of the hyperspectral image, the core tensor of the multispectral image, and the correlation model, an optimization objective function is constructed. The core tensor, factor matrix, and correlation model parameters in the optimization objective function are solved iteratively using the alternating direction multiplier method. S4 reconstructs the denoised hyperspectral image using the optimized hyperspectral core tensor and factor matrix.

2. The method according to claim 1, characterized in that, In step S2, the model-driven linear transformation constraint association model is as follows: a direct mapping function is established between the core tensor of the noisy hyperspectral image and the core tensor of the registered multispectral image through the linear transformation matrix, wherein the direct mapping function is the regularization term of the linear transformation matrix.

3. The method according to claim 1, characterized in that, In step S2, the association model obtained by the data-driven multilayer perceptron network is as follows: the multilayer perceptron network is used to learn and approximate the nonlinear mapping function between the core tensor of the noisy hyperspectral image and the core tensor of the registered multispectral image. The multilayer perceptron network contains at least one linear layer and a LeakyReLu activation function, and is trained and solved using the stochastic gradient descent algorithm.

4. The method according to claim 1, characterized in that, In step S3, the optimization objective function is expressed as minimizing the reconstruction error and sparse noise term of the hyperspectral image. L 1-norm, core tensor of noisy hyperspectral images L 1-norm, core tensor of registered multispectral images L The objective function for optimizing the L1 norm and the sum of the modeling errors of the core tensor correlation is as follows: The optimization objective function constructed through model-driven linear transformation is shown in the following equation: ； The optimization objective function for constructing a data-driven multilayer perceptron network is shown in the following equation: ； In the above formula, X represents the reconstructed noise-free hyperspectral image, S represents the sparse noise component, C1 and C2 are the core tensors of the noise-free spectral image and the registered multispectral image, respectively, and U i V i denoted by , P represents the linear transformation matrix in the model-driven strategy, Φ(·;Θ) represents the multilayer perceptron network in the data-driven strategy, and Θ represents the network parameters. st The constraints representing the optimization objective function are ×1, ×2, and ×3, which represent multiplication along modulo 1, modulo 2, and modulo 3, respectively, and ||·||1 represents... L 1-normal form computation This means stretching all elements of a tensor into a long vector, and then calculating the Euclidean length of that long vector. It is the regularization parameter.

5. The method according to claim 4, characterized in that, The objective function is solved iteratively using the alternating direction multiplier method, which includes the following steps: The input consists of two image pairs, Y and M, to be processed, and input parameters, where Y is a noisy hyperspectral remote sensing image with m rows, n columns, and z bands, and M is a registered multispectral remote sensing image with m rows, n columns, and p bands. Input parameters include: the maximum number of iterations t. max Preset threshold tol, Tucker decomposition rank of hyperspectral image Tucker decomposition rank of multispectral images and model regularization parameters Penalty parameters ; Initialize each variable: sparse noise Core tensor auxiliary variables Lagrange multipliers ; Introducing auxiliary variables and The latter two optimization objective functions are rewritten as follows: ； ； Update the core tensor sequentially and The associated modeling parameters include the linear transformation matrix P in the model-driven strategy or the multilayer perceptron network parameters Θ and multispectral factor matrix V in the data-driven strategy. i 、 Core Tensor Auxiliary variables Q 1 and core tensor Auxiliary variables Q 2 、 sparse noise Hyperspectral factor matrix U i The reconstructed noiseless hyperspectral image X, and its application to linear constraints. = The dual variables, the Lagrange multipliers Λ1, and the linear constraints should be applied. = The dual variable Λ2 is used until the maximum number of iterations is reached or the change in X between two adjacent iterations is less than the preset threshold tol.

6. The method according to claim 5, characterized in that, Update core tensor and The details are as follows: Under model-driven strategy: Update With other variables fixed, solve using the following formula. : ， The above formula is obtained The solution in matrix form is ,in, , Indicates the size is The identity matrix, express The reverse, , unfold means expanding the tensor according to mode 3; renew By fixing other variables, we get: ； Seeking answers regarding The subproblem of has a solution of the form of . ,in, ; Under a data-driven strategy: Update With other variables fixed, solve using the following formula: ； The above formula is obtained The solution in matrix form is ,in, A1 -1 Indicates the inverse of A1. ; renew With other variables fixed, solve for the equation regarding... Subproblems: ； Seeking answers regarding The subproblem of has a solution of the form of . ,in, .

7. The method according to claim 5, characterized in that, The specific steps for updating the association modeling parameters are as follows: Under model-driven strategy: Update With other variables fixed, solve for the transformation matrix using the following formula. : ； The solution is in the form of ,in, Indicate size An identity matrix of 3D; Under a data-driven strategy: updating network parameters Θ: keeping other variables fixed, the stochastic gradient descent algorithm is used to solve the minimization problem of Θ using the following formula to update the multilayer perceptron network: 。 8. The method according to claim 5, characterized in that, Update auxiliary variables , and sparse noise Specifically as follows: Update auxiliary variables , With other variables fixed, solve for the equation regarding... and Subproblems: ； sparse noise With other variables fixed, solve for the equation regarding... Subproblems: ； By solving the above variables using the soft threshold shrinkage operator, we obtain the following results: , , ,in, , This represents element-wise multiplication. Represents a symbolic function. q To substitute variables, This represents the constant term substituted into the formula.

9. The method according to claim 5, characterized in that, Update the multispectral image factor matrix Hyperspectral image factor matrix The reconstructed noiseless hyperspectral image X is as follows: Update the multispectral image factor matrix With other variables fixed, solve using the following formula: ； Update the hyperspectral image factor matrix With other variables fixed, solve using the following formula: ； Solving for the problem using a standard high-order orthogonal iterative algorithm. and The optimization problem; The reconstructed noise-free hyperspectral image X is: 。 10. The method according to claim 5, characterized in that, Update Lagrange multipliers , The Lagrange multiplier update formula, updated according to the standard update rule of the alternating direction multiplier algorithm framework, is as follows: , .

Images