Hyperspectral and multispectral image fusion algorithm

Abstract

The invention discloses a hyperspectral and multispectral image fusion algorithm. The algorithm comprises the following steps: step 1, constructing a fusion model based on regular term constraint; step 2, constructing a multi-view clustering model of hypergraph manifold and low-rank tensor constraint; 3, performing spatial and spectral manifold learning based on a multi-view clustering model; 4, introducing a regular hyperspectral and multispectral image fusion model by utilizing the space and spectral manifold constraint obtained by learning; and 5, solving a fusion problem by adopting an ADMM algorithm. On the basis of inspiration of a low-dimensional manifold structure in a hyperspectral image, by considering space and spectrum height correlation and utilizing manifold constraints and high-dimensional characteristics to cluster an internal low-dimensional space of the hyperspectral image, a low-dimensional structure of the hyperspectral image in space and spectrum dimensions is described, and multi-view constraints are introduced to ensure the accuracy of an incidence matrix. A solution space of a to-be-restored hyperspectral image is constrained, self-expression constraints of a space and a spectral low-dimensional space are introduced, and a proposed algorithm is optimized by using an alternating direction multiplier method.

Description

Technical Field

[0001] The present invention relates to the technical fields of spectral remote sensing and image processing, and in particular to a hyperspectral and multispectral image fusion algorithm. Background Technique

[0002] During the imaging process of hyperspectral images, due to the technical limitations of imaging devices, there is a mutual restriction between spatial resolution and spectral resolution, making it difficult for imaging sensors to simultaneously obtain images with high spatial resolution and high spectral resolution. This reduces the accurate extraction and expression of the spatial and spectral features of hyperspectral images and limits the subsequent analysis and application of the images.

[0003] In recent years, with the rapid development of optoelectronic detection technology, sensors can obtain high-resolution multispectral images and low-resolution hyperspectral images of the same scene. Therefore, fusing hyperspectral images and multispectral images has become an effective way to obtain high-spatial-resolution hyperspectral images. Early fusion methods mainly dealt with the pan-sharpening of panchromatic images and hyperspectral images. However, due to the lack of spectral information in panchromatic images, serious information distortion occurred in the fusion results. To overcome this problem, fusion methods based on multispectral images and hyperspectral images emerged as the times require. For example, methods based on sparse representation / dictionary learning and spectral unmixing model the spatial and spectral information of data in vector space. The disadvantage of these methods is that they ignore the high-dimensional structure of hyperspectral images in the spatial and spectral dimensions, resulting in obvious deviations in the reconstructed structure.

[0004] Fusion methods based on tensor decomposition can avoid the above problems and show great advantages in restoring spatial and spectral information. Inspired by the nonlinear expression and feature learning advantages of deep models, convolutional neural networks have received increasing attention in the fusion of hyperspectral and multispectral images. However, the hyperparameters in the network reduce the fusion accuracy. In addition, the interpretability of the network model is poor, reducing the transfer ability of the model. A network model trained on one type of hyperspectral data is difficult to obtain good fusion performance on other types of images, and the stability of the model is easily affected by the training data. More importantly, current methods ignore the multi-view prior information of hyperspectral images in the spatial and spectral dimensions during the design of constraint terms or the learning of models, resulting in a reduced accuracy of the correlation expression between known data and unknown images. Summary of the Invention

[0005] The object of the present invention is to provide a hyperspectral and multispectral image fusion algorithm for the above-mentioned existing problems.

[0006] The technical solution adopted by the present invention is as follows: A hyperspectral and multispectral image fusion algorithm includes the following steps: Step 1: Construct a fusion model based on regular term constraints; Step 2: Construct a multi-view clustering model with hypergraph manifold and low-rank tensor constraints; Step 3: Perform spatial and spectral manifold learning based on the multi-view clustering model in Step 2; Step 4: Incorporate the spatial and spectral manifold constraints learned in Step 3 into the regularized hyperspectral and multispectral image fusion model in Step 1; Step 5: Use the ADMM algorithm to solve the fusion problem in Step 4.

[0007] In Step 1, it includes: Step 1.1: Express the relationship between the multispectral and hyperspectral images through the observation equation, and define the hyperspectral image with high spatial resolution to be restored; Step 1.2: Design the objective function, introduce the regularization term, and solve the hyperspectral image with high spatial resolution to be restored by minimizing the objective function.

[0008] In Step 1.1, the observed values of the known multispectral and hyperspectral images are expressed as: , (1) where, is the hyperspectral image to be restored, with a spatial resolution of , and a spectral band of B ; is the multispectral image with high spatial resolution, with a spectral band of ; is the hyperspectral image with low spatial resolution, with a spatial resolution of ; and are the two-dimensional spatial degradation matrices, which are usually modeled by the circular convolution operator and the downsampling matrix; is the spectral degradation matrix, represents the tensor modulus - i product; In Step 1.2, solve by minimizing the following objective function: (2) where, and are the regularization parameters, represents the square of the F-norm, and respectively represent the spatial and spectral information constraints of the hyperspectral image to be restored.

[0009] In Step 2, it includes: Step 2.1: Treat multiple bands of the multispectral image as independent views and construct a low-rank tensor clustering model; Step 2.2: Introduce the regularization term constraint of the hypergraph manifold to enhance the accuracy of the correlation matrix and capture the complex non-linear relationship in the spatial dimension; Step 2.3: Use ADMM to optimize the model and solve the correlation matrix and error tensor step by step.

[0010] In Step 2.1, regarding multiple spectral bands of the multispectral image as different views, the multi-view clustering model based on low-rank tensors is expressed as: (3) where, is the i th band, is the correlation matrix of the i th band, is the corresponding error matrix; the tensor composed of is denoted as , and ; is the regularization parameter, is the rotational tensor nuclear norm, is the (2,1)-norm of the matrix; In Step 2.2, introduce the regularization term of the hypergraph manifold to constrain the correlation matrix for each perspective as follows: (4) where, the i th term is the hypergraph constraint term of the i th band or the th view, is the corresponding hyper-Laplacian matrix; Introduce auxiliary variables, and such that and , and formula (4) is equivalent to the following form: (5) This problem is equivalently transformed into the following unconstrained minimization problem: (6) where, and are Lagrange multipliers and , is an auxiliary variable and , is the penalty term parameter; In step 2.3, each variable in formula (6) is alternately solved by fixing other variables: (I) Optimization : Fix other variables and optimize The objective function of is: (7) Formula (7) has a closed-form solution as follows: (8) where I is the identity matrix of the same size as ; (II) Optimization : Fix other variables and optimize The objective function of is: (9) where D is formed by concatenating along the columns of matrix and its closed-form solution is: (10) (III) Optimization : Fix other variables and optimize The objective function of is: (11) This formula has a closed-form solution as follows: (12) where I is the identity matrix of the same size as ; (IV) Optimization : Fix other variables and optimize The objective function of is: (13) Solve by the singular value threshold method of t-SVD; (V) Update the Lagrange multiplier: (14) where is the update parameter scale factor of.

[0011] In step 3, it includes: Step 3.1: Extract the spatial dimension correlation matrix through the multi-view clustering model; Step 3.2: Perform slice analysis on the spectral dimension of the hyperspectral image, construct a spectral dimension manifold expression, and fuse the multi-view results to generate a spectral correlation matrix.

[0012] In Step 3.1, by solving the correlation matrix of the multispectral image in each band , the associated matrix in the spatial dimension is obtained as follows: (15) where is the average value of the associated matrix of each band of the multispectral image in the spatial dimension; In Step 3.2, the multi-view clustering model is used to learn the manifold expression in the spectral dimension from the known hyperspectral image as follows: (16) where is the th side slice of j , is the error matrix corresponding to the j th side slice; is the correlation matrix corresponding to , and the tensor composed of is , and , and are auxiliary variables such that and , , and are Lagrange multipliers; the solution method of formula (6) is used to optimize (16) to obtain h matrices which are fused to obtain the associated matrix in the spectral dimension as follows: (17) where is the average value of the associated matrix of each band of the hyperspectral image in the spectral dimension.

[0013] In Step 4, it includes: Step 4.1, taking the spatial and spectral association matrices as regular terms to reconstruct the objective function of the fusion model; Step 4.2, introducing auxiliary variables to transform the constrained optimization problem into an unconstrained form.

[0014] In Step 4.1, by solving the optimal and , the learned and constraints are introduced into the hyperspectral and multispectral image fusion algorithm formula (2) to obtain the following expression: (18) Introduce auxiliary variables , we get: (19) By introducing auxiliary variables and , (19) can be equivalently expressed as: (20).

[0015] In step 5, it includes: Step 5.1: Decompose the augmented Lagrangian function into multiple sub-problems, and alternately optimize the image variables, auxiliary variables, and Lagrange multipliers; Step 5.2: Quickly update the variables through the closed-form solution or the singular value threshold method to ensure the convergence and computational efficiency of the algorithm.

[0016] In step 5, based on ADMM, the augmented Lagrangian function corresponding to formula (20) is as follows: (21) where and are Lagrange multipliers, is a non-negative scalar parameter.

[0017] Based on the iterative minimization strategy, (21) can be equivalently solved as the following sub-problems: (I) Optimize and : Fix other variables, and the optimization form with respect to and is: (22) (II) Optimize : Fix other variables, and the optimization function with respect to is: (23) Formula (23) has a closed-form solution as follows: (24) (III) Optimize : Fix other variables, and the optimization function with respect to is: (25) Expand (25) along the third dimension, we have (26) Similar to the solution method of (23), its closed-form solution is: (27) Among them, I is an identity matrix with the same size.

[0018] (IV) Optimization : Fix other variables. Regarding the optimization function is: (28) Let , and then expand formula (28) along the third dimension, and we can get (29) Similar to the solution method of (23), its closed-form solution is: (30) (V) Update Lagrange multipliers and : (31).

[0019] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are as follows: A hyperspectral and multispectral image fusion algorithm provided by the present invention. First, inspired by the low-dimensional manifold structure in hyperspectral images, by considering the high spatial and spectral correlations, the internal low-dimensional space is clustered using manifold constraints and high-dimensional characteristics, and a multi-view clustering framework of hypergraph manifolds and low-rank tensors is proposed. Secondly, in order to characterize the low-dimensional structure of hyperspectral images in the spatial and spectral dimensions, the proposed multi-view clustering framework is used to learn the low-dimensional information in the spatial and spectral dimensions. During this process, multi-view constraints are introduced to ensure the accuracy of the association matrix. Then, in order to constrain the solution space of the hyperspectral image to be restored, self-expression constraints of the spatial and spectral low-dimensional spaces are introduced, and a hyperspectral and multispectral image fusion algorithm based on hypergraph manifolds and tensor low-rank multi-view clustering is proposed, and the proposed algorithm is optimized using the alternating direction multiplier method. Finally, the experimental results on hyperspectral data verify the fusion performance of the present invention. Description of the Drawings

[0020] The present invention will be described by way of examples with reference to the drawings, where: Figure 1 is the schematic flow diagram of the present invention.

[0021] Figure 2 is the comparison of the restoration results of the embodiments.

[0022] Figure 3 is the comparison of the reconstruction errors of the embodiments. Detailed implementation manners

[0023] The present invention will be described in detail below with reference to the accompanying drawings.

[0024] All features disclosed in this specification, or all steps in the disclosed methods or processes, except for mutually exclusive features and / or steps, can be combined in any manner.

[0025] Any feature disclosed in this specification, unless specifically described, can be replaced by other equivalent or similar-purpose alternative features. That is, unless specifically described, each feature is only an example of a series of equivalent or similar features.

[0026] A hyperspectral and multispectral image fusion algorithm, as Figure 1 shown, includes the following steps: Step 1: Construct a fusion model based on regular term constraints; establish a mathematical model to fuse low-spatial-resolution hyperspectral and high-spatial-resolution multispectral images.

[0027] Step 2: Construct a multi-view clustering model with hypergraph manifold and low-rank tensor constraints; utilize the multi-band characteristics of the multispectral image to mine the low-dimensional structures in the spatial and spectral dimensions.

[0028] Step 3: Perform spatial and spectral manifold learning based on the multi-view clustering model in Step 2; learn the low-dimensional manifold structures in the spatial and spectral dimensions from the multispectral and hyperspectral data.

[0029] Step 4: Introduce the spatial and spectral manifold constraints learned in Step 3 into the regular hyperspectral and multispectral image fusion model in Step 1; apply the learned low-dimensional structure constraints to the image fusion problem.

[0030] Step 5: Use the ADMM algorithm to solve the fusion problem in Step 4; efficiently solve the fusion model to obtain a high-resolution hyperspectral image.

[0031] Hyperspectral images have rich spectral information but low spatial resolution; multispectral images have high spatial resolution but few spectral bands. The goal of fusing the two is to generate high spatial-spectral resolution images. Hypergraph manifolds allow hyperedges to connect multiple nodes and can model the group spatial correlations between pixels. Assuming that hyperspectral images exist on a low-dimensional manifold, the data is mapped to a low-dimensional space through manifold learning constraints, preserving its essential structure. Hypergraph manifolds construct the Laplacian matrix of the hypergraph, which serves as a regularization term to constrain the multi-view clustering process, ensuring that the association matrix reflects the complex dependencies in the spatial dimension. The tensor representation of low-rank tensor clustering treats multiple bands of the multispectral image as different views and constructs a three-dimensional tensor (space × spectrum × view), avoiding the loss of structural information caused by vectorization. The low-rank constraint assumes that there is a shared low-dimensional structure between different bands, and through the minimization of the tensor nuclear norm, global consistency features are extracted, suppressing noise and redundancy. Multi-view collaborative learning models each band independently, and through low-rank tensor constraints, information complementarity between views is achieved, enhancing the robustness of the association matrix. The joint optimization framework and ADMM solver fuse the data fidelity term, hypergraph manifold regularization term, and low-rank tensor term to form a multi-objective optimization problem. The complex problem is decomposed into multiple sub-problems, and the global optimal solution is gradually approximated through alternating optimization and multiplier updates.

[0032] Furthermore, hypergraph manifolds solve spatial non-linearity, and low-rank tensors integrate spectral multi-views, forming "spatial-spectral two-dimensional optimization", ensuring that the fusion result not only retains spatial details but also maintains spectral consistency. The two complement each other to avoid information bias in a single dimension. The multi-view design avoids information bias from a single perspective, the low-rank tensor ensures consistency between views, and the hypergraph manifold further constrains the correlations in the spatial dimension, forming a closed-loop optimization of "multi-view → low-rank integration → hypergraph correction". The regularization parameter is adaptively adjusted by ADMM to ensure the balance between the fidelity term and the prior constraint. ADMM decomposes the complex problem that fuses fidelity, hypergraph constraint, and low-rank constraint into sub-problems that can be processed in parallel, and combines the closed-form solution to accelerate convergence, achieving the unity of accuracy and efficiency.

[0033] As specific steps: Step 1: Construct a fusion model based on regularization term constraints The observed values of the known multispectral and hyperspectral images can be expressed as: , (1) where is the hyperspectral image to be restored, with a spatial resolution of and a spectral band of B ; is the multispectral image with high spatial resolution, and the spectral band is ; is the hyperspectral image with low spatial resolution, and the spatial resolution is ; and are degenerate matrices in two spatial dimensions and are usually modeled by a circular convolution operator and a downsampling matrix; is a spectral degradation matrix, denotes the tensor modulus - i product.

[0034] It is difficult to directly reconstruct a high - spatial - resolution hyperspectral image from (1). Therefore, it can be solved by minimizing the following objective function : (2) where and are regularization parameters, denotes the square of the F - norm, and respectively represent the spatial and spectral information constraints of the hyperspectral image to be restored.

[0035] Step 2: Construct a multi - view clustering model with hypergraph manifold and low - rank tensor constraints Regarding the multiple spectral bands of the multispectral image as different views, the multi - view clustering model based on low - rank tensors is expressed as: (3) where is the th band of i , is the correlation matrix of the i th band; the tensor composed of is denoted as , and ; is a regularization parameter, is the rotational tensor nuclear norm, is the (2,1) - norm of the matrix. In the experiment, is set. However, this method ignores the low - dimensional manifold structure of the multispectral image in the spatial dimension. Therefore, a regularization term of the hypergraph manifold is introduced to constrain the correlation matrix of each view as follows: (4) where the i th term is the hypergraph constraint term of the i th band or the th view, is the regularization parameter for the hypergraph constraint, which is set in the experiment . To facilitate the solution of (4), auxiliary variables are introduced, and such that and . Formula (4) is equivalent to the following form: (5) This problem is equivalently transformed into the following unconstrained minimization problem: (6) where and are Lagrange multipliers and , is an auxiliary variable and , is the penalty term parameter, which is initialized as and in the experiment, . In the ADMM framework, each variable in (6) is alternately solved by fixing other variables.

[0036] (I) Optimize : Fix other variables and optimize the objective function of as follows: (7) Formula (7) has a closed-form solution as follows: (8) where I is the identity matrix of the same size as ; (II) Optimize : Fix other variables and optimize the objective function of as follows: (9) where D is formed by concatenating along the columns of matrix , and its closed-form solution is: (10) (III) Optimize : Fix other variables and optimize the objective function of as follows: (11) This formula has a closed-form solution as follows: (12) where I is the same size as Identity matrices of the same size; (IV) Optimization : Fix other variables and optimize the objective function as: (13) Solve it by the singular value threshold method of t-SVD; (V) Update the Lagrange multipliers: (14) where is the scaling factor for updating the parameter In the experiment, set .

[0037] Step 3: Spatial and spectral manifold learning based on multi-view clustering By solving the correlation matrix of the multi-spectral image in each band , obtain the associated matrix in the spatial dimension as follows: (15) where is the average value of the associated matrix of each band of the multi-spectral image in the spatial dimension; In Step 3.2, use the multi-view clustering model to learn the manifold expression in the spectral dimension from the known hyperspectral image The model is as follows: (16) where is the th side slice of j , and is the error matrix corresponding to the j th side slice; is the correlation matrix corresponding to , and the tensor composed of is , and , and are auxiliary variables such that and , , and are Lagrange multipliers; optimize (16) using the solution method of formula (6) to obtain h matrices Fuse them to obtain the associated matrix in the spectral dimension as follows: (17) where It is the average value of the correlation matrix for each band in the spectral dimension of the hyperspectral image.

[0038] Step 4: Utilize the learned spatial and spectral manifold constraints to regularize the hyperspectral and multispectral image fusion problem By solving the optimal and , introduce the learned and constraints into the hyperspectral and multispectral image fusion algorithm (2), obtaining the following expression: (18) Introduce the auxiliary variable , obtaining: (19) Since (19) is a constrained optimization problem, resort to the ADMM framework. First, by introducing the auxiliary variables and , (19) is equivalently expressed as: (20).

[0039] Step 5: Solve the fusion problem using the ADMM algorithm Based on ADMM, the augmented Lagrangian function corresponding to formula (20) is as follows: (21) where and are Lagrange multipliers, is a non - negative scalar parameter, initialized as , .

[0040] Based on the iterative minimization strategy, (21) can be equivalently solved as the following sub - problems: (I) Optimize and : Fix other variables, and the optimization form with respect to and is: (22) (II) Optimize : Fix other variables, and the optimization function with respect to is: (23) Formula (23) has a closed - form solution, as follows: (24) (III) Optimize : Fix other variables, with respect to The optimization function is: (25) Unfolding along the third dimension (25), we have (26) Similar to the solution method of (23), its closed-form solution is: (27) where I is an identity matrix of the same size.

[0041] (IV) Optimization : Fixing other variables, the optimization function with respect to is: (28) Let , and then unfold formula (28) along the third dimension, we can obtain (29) Similar to the solution method of (23), its closed-form solution is: (30) (V) Update Lagrange multipliers and : (31).

[0042] As a specific algorithm example: Input: Hyperspectral image , multispectral image and .

[0043] Initialization: Initialize by upsampling each band , , , , and .

[0044] 1: (a) Learn the manifold expression matrices of the spatial and spectral dimensions and ; 2: (b) Hyperspectral and multispectral image fusion algorithm based on hypergraph manifold and low-rank tensor clustering 3: for do 4: (b1) Update and using formula (22); 5: (b2) Update using formula (24) ; 6: (b3) Update using formula (27) ; 7: (b4) Update using formula (30) ; 8: (b5) Update the Lagrange multipliers using formula (31) and ; 9: (b6) Update ; 10: (b7) ; 11: while ; 12: end for Output: Hyperspectral image with high spatial resolution 。

[0045] Embodiment The data Stuffed_toys used in the present invention comes from a publicly available hyperspectral dataset. In the experiment, the performance of the algorithm is verified using the fusion result of 8-fold downsampling.

[0046] To quantitatively and qualitatively evaluate the fusion effect, the present invention selects the Sparse Spatial-Spectral Representation (SSSR) method, the Bayesian Sparse Representation (BSR) method, the Coupled Spectral Unmixing (CSU) method, and the Cross-attention in Coupled Unmixing Network (CUCaNet) method as comparison methods, and uses these comparison methods to compare the fusion results with the results of the present invention. The present invention uses four quality evaluation indicators: PSNR (dB), RMSE, SSIM, and SAM. At the same time, the running time is also compared. The quantitative analysis results are shown in Table 1, where the bold indicates the best result, and the visual comparison results are as Figure 2 shown.

[0047] Table 1

[0048] As can be seen from Table 1, compared with the four comparison methods, the method of the present invention has higher PSNR and SSIM, and smaller RMSE and SAM values, which indicates that the method of the present invention has obvious advantages in restoring the structure of data and is superior to other methods in spectral information reconstruction. In addition, the running time of this method is the shortest, thus reducing the power consumption of the algorithm in practical applications.

[0049] Visual comparison of the fusion results of the Stuffed-toys data at a 8-fold sampling ratio in the 640nm band is as Figure 2 and Figure 3 shown. From Figure 2 it can be seen that all methods can obtain good restoration results. However, from the Figure 3 magnified results within the red box in

[0050] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0051] The present invention is not limited to the specific embodiments described above. The present invention extends to any new feature or any new combination disclosed in this specification, as well as any new combination of the steps of any new method or process disclosed.

Claims

1. A hyperspectral and multispectral image fusion algorithm, characterized in that: It includes the following steps: Step 1: Construct a fusion model based on regular term constraints; Step 2: Construct a multi-view clustering model with hypergraph manifold and low-rank tensor constraints; Step 3: Conduct spatial and spectral manifold learning based on the multi-view clustering model in Step 2; Step 4: Introduce the spatial and spectral manifold constraints learned in Step 3 into the regular hyperspectral and multispectral image fusion model in Step 1; Step 5: Use the ADMM algorithm to solve the fusion problem in Step 4.

2. The hyperspectral and multispectral image fusion algorithm according to claim 1, wherein: In Step 1, it includes: Step 1.1: Express the relationship between the multispectral and hyperspectral images through the observation equation, and define the hyperspectral image with high spatial resolution to be restored; Step 1.2: Design the objective function, introduce the regularization term, and solve the hyperspectral image with high spatial resolution to be restored by minimizing the objective function.

3. The hyperspectral and multispectral image fusion algorithm according to claim 1, characterized in that: In Step 2, it includes: Step 2.1: Regard multiple bands of the multispectral image as independent views and construct a low-rank tensor clustering model; Step 2.2: Introduce the regular term constraint of the hypergraph manifold to enhance the accuracy of the correlation matrix and capture the complex non-linear relationship in the spatial dimension; Step 2.3: Use the ADMM optimization model to solve the correlation matrix and error tensor step by step.

4. The hyperspectral and multispectral image fusion algorithm according to claim 1, characterized in that: In Step 3, it includes: Step 3.1: Extract the spatial dimension correlation matrix through the multi-view clustering model; Step 3.2: Conduct slice analysis on the spectral dimension of the hyperspectral image, construct a spectral dimension manifold expression, and fuse the multi-view results to generate a spectral correlation matrix.

5. The hyperspectral and multispectral image fusion algorithm according to claim 1, characterized in that: In Step 4, it includes: Step 4.1: Use the spatial and spectral correlation matrices as regular terms to reconstruct the objective function of the fusion model; Step 4.2: Introduce auxiliary variables to transform the constrained optimization problem into an unconstrained form.

6. The hyperspectral and multispectral image fusion algorithm according to claim 1, characterized in that: In Step 5, it includes: Step 5.1: Decompose the augmented Lagrangian function into multiple sub-problems, and alternately optimize the image variable, auxiliary variable, and Lagrange multiplier; Step 5.2: Quickly update the variables through the closed-form solution or the singular value threshold method to ensure the convergence and computational efficiency of the algorithm.

7. The hyperspectral and multispectral image fusion algorithm according to claim 1, characterized in that: In Step 1, the observed values of the known multispectral and hyperspectral images are expressed as: ， (1) Among them, is the hyperspectral image to be restored, with a spatial resolution of , and a spectral band of B ; is a multispectral image with high spatial resolution, and the spectral band is ; is a hyperspectral image with low spatial resolution, and the spatial resolution is ; and are two spatial - dimensional degradation matrices, which are usually modeled by a circular convolution operator and a down - sampling matrix; is a spectral degradation matrix, represents the tensor modulus - i product; Solve by minimizing the following objective function : (2) Among them, and are regularization parameters, represents the square of the F-norm, and represent the spatial and spectral information constraints of the hyperspectral image to be restored, respectively.

8. The hyperspectral and multispectral image fusion algorithm according to claim 7, wherein: In step 2, regarding multiple spectral bands of the multispectral image as different views, the multi-view clustering model based on low-rank tensors is expressed as: (3) Among them, is the i th band, is the correlation matrix of the i th band, is the corresponding error matrix; the tensor composed of is represented as , and ; is the regularization parameter, is the rotational tensor nuclear norm, is the (2,1)-norm of the matrix; Introduce a regularization term for the hypergraph manifold to constrain the association matrix for each perspective as follows: The constraints are as follows: (4) Among them, item is the hypergraph constraint item of the i th band or the i th view, is the corresponding hyper-Laplacian matrix; is the regularization parameter of the hypergraph constraint; Introduce auxiliary variables, and such that and Formula (4) is equivalent to the following form: (5) This problem is equivalently transformed into the following unconstrained minimization problem: (6) Among them, and are Lagrange multipliers and , is an auxiliary variable and , is the penalty term parameter; Alternately solve each variable in Equation (6) by fixing other variables: (I) Optimization : Fix other variables and optimize The objective function is as follows: (7) Equation (7) has a closed-form solution as follows: (8) Among them, I is the identity matrix with the same size as (II) Optimization : Fix other variables and optimize The objective function is as follows: (9) Among them, D is formed by connecting along the columns of the matrix and its closed-form solution is: (10) (III) Optimization : Fix other variables and optimize The objective function is as follows: (11) This formula has a closed-form solution as follows: (12) Among them, I is the identity matrix of the same size as (IV) Optimization : Fix other variables and optimize The objective function is as follows: (13) Solve it through the singular value threshold method of t-SVD; (V) Update the Lagrange multiplier: (14) Among them, is the scaling factor for updating the parameter .

9. The hyperspectral and multispectral image fusion algorithm according to claim 8, wherein: In step 3, by solving the correlation matrix of the multispectral image in each band , the associated matrix in the spatial dimension is obtained as follows: (15) Among them, is the average value of the correlation matrix associated with each band in the spatial dimension of the multispectral image; Learn the manifold representation of the spectral dimension from known hyperspectral images using a multi-view clustering model The model is as follows: (16) Among them, is the j th side slice, j and is the correlation matrix corresponding to , and , and are and auxiliary variables such that , and are h Lagrange multipliers; optimizing (16) using the solution method of formula (6) gives a matrix Fusing them gives the correlation matrix in the spectral dimension as follows: (17) Among them, is the average value of the correlation matrix associated with each band in the spectral dimension of the hyperspectral image.

10. The hyperspectral and multispectral image fusion algorithm according to claim 9, characterized in that: In step 4, by solving the optimal and , the learned and constraints are introduced into the hyperspectral and multispectral image fusion algorithm formula (2) to obtain the following expression: (18) Introduce auxiliary variables , we get: (19) By introducing auxiliary variables and , (19) can be equivalently expressed as: (20)。

Images