Remote sensing image fusion method based on spatial-spectral tensor subspace representation

Abstract

The present invention relates to a remote sensing image fusion method based on spatial-spectral tensor subspace representation, which belongs to the field of remote sensing image fusion. The method represents the input LR-HSI, HR-MSI and HR-HSI as three-dimensional tensors. A spatial observation model and a spectral observation model based on super-Laplacian prior are constructed to effectively characterize the relationship between the input image and the target HR-HSI. The spatial-spectral tensor subspace representation model is used to decompose the HR-HSI into a low-dimensional coefficient tensor, a spatial subspace tensor and a spectral subspace tensor. The spatial subspace tensor and the spectral subspace tensor are estimated from the HR-MSI and LR-HSI respectively by the tensor singular value decomposition algorithm T-SVD, and the coefficient tensor is optimized and solved using sparse regularization constraints and proximal alternating minimization algorithm. The estimated subspace tensor and coefficient tensor are input into the tensor subspace representation model to reconstruct the HR-HSI.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image fusion, and relates to a remote sensing image fusion method based on spatial-spectral tensor subspace representation. Background Art

[0002] In recent years, the rapid development of Earth observation technology has greatly enriched the variety of remote sensing imagery. However, due to limitations such as satellite orbits, observation modes, and sensor performance, the spatial and spectral resolutions of a single sensor are contradictory, unable to simultaneously achieve "clear vision" and "accurate calculations," making it difficult to accurately depict the spatial patterns and evolution of complex geological phenomena. Multi-source remote sensing imagery is complementary and cooperative in terms of spatial and spectral resolution. Therefore, the use of multi-source multispectral and hyperspectral image fusion technology to organically fuse the spatial and spectral complementary information from high-spatial-resolution multispectral images and low-spatial-resolution hyperspectral images from different sensors to generate hyperspectral images with high spatial resolution can overcome the limitations of single-sensor observations and achieve a more accurate representation of ground object information.

[0003] However, existing fusion methods based on subspace representation still have the following problems:

[0004] (1) The spectral observation model in the existing fusion method is overly dependent on the unknown spatial blur kernel. The existing spectral observation model needs to use some empirically set spatial blur kernel to construct the relationship between the target high spatial resolution hyperspectral image (HR-HSI) and the input hyperspectral image (LR-HSI). This is easily affected by human factors and reduces the efficiency of the fusion model.

[0005] (2) The subspace low-rank representation methods used in existing fusion methods are incomplete and inefficient in mining and utilizing the prior knowledge of the target HR-HSI. Most existing subspace representation methods focus only on spectral similarity and ignore spatial self-similarity. A few methods use tensor subspace representation technology to simultaneously characterize the low-rank properties of space and spectrum, but they treat these two low-rank properties equally and cannot fully capture the differences between them. Summary of the Invention

[0006] In view of this, the object of the present invention is to provide a remote sensing image fusion method based on spatial-spectral tensor subspace representation.

[0007] To address the problem of over-reliance on unknown spatial blur kernels faced by existing fusion methods, a spectral observation model based on super-Laplacian prior is used to describe the relationship between the input image and the target fused image. This model implements super-Laplacian regularization (l p(0 < p < 1) constraint is used to describe the relationship between the input image and the target fused image. This process has certain statistical significance and no longer depends on the spatial blur kernel, so it can effectively characterize the relationship between the input image and the target HR-HSI.

[0008] Aiming at the problems of insufficient comprehensive utilization and low efficiency in mining and using prior knowledge faced by existing fusion methods, a spatial-spectral tensor subspace representation model is adopted. This model decomposes the target HR-HSI into a low-dimensional coefficient tensor and two low-dimensional subspace tensors. Among them, these two low-dimensional subspace tensors are the spectral subspace tensor and the spatial subspace tensor respectively. The spatial-spectral tensor subspace representation model allows the spectral and spatial subspace tensors to have different subspace dimensions to depict the different low-rank structures of the target HR-HSI in the spatial domain and the spectral domain. The low-rank characteristics of the target HR-HSI in the spatial domain and the spectral domain have certain differences. Therefore, it is very necessary to model it with subspace tensors of different dimensions.

[0009] Combining a new spectral observation model based on the hyper-Laplacian prior and the spatial-spectral tensor subspace representation model, a remote sensing image fusion method based on the spatial-spectral tensor subspace representation is constructed to achieve high-quality fusion of multi-source multispectral images and hyperspectral images.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] A remote sensing image fusion method based on the spatial-spectral tensor subspace representation, the method includes the following steps:

[0012] S1: Utilizing the advantages of the high-dimensional expression of tensors, the input low spatial resolution hyperspectral image LR-HSI, the input high spatial resolution multispectral image HR-MSI, and the target high spatial resolution hyperspectral image HR-HSI are respectively expressed in the form of three-dimensional tensors;

[0013] S2: Data preprocessing; spatially upsampling the LR-HSI to the same size as the target HR-HSI; subsequently, rearranging the dimensions of the target HR-HSI, HR-MSI, and the upsampled LR-HSI to obtain the LR-HSI, HR-MSI, and target HR-HSI with the spectral dimension in the second mode;

[0014] S3: Constructing spatial and spectral observation models; based on the rearranged data in S2, constructing a spectral observation model between the LR-HSI and the target HR-HSI based on the hyper-Laplacian prior and constructing a spatial observation model between the HR-MSI and the target HR-HSI;

[0015] S4: Spatial-spectral tensor subspace representation of target HR-HSI; the spatial-spectral tensor subspace representation model is used to decompose the target HR-HSI into a low-dimensional coefficient tensor, a low-dimensional spatial subspace tensor, and a low-dimensional spectral subspace tensor;

[0016] S5: Estimation of spatial and spectral subspace tensors: Under the spatial-spectral tensor subspace representation model, the tensor singular value decomposition algorithm T-SVD is used to estimate the spatial subspace tensor and spectral subspace tensor of the target HR-HSI from the dimensionality-rearranged HR-MSI and LR-HSI, respectively.

[0017] S6: Optimization and estimation of coefficient tensors. Sparse regularization constraints are implemented on the coefficient tensors. The objective function for estimating the coefficient tensors is constructed by combining the spatial and spectral observation models in S3 with the spatial-spectral tensor subspace representation model in S4. The objective function is then optimized using the proximal alternating minimization algorithm.

[0018] S7: Reconstruction of target HR-HSI; the spatial subspace tensor and spectral subspace tensor obtained in S5 and the coefficient tensor obtained in S6 are input into the spatial-spectral tensor subspace representation model in S4 to reconstruct the target HR-HSI.

[0019] Furthermore, the S2 is specifically:

[0020] Input LR-HSI Spatial upsampling to the target HR-HSI The same size, denoted as Then, the target HR-HSI HR-MSI and upsampled LR-HSI Perform dimension rearrangement to obtain the LR-HSI after dimension rearrangement HR-MSI after dimension rearrangement and HR-HSI after dimension rearrangement The spectral dimensions of LR-HSI, HR-MSI, and HR-HSI after dimension rearrangement are located in the second mode.

[0021] Furthermore, the S3 is specifically:

[0022] S3.1: Construct a spatial observation model. Assuming that HR-MSI is the result of spectral downsampling of the target HR-HSI, the spatial observation model can be expressed as:

[0023]

[0024] Among them, E (1) represents the spatial observation model, represents the spectral response function, represents the F norm;

[0025] S3.2: Construct a spectral observation model based on super-Laplacian prior; according to the maximum a posteriori probability theory, use the l-based 1 / 2 It is more reasonable to use the super-Laplacian regularization of the norm to describe the relationship between the LR-HSI after dimension rearrangement and the target HR-HSI; the spectral observation model is defined as:

[0026]

[0027] Among them, E (2) represents the spectral observation model, Represents the gradient operators along the horizontal, vertical and spectral directions respectively, α i is a parameter, ||·|| 1 / 2 Indicates l 1 / 2 norm; From formula (2), we can see that the spectral observation model based on super-Laplacian prior will no longer rely on the unknown spatial blur kernel; no scale transformation is involved in the optimization process.

[0028] Furthermore, the S4 is specifically:

[0029] The target HR-HSI after dimension rearrangement is represented by the spatial-spectral tensor subspace model. Decomposed into a low-dimensional coefficient tensor, a low-dimensional spatial subspace tensor, and a low-dimensional spectral subspace tensor:

[0030]

[0031] Among them, the symbol * represents the circular convolution operator; and Represent the low-dimensional spatial subspace tensor and the low-dimensional spectral subspace tensor respectively, and these two subspace tensors are orthogonal; Represents a low-dimensional coefficient tensor; under the spatial-spectral tensor subspace representation model, the fusion problem is converted into a spatial subspace tensor Spectral subspace tensor and the coefficient tensor Estimates.

[0032] Furthermore, the S5 is specifically:

[0033] S5.1: Spatial Subspace Tensors The spatial structure of the target HR-HSI is derived from HR-MSI. Assume that HR-MSI and target HR-HSI have the same spatial subspace, and estimate the spatial subspace tensor of the target HR-HSI from HR-MSI by T-SVD method. Use T-SVD to decompose the HR-MSI after dimension rearrangement:

[0034]

[0035] in, and denote the spatial subspace tensor and spectral subspace tensor of HR-MSI respectively, and they are orthogonal; Represents the coefficient tensor of HR-MSI; since HR-MSI and HR-HSI share the same spatial subspace, the spatial subspace tensor of the target HR-HSI It is estimated by the following formula:

[0036]

[0037] Among them, r1 represents the spatial subspace tensor The dimension is much smaller than the spatial dimension W;

[0038] S5.2: Spectral Subspace Tensor The spectral structure of the target HR-HSI is derived from the LR-HSI. Assume that the LR-HSI and the target HR-HSI have the same spectral subspace, and estimate the spectral subspace tensor of HR-HSI from the LR-HSI by T-SVD method. Use T-SVD to decompose the LR-HSI after dimension rearrangement:

[0039]

[0040] in, and They are the spatial subspace tensor and spectral subspace tensor of LR-HSI, respectively, and they are orthogonal; Represents the coefficient tensor of LR-HSI; LR-HSI and HR-HSI share the same spectral subspace, the spectral subspace tensor of the target HR-HSI It is estimated by the following formula:

[0041]

[0042] Where r2 represents the spectral subspace tensor , which is much smaller than the spectral dimension S.

[0043] Furthermore, the S6 is specifically as follows:

[0044] S6.1: Estimating the coefficient tensor The objective function is to implement sparse regularization constraints on the coefficient tensor and construct an estimated coefficient tensor by combining the spatial observation model and spectral observation model in S3 and the spatial-spectral tensor subspace representation model in S4. The objective function is:

[0045]

[0046] in, Represents the coefficient tensor The sparse constraint of α, β and λ are all positive; the spatial and spectral subspace tensors directly obtained from low-quality input images may contain noise, and the spatial subspace tensors obtained in S5 are directly converted to and the spectral subspace tensor Application to target HR-HSI reconstruction may produce errors; by Sparse regularization constraints are imposed on the target HR-HSI to explore the spatial and spectral sparsity of the target HR-HSI and reduce the impact of noise;

[0047] S6.2: Optimization of the objective function; Relaxation of the objective function (8) using the semi-quadratic splitting technique; Introduction of auxiliary variables The objective function (8) is rewritten as:

[0048]

[0049] Among them, τ represents a regularization parameter; use To express problem (9); Since problem (9) is a non-convex problem, it cannot be directly optimized and solved; For a certain variable in the fusion problem (9), fix other variables and transform the sub-problem of the variable into a convex problem, and use the relevant convex optimization method to solve it; Use the proximal alternating minimization algorithm to perform alternating optimization to solve problem (9); and The iterative update method is as follows:

[0050]

[0051] Where ρ represents the proximal parameter, k represents the number of iterations in the optimization framework, and Represents the estimated value at the kth iteration and

[0052] The beneficial effects of the present invention are:

[0053] (1) This paper establishes a spectral observation model based on a super-Laplacian prior. This model eliminates the reliance on an unknown spatial blur kernel by applying a super-Laplacian constraint on the gradient of the difference image between the input hyperspectral image and the target fusion.

[0054] (2) Since the spectral observation model based on the super-Laplacian prior does not involve scale transformation, the computational cost of the present invention is lower and the fusion efficiency is higher.

[0055] (3) The present invention establishes a spatial-spectral tensor subspace representation model. Since this model allows the spectral and spatial subspace tensors to have different subspace dimensions, it can more effectively depict the different low-rank structures of the target HR-HSI in the spatial and spectral domains.

[0056] (4) Under the spatial-spectral tensor subspace representation model, the spectral and spatial subspace tensors can be directly obtained from the input LR-HSI and HR-MSI through a simple T-SVD algorithm, which is beneficial to reduce the computational cost and improve the fusion efficiency.

[0057] (5) By imposing a sparse regularization constraint on the coefficient tensor, the present invention can further exploit the sparsity of the target HR-HSI space and spectrum and reduce the impact of noise.

[0058] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0060] Figure 1 This is a flow chart of the remote sensing image fusion method based on spatial-spectral tensor subspace representation;

[0061] Figure 2 Schematic diagram of the principle of remote sensing image fusion method based on spatial-spectral tensor subspace representation. DETAILED DESCRIPTION

[0062] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0063] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0064] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0065] like Figure 1 and Figure 2 As shown, the technical solution of the present invention includes the following steps:

[0066] S1: Taking advantage of the high-dimensional expression of tensors, the input low spatial resolution hyperspectral image (LR-HSI), input high spatial resolution multispectral image (HR-MSI) and target HR-HSI are represented as three-dimensional tensors respectively.

[0067] S2: Data preprocessing. The LR-HSI is spatially upsampled to the same size as the target HR-HSI. Subsequently, the LR-HSI, HR-MSI, and target HR-HSI are dimensionalized to obtain the LR-HSI, HR-MSI, and target HR-HSI in the second mode of spectral dimension.

[0068] S3: Construct spatial and spectral observation models. Based on the rearranged data in S2, construct a spectral observation model between the LR-HSI and the target HR-HSI based on the hyper-Laplacian prior, and construct a spatial observation model between the HR-MSI and the target HR-HSI;

[0069] S4: Spatial-spectral tensor subspace representation of the target HR-HSI. Use the spatial-spectral tensor subspace representation model to decompose the target HR-HSI into a low-dimensional coefficient tensor, a low-dimensional spatial subspace tensor, and a low-dimensional spectral subspace tensor.

[0070] S5: Estimation of spatial and spectral subspace tensors. Under the spatial-spectral tensor subspace representation model, use the tensor singular value decomposition algorithm (T-SVD) to estimate the spatial subspace tensor and spectral subspace tensor of the target HR-HSI from the HR-MSI and LR-HSI respectively.

[0071] S6: Optimization and estimation of the coefficient tensor. Implement sparse regularization constraints on the coefficient tensor, and combine the spatial observation model and spectral observation model in S3 and the spatial-spectral tensor subspace representation model in S4 to construct an objective function for estimating the coefficient tensor, and use the proximal alternating minimization algorithm for optimization and calculation.

[0072] S7: Reconstruction of the target HR-HSI. Input the spatial subspace tensor and spectral subspace tensor obtained in S5 and the coefficient tensor obtained in S6 into the spatial-spectral tensor subspace representation model in S4 to reconstruct the target HR-HSI.

[0073] The spectral observation model based on the hyper-Laplacian prior imposes hyper-Laplacian regularization (l p (0 < p < 1)) constraints on the gradient of the difference image between the target HR-HSI and the LR-HSI to describe the relationship between the input image and the target fusion image. This process has a certain statistical significance and no longer involves the spatial blur kernel, so it can effectively characterize the relationship between the input image and the target HR-HSI.

[0074] The spatial-spectral tensor subspace representation model decomposes the target HR-HSI into a low-dimensional coefficient tensor, a low-dimensional spatial subspace tensor, and a low-dimensional spectral subspace tensor. This model allows the spectral and spatial subspace tensors to have different subspace dimensions. Therefore, it can depict different low-rank structures of the target HR-HSI in the spatial domain and spectral domain, which also conforms to the fact that the low-rank characteristics of the target HR-HSI in the spatial domain and spectral domain are different. At the same time, the spatial subspace tensor and spectral subspace tensor can be obtained from the input HR-MSI and LR-HSI respectively through a simple tensor singular value decomposition algorithm, so the computational cost can be reduced.

[0075] The hyperspectral image data and multispectral image data of S1 were obtained from professional websites.

[0076] S2 is data preprocessing. First, LR-HSI Spatial upsampling to the target HR-HSI The same size, denoted as Then, LR-HSI HR-MSI and target HR-HSI Perform dimensional rearrangement to obtain LR-HSI with the spectral dimension in the second mode HR-MSI and target HR-HSI

[0077] S3 is to build a spatial and spectral observation model based on the rearranged data in S2. S3 specifically includes the following steps:

[0078] S3.1: Construct a spatial observation model. Assuming that HR-MSI is the result of spectral downsampling of the target HR-HSI, the spatial observation model can be expressed as:

[0079]

[0080] Among them, E (1) represents the spatial observation model, represents the spectral response function, represents the F-norm.

[0081] S3.2: Construct a spectral observation model based on super Laplace prior. According to the maximum a posteriori probability theory, the super Laplace prior (l 1 / 2 It is more reasonable to describe the relationship between LR-HSI and target HR-HSI using the norm. Therefore, the spectral observation model is defined as:

[0082]

[0083] Among them, E (2) represents the spectral observation model, Represents the gradient operators along the horizontal, vertical and spectral directions respectively, α i is a parameter, ||·|| 1 / 2 Indicates l 1 / 2norm. From formula (2), we can see that the spectral observation model based on the super-Laplacian prior no longer relies on the unknown spatial blur kernel. At the same time, since scale transformation is no longer involved in the optimization process, the spectral observation model based on the super-Laplacian prior will reduce computational costs and improve fusion efficiency.

[0084] S4 uses the spatial-spectral tensor subspace representation model to represent the target HR-HSI Decomposed into a low-dimensional coefficient tensor, a low-dimensional spatial subspace tensor, and a low-dimensional spectral subspace tensor:

[0085]

[0086] The symbol * represents the circular convolution operator. and They represent the low-dimensional spatial subspace tensor and the low-dimensional spectral subspace tensor respectively, and these two subspace tensors are orthogonal. Represents a low-dimensional coefficient tensor. Under the spatial-spectral tensor subspace representation model, the fusion problem can be converted into a subspace tensor ( and ) and the coefficient tensor Estimates.

[0087] There are two points to note about the subspace tensor: 1) The spatial subspace tensor and the spectral subspace tensor can be directly obtained from HR-MSI and LR-HSI, respectively, using a simple T-SVD algorithm, which means that the computational cost of solving the subspace is relatively low. 2) The dimensions of the spatial subspace tensor and the spectral subspace tensor (r1 < W and r2 < S) ​​are relatively low and can be set to different values. This allows the spatial-spectral tensor subspace representation model to effectively capture both spatial self-similarity and spectral correlation while taking into account the difference between the two features. In addition, lower dimensions mean lower computational cost.

[0088] S5 is a tensor of spatial and spectral subspaces ( and ). S5 specifically includes the following steps:

[0089] S5.1: Spatial Subspace Tensors Since the spatial structure of the target HR-HSI is derived from the HR-MSI, it can be assumed that the HR-MSI and the target HR-HSI have the same spatial subspace. Therefore, the spatial subspace tensor of the target HR-HSI can be estimated from the HR-MSI. Specifically, the HR-MSI is decomposed using T-SVD:

[0090]

[0091] in, and denote the spatial subspace tensor and spectral subspace tensor of HR-MSI respectively, and they are orthogonal. Represents the coefficient tensor of HR-MSI. Since HR-MSI and HR-HSI share the same spatial subspace, the spatial subspace tensor of HR-HSI It can be estimated by the following formula:

[0092]

[0093] Among them, r1 represents the spatial subspace tensor The dimension is much smaller than the spatial dimension W.

[0094] S5.2: Spectral Subspace Tensor Since the spectral structure of the target HR-HSI is derived from the LR-HSI, it can be assumed that the LR-HSI and the target HR-HSI have the same spectral subspace. Therefore, the spectral subspace tensor of HR-HSI can be estimated from the LR-HSI by the T-SVD method. Specifically, T-SVD is used to decompose LR-HSI:

[0095]

[0096] in, and They are the spatial subspace tensor and spectral subspace tensor of LR-HSI, respectively, and they are both orthogonal. Represents the coefficient tensor of LR-HSI. Since LR-HSI and HR-HSI share the same spectral subspace, the spectral subspace tensor of the target HR-HSI is It can be estimated by the following formula:

[0097]

[0098] Where r2 represents the spectral subspace tensor , which is much smaller than the spectral dimension S.

[0099] S6 is the optimization and estimation of the coefficient tensor.

[0100] S6 specifically includes the following steps:

[0101] S6.1: Estimating the coefficient tensor The objective function is to implement sparse regularization constraints on the coefficient tensor and construct the estimated coefficient tensor by combining the spatial observation model and spectral observation model in S3 and the spatial-spectral tensor subspace representation model in S4. The objective function is:

[0102]

[0103] in, Represents the coefficient tensor The sparse constraint of α, β and λ are all positive. The spatial and spectral subspace tensors directly obtained from low-quality input images may contain noise, so the subspace tensors obtained in S5 ( and ) may produce errors when applied to target HR-HSI reconstruction. Based on this, by Imposing a sparse regularization constraint on the target HR-HSI can explore the spatial and spectral sparsity of the target HR-HSI and reduce the impact of noise.

[0104] S6.2: Optimization of the objective function. The objective function (8) is relaxed using the semi-quadratic splitting technique. Specifically, the auxiliary variable The objective function (8) can be rewritten as:

[0105]

[0106] Among them, τ represents a regularization parameter. For convenience, we use To express problem (9). Since problem (9) is a non-convex problem, it cannot be directly optimized and solved. However, for a certain variable in the fusion problem (9), by fixing other variables, the sub-problem of the variable can be transformed into a convex problem, and the relevant convex optimization method can be used to solve it. Therefore, the proximal alternating minimization algorithm is used to perform alternating optimization to solve problem (9). Specifically, and The iterative update method is as follows:

[0107]

[0108] Where ρ represents the proximal parameter, k represents the number of iterations in the optimization framework, and Represents the estimated value at the kth iteration and

[0109] S7 is the reconstruction of the target HR-HSI. The spatial subspace tensor is obtained in S5. and the spectral subspace tensor And get the coefficient tensor in S6 Then, formula (3) is used to reconstruct the target HR-HSI after dimension rearrangement Then, through the dimension rearrangement operation, the final target HR-HSI is obtained

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A remote sensing image fusion method based on spatial-spectral tensor subspace representation, characterized by: The method comprises the following steps: S1: Taking advantage of the high-dimensional expression of tensors, the input low spatial resolution hyperspectral image LR-HSI, the input high spatial resolution multispectral image HR-MSI and the target high spatial resolution hyperspectral image HR-HSI are represented as three-dimensional tensors respectively; S2: Data preprocessing: spatially upsample the LR-HSI to the same size as the target HR-HSI. Subsequently, the target HR-HSI, HR-MSI, and upsampled LR-HSI are dimensionally rearranged to obtain the LR-HSI, HR-MSI, and target HR-HSI in the second mode of spectral dimension. S3: Construct a spatial observation model between HR-MSI and target HR-HSI and a spectral observation model between LR-HSI and target HR-HSI based on super-Laplacian prior; Based on the rearranged data in S2, construct a spectral observation model between LR-HSI and target HR-HSI based on super-Laplacian prior and a spatial observation model between HR-MSI and target HR-HSI; S4: Spatial-spectral tensor subspace representation of target HR-HSI; the spatial-spectral tensor subspace representation model is used to decompose the target HR-HSI into a low-dimensional coefficient tensor, a low-dimensional spatial subspace tensor, and a low-dimensional spectral subspace tensor; S5: Estimation of the spatial subspace tensor and spectral subspace tensor of the target HR-HSI; Under the spatial-spectral tensor subspace representation model, the tensor singular value decomposition algorithm T-SVD is used to estimate the spatial subspace tensor and spectral subspace tensor of the target HR-HSI from the dimensionality-rearranged HR-MSI and LR-HSI, respectively; S6: Optimization and estimation of coefficient tensors. Sparse regularization constraints are implemented on the coefficient tensors. The objective function for estimating the coefficient tensors is constructed by combining the spatial and spectral observation models in S3 with the spatial-spectral tensor subspace representation model in S4. The objective function is then optimized using the proximal alternating minimization algorithm. S7: Reconstruction of target HR-HSI; the spatial subspace tensor and spectral subspace tensor obtained in S5 and the coefficient tensor obtained in S6 are input into the spatial-spectral tensor subspace representation model in S4 to reconstruct the target HR-HSI.

2. The remote sensing image fusion method based on spatial-spectral tensor subspace representation according to claim 1 is characterized by: The S2 is specifically: Input LR-HSI Spatial upsampling to the target HR-HSI The same size, denoted as Then, the target HR-HSI HR-MSI and upsampled LR-HSI Perform dimension rearrangement to obtain the LR-HSI after dimension rearrangement HR-MSI after dimension rearrangement and HR-HSI after dimension rearrangement The spectral dimensions of LR-HSI, HR-MSI, and HR-HSI after dimension rearrangement are located in the second mode.

3. The remote sensing image fusion method based on spatial-spectral tensor subspace representation according to claim 2 is characterized by: The S3 is specifically: S3.1: Construct a spatial observation model. Assuming that HR-MSI is the result of spectral downsampling of the target HR-HSI, the spatial observation model can be expressed as: Among them, E (1) represents the spatial observation model, represents the spectral response function, represents the F norm; S3.2: Construct a spectral observation model based on super-Laplacian prior; according to the maximum a posteriori probability theory, use the l-based 1 / 2 It is more reasonable to use the super-Laplacian regularization of the norm to describe the relationship between the LR-HSI after dimension rearrangement and the target HR-HSI; the spectral observation model is defined as: Among them, E (2) represents the spectral observation model, i=1,2,3, respectively represent the gradient operators along the horizontal, vertical and spectral directions, α i is a parameter, ||·|| 1 / 2 Indicates l 1 / 2 norm; From formula (2), we can see that the spectral observation model based on super-Laplacian prior will no longer rely on the unknown spatial blur kernel; no scale transformation is involved in the optimization process.

4. The remote sensing image fusion method based on spatial-spectral tensor subspace representation according to claim 3 is characterized by: The S4 is specifically: HR-HSI after dimension rearrangement using the spatial-spectral tensor subspace representation model Decomposed into a low-dimensional coefficient tensor, a low-dimensional spatial subspace tensor, and a low-dimensional spectral subspace tensor: Among them, the symbol * represents the circular convolution operator; and denote the spatial subspace tensor and the spectral subspace tensor respectively, and these two subspace tensors are orthogonal; Represents the coefficient tensor; Under the spatial-spectral tensor subspace representation model, the fusion problem is converted into the spatial subspace tensor Spectral subspace tensor and the coefficient tensor Estimates.

5. The remote sensing image fusion method based on spatial-spectral tensor subspace representation according to claim 4 is characterized by: The S5 is specifically: S5.1: Spatial Subspace Tensors The spatial structure of the target HR-HSI is derived from HR-MSI. Assume that HR-MSI and target HR-HSI have the same spatial subspace, and estimate the spatial subspace tensor of the target HR-HSI from HR-MSI by T-SVD method. Use T-SVD to decompose the HR-MSI after dimension rearrangement: in, and denote the spatial subspace tensor and spectral subspace tensor of HR-MSI respectively, and they are orthogonal; Represents the coefficient tensor of HR-MSI; since HR-MSI and HR-HSI share the same spatial subspace, the spatial subspace tensor of the target HR-HSI It is estimated by the following formula: Among them, r1 represents the spatial subspace tensor The dimension is much smaller than the spatial dimension W; S5.2: Spectral Subspace Tensor The spectral structure of the target HR-HSI is derived from the LR-HSI. Assume that the LR-HSI and the target HR-HSI have the same spectral subspace, and estimate the spectral subspace tensor of the target HR-HSI from the LR-HSI by the T-SVD method. Use T-SVD to decompose the LR-HSI after dimension rearrangement: in, and They are the spatial subspace tensor and spectral subspace tensor of LR-HSI, respectively, and they are orthogonal; Represents the coefficient tensor of LR-HSI; LR-HSI and HR-HSI share the same spectral subspace, the spectral subspace tensor of the target HR-HSI It is estimated by the following formula: Where r2 represents the spectral subspace tensor , which is much smaller than the spectral dimension S.

6. The remote sensing image fusion method based on spatial-spectral tensor subspace representation according to claim 5, characterized in that: The S6 is specifically: S6.1: Estimating the coefficient tensor The objective function of Sparse regularization constraints are implemented on the coefficient tensor, and the estimated coefficient tensor is constructed by combining the spatial observation model and spectral observation model in S3 and the spatial-spectral tensor subspace representation model in S4. The objective function is: in, Represents the coefficient tensor The sparse constraint of α, β and λ are all positive; the spatial and spectral subspace tensors directly obtained from low-quality input images may contain noise, and the spatial subspace tensors obtained in S5 are directly converted to and the spectral subspace tensor Application to target HR-HSI reconstruction may produce errors; by Sparse regularization constraints are imposed on the target HR-HSI to explore the spatial and spectral sparsity of the target HR-HSI and reduce the impact of noise; S6.2: Optimization of the objective function; Relaxation of the objective function (8) using the semi-quadratic splitting technique; Introduction of auxiliary variables The objective function (8) is rewritten as: Among them, τ represents a regularization parameter; use To express problem (9); Since problem (9) is a non-convex problem, it cannot be directly optimized and solved; For a certain variable in the fusion problem (9), fix other variables and transform the sub-problem of the variable into a convex problem, and use the relevant convex optimization method to solve it; Use the proximal alternating minimization algorithm to perform alternating optimization to solve problem (9); and The iterative update method is as follows: Where ρ represents the proximal parameter, k represents the number of iterations in the optimization framework, and Represents the estimated value at the kth iteration and

Images