Universal image fusion method and system based on low-rank and sparse prior

Abstract

The present invention relates to the field of image fusion technology, and provides a general image fusion method and system based on low-rank and sparse priors. The method comprises extracting common features of two source images through a common feature encoder based on low-rank decomposition; extracting sparse unique features of the two source images respectively through a unique feature encoder based on sparse representation; fusing the sparse unique features of the two source images through a serial adaptive fusion module, and fusing them with the common features of the two source images to obtain fused features; and inputting the fused features into a feature reconstruction module to obtain a fused image. The present invention solves the image fusion problem by exploring the intrinsic relationship between the features of the source images, dividing the image features into common features and sparse unique features, making the information flow within the network clearer, greatly increasing the interpretability of the network, and not only achieving effective cross-modal interaction, but also fully exploring the unique features of the modalities.

Description

Technical Field

[0001] The present invention relates to the field of image fusion technology, and in particular to a universal image fusion method and system based on low rank and sparse prior. Background Art

[0002] Due to the differences in the imaging mechanisms of different sensors, images captured from the same scene often contain complementary information. For example, infrared sensors capture thermal radiation to highlight salient objects, while visible light sensors use reflected light to produce images with rich textural detail. Furthermore, positron emission tomography (PET) can reflect the metabolic activity of tissues, while magnetic resonance imaging (MRI) provides high-resolution anatomical information. A single sensor can only capture a portion of the information within its dynamic range at different exposure settings. Furthermore, cameras with different focal lengths can clearly present objects within their depth of field. The goal of image fusion is to integrate the complementary information from multiple source images to produce a fused image containing rich information. Current image fusion tasks primarily include infrared-visible image fusion (IVF), medical image fusion (MIF), multi-exposure image fusion (MEF), and multi-focus image fusion (MFF).

[0003] Despite differences in specific tasks, the overall goal of image fusion remains the same: to produce an information-rich fused image. Consequently, much research has focused on general methods for unifying the multi-source image fusion task. PMGI (Fast Unified Image Fusion Network Based on Gradient and Intensity Ratio Preservation) employs a dual-branch feature extraction network to capture modality-specific information and models multi-source image fusion as a task that simultaneously preserves both pixel intensity and gradient details. This method achieves fusion by artificially setting the weight ratios between different tasks. TUFusion (Transformer Network) fuses Transformer and CNN architectures to construct a hybrid encoder, balancing the modeling of global representations with the preservation of local details, further enhancing the quality of the fused image. VDMUFusion (Universal Image Fusion Network Based on Diffusion Model) models image fusion as a pixel-by-pixel weighted averaging process and introduces a multi-task learning framework to replace the traditional noise prediction network, thereby extending the diffusion model into a general image fusion method.

[0004] Existing fusion methods employ a single encoder architecture, which can extract common features across modalities but may overlook unique features. Dual encoder designs emphasize modality uniqueness but overlook shared representations. From a network perspective, existing methods typically stack Transformer and convolutional neural network modules, resulting in high computational complexity and low fusion efficiency. This lack of in-depth exploration of commonalities across different fusion tasks results in models that exhibit "black box" characteristics and lack interpretability. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a general image fusion method and system based on low-rank and sparse priors. A common feature encoder (CFE) based on low-rank decomposition extracts common features, while a unique feature encoder (UFE) based on sparse representation extracts sparse unique features of the two source images. The sparse unique features of the two source images are fused through an entropy-based feature fusion module and then fused with the common features of the two source images to obtain fused features. The fused features are then input into a feature reconstruction module to obtain a fused image. This method not only achieves effective cross-modal interaction but also fully exploits the unique features of the modalities, achieving feature decoupling and increasing the interpretability of the network.

[0006] The present invention provides a general image fusion method based on low rank and sparse prior, comprising:

[0007] S1: Extract the common features of the two source images through a common feature encoder based on low-rank decomposition;

[0008] S11: Extracting shallow joint features of the two source images through the joint feature extraction module;

[0009] S12: Use non-negative matrix decomposition to perform low-rank approximation on the shallow joint features to obtain the basis matrix and coefficient matrix;

[0010] S13: performing matrix multiplication on the basis matrix and the coefficient matrix to obtain the common features of the two source images;

[0011] S2: extracting sparse unique features of the two source images respectively through a unique feature encoder based on sparse representation;

[0012] S21: obtaining the rough unique features of the two source images by subtracting the common features of the two source images from the source images;

[0013] S22: Construct a sparse decomposition model based on the rough unique features of the two source images;

[0014] S23: Solve the sparse decomposition model through the kernel transposed convolution block to obtain the sparse unique features of the two source images;

[0015] S3: The sparse unique features of the two source images are fused through the serial adaptive fusion module, and then fused with the common features of the two source images to obtain the fused features;

[0016] S4: Input the fused features into the feature reconstruction module to obtain a fused image.

[0017] Furthermore, the joint feature extraction module includes Convolution and depth-wise separable convolution, the depth-wise separable convolution includes depth-wise convolution and point-wise convolution.

[0018] Furthermore, in step S12, the non-negative matrix decomposition approximates the non-negative characteristic matrix by minimizing the reconstruction error, and the calculation expression is:

[0019]

[0020] in, is the basis matrix, is the coefficient matrix, is a shallow joint feature, is the Frobenius norm of the matrix, To find the minimum function.

[0021] Furthermore, in step S22, the calculation expression of the sparse decomposition model is:

[0022]

[0023]

[0024] in, For the source image The rough unique characteristics of for The sparse coefficient of for Dictionary, is the Manhattan norm, is the first learnable parameter, For the source image The rough unique characteristics of for The sparse coefficient of for Dictionary, is the second learnable parameter, To find the minimum function, is the Frobenius norm of the matrix.

[0025] Furthermore, in step S23,

[0026] Using deep expansion technology, the matrix multiplication operation of the sparse decomposition model is replaced by convolution operation;

[0027] Expand the iterative solution process of the sparse decomposition model into a neural network consisting of K stages, replacing the dictionary with a convolutional layer and replacing the transposed convolutional layer with the transposed dictionary;

[0028] The solution process of each stage is expressed as a kernel transposed convolution block, and the calculation expression is:

[0029]

[0030]

[0031] in, For the Iterations The sparse coefficient of is the first soft threshold operation, For the Iterations The sparse coefficient of is the convolutional layer, is the transposed convolution layer, For the Iterations The sparse coefficient of For the Iterations The sparse coefficient of is the second soft threshold operation, is the convolution operation;

[0032] Input the sparse coefficients of the Kth stage obtained by the kernel transposed convolution block Convolution,obtains sparse unique features of the two source images.

[0033] Furthermore, in step S3, the serial adaptive fusion module includes two entropy-based feature fusion modules, which sequentially realize the fusion of the sparse unique features of the two source images, and further fuse the fused sparse unique features with the common features, including:

[0034] S31: Extract the details and brightness of the sparse unique features of the two source images through global average pooling and maximum pooling, and then Convolution obtains sparse unique fusion features of the two source images;

[0035] S32: Calculating the fusion weights of the two source images by calculating the information entropy of the sparse unique fusion features of the two source images;

[0036] S33: fusing the sparse unique fusion features of the two source images according to the fusion weights of the two source images to obtain a fusion unique feature;

[0037] S34: Fusing the unique features with the common features of the two source images to obtain fused features.

[0038] Furthermore, the feature reconstruction module includes a plurality of reconstruction convolution layers, each of which includes Convolution and ReLU activation functions.

[0039] The present invention also provides a universal image fusion system based on low rank and sparse prior, which is used to execute the above-mentioned universal image fusion method based on low rank and sparse prior, comprising:

[0040] A low-rank decomposition unit, wherein the low-rank decomposition unit extracts common features of the two source images through a common feature encoder based on low-rank decomposition;

[0041] A sparse representation unit, wherein the sparse representation unit extracts sparse unique features of the two source images respectively through a unique feature encoder based on sparse representation;

[0042] A fusion unit, wherein the fusion unit fuses the sparse unique features of the two source images through a serial adaptive fusion module, and fuses them with the common features of the two source images to obtain a fusion feature;

[0043] The reconstruction unit inputs the fused features into a feature reconstruction module to obtain a fused image.

[0044] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0045] The present invention extracts common features using a common feature encoder based on low-rank decomposition, and extracts sparse unique features from the two source images using a unique feature encoder based on sparse representation. The sparse unique features of the two source images are fused using an entropy-based feature fusion module, and then fused with the common features of the two source images to obtain fused features. A fused image is then obtained using a feature reconstruction module. The present invention divides image features into common features and sparse unique features, making the information flow within the network clearer and greatly increasing the interpretability of the network. This not only enables effective cross-modal interaction, but also fully exploits the unique features of the modalities.

[0046] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 It is a flowchart of a general image fusion method based on low rank and sparse prior provided by the present invention.

[0049] Figure 2This is a schematic diagram of a network framework of a universal image fusion method based on low rank and sparse priors provided by the present invention.

[0050] Figure 3 It is a schematic diagram of the structure of the entropy-based feature fusion module provided by the present invention.

[0051] Figure 4 This is a structural diagram of a universal image fusion system based on low rank and sparse priors provided by the present invention.

[0052] Figure 5 Schematic diagram of a subjective comparison experiment of the fusion results of infrared and visible light images of the M3FD dataset according to an embodiment of the present invention.

[0053] Reference numerals:

[0054] 101. Low-rank decomposition unit; 102. Sparse representation unit; 103. Fusion unit; 104. Reconstruction unit. DETAILED DESCRIPTION

[0055] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0056] In the description of the embodiments of the present invention, it should be noted that the terms "first", "second" and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. The description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in an appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0057] The following combination Figures 1 to 5 The present invention describes a general image fusion method and system based on low-rank and sparse priors.

[0058] In the image restoration task, the image can be decomposed into a low-rank matrix representing the background and a sparse matrix representing the abnormal area. In the image fusion task, the source images come from the same scene and therefore have similar scene information, but have different modal features. Therefore, the present invention regards the common features of the source images as the same low-rank matrix, and regards the modal difference features as the sparse matrix of the abnormal area. The present invention constructs a general image fusion network based on low-rank and sparse priors, named (A General Image Fusion Network Based on Low-Rank and Sparse Priors, abbreviated as LRSP-Fusion), which solves the image fusion problem by mining the intrinsic relationship between the source image features. The network framework is as follows Figure 2 shown.

[0059] like Figure 1 As shown in FIG, a general image fusion method based on low-rank and sparse priors includes:

[0060] S1: Extract the common features of the two source images through a common feature encoder based on low-rank decomposition;

[0061] Get any two source images that need to be fused. The source images can be infrared-visible light, medical images, multi-exposure images, and multi-focus images. The source images all come from the same scene and have the same basic structure and information. These can be regarded as common features of the source images and have the same low-rank representation. Their image matrices usually have low-rank or near-low-rank properties. The low-rank matrix represents the main structure and information of the image. The common features of the two source images are extracted through CFE. The calculation expression is:

[0062]

[0063] in, is the common feature of the two source images, For CFE operation, For splicing operation, is the first source image, is the second source image.

[0064] S11: Extract shallow joint features of the two source images through the joint feature extraction module (JFEB);

[0065] JFEB includes Convolution and depth-wise separable convolution, depth-wise separable convolution includes depth-wise convolution (DWConv) and point-wise convolution (PWConv); this process is expressed as:

[0066]

[0067] in, is a shallow joint feature, for convolution, for The depth of the convolution, for Point-wise convolution.

[0068] S12: Use non-negative matrix factorization (NMF) to perform low-rank approximation on the shallow joint features to obtain the basis matrix and coefficient matrix;

[0069] Non-negative matrix factorization (NMF) is a low-rank approximation model that can extract the underlying features of data. The image matrix itself is also a non-negative matrix. Therefore, this paper uses NMF technology to perform low-rank decomposition on the image matrix and reconstructs the common features using the basis matrix and coefficient matrix obtained from the decomposition.

[0070] Non-negative matrix factorization approximates the non-negative feature matrix by minimizing the reconstruction error. The calculation expression is:

[0071]

[0072] in, is the basis matrix, , The dimension is The matrix set of for The column dimension of is the height, is the width, is the coefficient matrix, , The dimension is The matrix set of is the number of channels, The column dimensions and The rows of , is a shallow joint feature, is the Frobenius norm of the matrix, To find the minimum function, the constraints are and All elements in are greater than or equal to 0, and are updated iteratively and To minimize the loss function, the update formula is as follows:

[0073]

[0074]

[0075] in, For the The basis matrix of the iteration, For the The basis matrix of the iteration, is the Hadamard product, is the transpose of the matrix, For the The coefficient matrix of the iteration, For the The coefficient matrix of the iteration,

[0076] S13: Perform matrix multiplication on the basis matrix and the coefficient matrix to obtain common features of the two source images.

[0077] S2: extracting sparse unique features of the two source images respectively through a unique feature encoder based on sparse representation;

[0078] Common features through a Convolution is performed to compress and obtain a common feature map, which is then subtracted element-by-element from the input source image. The calculation expression is:

[0079]

[0080]

[0081] in, for The sparse unique features of For UFE operation, for Convolution operation, is the common feature of the two source images, for sparse unique features.

[0082] S21: The coarse unique features of the two source images are obtained by subtracting the common features of the two source images from the source images. The calculation expression is:

[0083]

[0084]

[0085] in, for The rough unique characteristics of for The rough unique characteristics of is the common feature of the two source images, for The convolution is performed using Convolution compresses the channel dimension of common features to achieve the integration of common features.

[0086] S22: Construct a sparse decomposition model based on the rough unique features of the two source images;

[0087] Common features are the main structure and information of the image, while unique features represent the modal information of the source image and are sparse. UFE is constructed based on the sparse representation of unique features and is used to solve the following sparse decomposition model:

[0088]

[0089]

[0090] in, For the source image The rough unique characteristics of for The sparse coefficient of for Dictionary, is the Manhattan norm, is the first learnable parameter, For the source image The rough unique characteristics of for The sparse coefficient of for Dictionary, is the second learnable parameter, To find the minimum function, is the Frobenius norm of the matrix.

[0091] The traditional method can be solved by the proximal gradient descent method, and its iterative solution formula is as follows:

[0092]

[0093]

[0094] in, For the Iteration The sparse coefficient of is the first soft threshold operation, For the Iteration The sparse coefficient of For the Iteration The sparse coefficient of For the Iteration The sparse coefficient of is the second soft threshold operation, , For about The symbolic function of To find the maximum function, To find the absolute value function, is a learnable parameter, Desirable or .

[0095] S23: Solve the sparse decomposition model through the kernel transposed convolution block to obtain the sparse unique features of the two source images.

[0096] The present invention adopts deep expansion technology to replace the matrix multiplication operation of the sparse decomposition model with a convolution operation; the iterative solution process of the sparse decomposition model is expanded into a neural network containing K stages, the dictionary is replaced by a convolution layer, and the transposition of the dictionary is replaced by a transposed convolution layer;

[0097] like Figure 2 As shown, each iteration ( ) corresponds to a step in the solution process, and the solution process of each stage is represented as a kernel transposed convolution block (KTCB), and the calculation expression is:

[0098]

[0099]

[0100] in, For the Iterations The sparse coefficient of is the first soft threshold operation, For the Iterations The sparse coefficient of for Convolutional layers, for Transposed convolutional layer, For the Iterations The sparse coefficient of For the Iterations The sparse coefficient of is the second soft threshold operation, is the convolution operation;

[0101] Phase 0 is used for initialization The initial sparse coefficient and The initial sparse coefficient , the calculation expression is:

[0102]

[0103]

[0104] Among them, the initial sparse coefficient include as well as , the sparse coefficients of the first stage include The sparse coefficients of the first stage and The sparse coefficients of the first stage .

[0105] Input the sparse coefficients of the Kth stage obtained by the kernel transposed convolution block Convolution, obtains the sparse unique features of the two source images, and the calculation expression is:

[0106]

[0107]

[0108] in, For the Iterations The sparse coefficient of for The sparse unique features of For the Iterations The sparse coefficient of for sparse unique features.

[0109] S3: The sparse unique features of the two source images are fused through the sequential adaptive fusion module (SAFM), and then fused with the common features of the two source images to obtain the fused features;

[0110] like Figure 3 As shown in Figure 1, SAFM includes two entropy-based feature fusion blocks (EFFBs), which successively realize the fusion of the sparse unique features of the two source images and the further fusion of the fused sparse unique features with the common features. The calculation expression is:

[0111]

[0112] in, To fusion features, For EFFB operation;

[0113] EFFB structure is as follows Figure 3 shown.

[0114] S31: Extract the details and brightness of the sparse unique features of the two source images through global average pooling and maximum pooling, and then Convolution obtains the sparse unique fusion features of the two source images, and the calculation expression is:

[0115]

[0116]

[0117] in, for The sparse unique fusion features of is the global average pooling, is the maximum pooling operation, for The sparse unique fusion features.

[0118] S32: Calculating the fusion weights of the two source images by calculating the information entropy of the sparse unique fusion features of the two source images;

[0119] The entropy of an image reflects the average amount of information in the image. The higher the entropy value of a feature map, the richer the features it contains. Therefore, the present invention uses entropy calculation to evaluate the amount of information in the feature map, thereby obtaining the adaptive weights of the two sets of features. The calculation expression is:

[0120]

[0121] in, for The weight of for The weight of is the information entropy calculation function.

[0122] S33: fusing the sparse unique fusion features of the two source images according to the fusion weights of the two source images to obtain a fusion unique feature;

[0123]

[0124] in, To integrate unique features.

[0125] S34: Fusing the unique features with the common features of the two source images to obtain fused features.

[0126] S4: Input the fused features into the feature reconstruction module (FRM) to obtain a fused image.

[0127] The feature reconstruction module includes multiple reconstruction convolution layers, each of which includes a Convolution and a ReLU activation function, the calculation expression is:

[0128]

[0129] in, To fuse the images, For FRM operation, To reconstruct the convolutional layer.

[0130] This paper deeply explores the common problems that are prevalent in multi-source image fusion, namely low-rank background information and sparse unique information. On this basis, it conducts unified modeling of the multi-source image fusion task from a mathematical perspective and designs an efficient and lightweight fusion network, which not only realizes effective cross-modal interaction, but also fully explores the unique features of the modalities; realizes feature decoupling and increases the interpretability of the network.

[0131] like Figure 4 As shown, a universal image fusion system based on low rank and sparse prior is used to perform a universal image fusion method based on low rank and sparse prior, including:

[0132] The low-rank decomposition unit 101 extracts common features of the two source images through a common feature encoder based on low-rank decomposition;

[0133] The sparse representation unit 102 extracts sparse unique features of the two source images respectively through a unique feature encoder based on sparse representation;

[0134] The fusion unit 103 fuses the sparse unique features of the two source images through a serial adaptive fusion module, and fuses them with the common features of the two source images to obtain a fused feature;

[0135] The reconstruction unit 104 inputs the fused features into the feature reconstruction module to obtain a fused image.

[0136] Through the collaborative work of the above units, the common feature encoder based on low-rank decomposition extracts common features, while the unique feature encoder based on sparse representation extracts sparse unique features of the two source images. The sparse unique features of the two source images are fused through the entropy-based feature fusion module, and then fused with the common features of the two source images to obtain fused features. The fused image is obtained through the feature reconstruction module. The present invention divides image features into common features and sparse unique features, making the information flow within the network clearer and greatly increasing the interpretability of the network. It not only achieves effective cross-modal interaction, but also fully explores the unique features of the modalities.

[0137] This paper conducted quantitative comparative experiments and qualitative experimental analysis on the public dataset M3FD with several state-of-the-art algorithms. These include MMDRFuse, TUFusion, VDMUFusion, and GIFNet. MMDRFuse is the 2024 version of the Dynamic Refresh Distillation Micro Model, TUFusion is the 2024 version of the Transformer Network, VDMUFusion is the 2024 version of the Diffusion Model-Based Universal Image Fusion Network, and GIFNet is the 2025 version of the Three-Branch Universal Image Fusion Network.

[0138] The comparative experiment results of the fusion quantitative indicators of the M3FD dataset are shown in Table 1.

[0139] Table 1 Comparative experiment results of fusion quantitative indicators of M3FD dataset

[0140]

[0141] The higher the indexes in Table 1, the better the fusion effect. As shown in Table 1, the present invention performs well in all evaluation indicators. Figure 5 As shown, the present invention can effectively preserve the texture details of visible light images and extract salient targets in infrared images.

[0142] To further validate the effectiveness of this method in practical applications, we applied the fused infrared-visible image to object detection. Specifically, we used the fused image as input to an object detector, and employed YOLOv5 as the detection model to evaluate its effectiveness in improving detection accuracy. Table 2 shows the comparative results of quantitative metrics for pedestrian and vehicle detection on the M3FD dataset.

[0143] Table 2 Comparative experiment results of quantitative indicators of pedestrian and vehicle detection on the M3FD dataset

[0144]

[0145] Using the mean average precision calculated with an IoU threshold of 0.5 as the evaluation metric, as shown in Table 2, the proposed method achieved higher detection accuracy and better target recognition in both pedestrian and vehicle detection tasks. Quantitative metrics, as shown in Table 2, comprehensively outperformed the comparison methods. These experimental results demonstrate that the proposed method not only improves image fusion quality but also effectively enhances the performance of downstream tasks, demonstrating its strong practical value.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A general image fusion method based on low-rank and sparse priors, characterized by: include: S1: Extract the common features of the two source images through a common feature encoder based on low-rank decomposition; S11: Extracting shallow joint features of the two source images through the joint feature extraction module; S12: Use non-negative matrix decomposition to perform low-rank approximation on the shallow joint features to obtain the basis matrix and coefficient matrix; S13: performing matrix multiplication on the basis matrix and the coefficient matrix to obtain the common features of the two source images; S2: extracting sparse unique features of the two source images respectively through a unique feature encoder based on sparse representation; S21: obtaining the rough unique features of the two source images by subtracting the common features of the two source images from the source images; S22: Construct a sparse decomposition model based on the rough unique features of the two source images; S23: Solve the sparse decomposition model through the kernel transposed convolution block to obtain the sparse unique features of the two source images; S3: The sparse unique features of the two source images are fused through the serial adaptive fusion module, and then fused with the common features of the two source images to obtain the fused features; The serial adaptive fusion module includes two entropy-based feature fusion modules, which sequentially realize the fusion of the sparse unique features of the two source images and the further fusion of the fused sparse unique features with the common features, including: S31: Extract the details and brightness of the sparse unique features of the two source images through global average pooling and maximum pooling, and then Convolution obtains the sparse unique fusion features of the two source images, and the calculation expression is: in, for The sparse unique fusion features of is the global average pooling, is the maximum pooling operation, for The sparse unique features of for The sparse unique fusion features of for The sparse unique features of for Convolution operation; S32: Calculate the fusion weights of the two source images by calculating the information entropy of the sparse unique fusion features of the two source images; use the entropy calculation to evaluate the amount of information in the feature map, thereby obtaining the adaptive weights of the two sets of features. The calculation expression is: in, for The weight of for The weight of is the information entropy calculation function; S33: fusing the sparse unique fusion features of the two source images according to the fusion weights of the two source images to obtain a fusion unique feature; in, To integrate unique features; S34: Fusing the unique fusion feature with the common features of the two source images to obtain a fusion feature; S4: Input the fused features into the feature reconstruction module to obtain a fused image.

2. A general image fusion method based on low rank and sparse prior according to claim 1, characterized in that: The joint feature extraction module includes Convolution and depth-wise separable convolution, the depth-wise separable convolution includes depth-wise convolution and point-wise convolution.

3. The general image fusion method based on low rank and sparse prior according to claim 1, characterized in that: In step S12, the non-negative matrix decomposition approximates the non-negative characteristic matrix by minimizing the reconstruction error. The calculation expression is: in, is the basis matrix, is the coefficient matrix, is a shallow joint feature, is the Frobenius norm of the matrix, To find the minimum function.

4. The general image fusion method based on low rank and sparse prior according to claim 1, characterized in that: In step S22, the calculation expression of the sparse decomposition model is: in, For the source image The rough unique characteristics of for The sparse coefficient of for Dictionary, is the Manhattan norm, is the first learnable parameter, For the source image The rough unique characteristics of for The sparse coefficient of for Dictionary, is the second learnable parameter, To find the minimum function, is the Frobenius norm of the matrix.

5. The general image fusion method based on low rank and sparse prior according to claim 4, characterized in that: In step S23, Using deep expansion technology, the matrix multiplication operation of the sparse decomposition model is replaced by convolution operation; Expand the iterative solution process of the sparse decomposition model into a neural network consisting of K stages, replacing the dictionary with a convolutional layer and replacing the transposed convolutional layer with the transposed dictionary; The solution process of each stage is expressed as a kernel transposed convolution block, and the calculation expression is: in, For the Iterations The sparse coefficient of is the first soft threshold operation, For the Iterations The sparse coefficient of is the convolutional layer, is the transposed convolution layer, For the Iterations The sparse coefficient of For the Iterations The sparse coefficient of is the second soft threshold operation, is the convolution operation; Input the sparse coefficients of the Kth stage obtained by the kernel transposed convolution block Convolution,obtains sparse unique features of the two source images.

6. The general image fusion method based on low rank and sparse prior according to claim 1, characterized in that: The feature reconstruction module includes multiple reconstruction convolution layers, each of which includes Convolution and ReLU activation functions.

7. A general image fusion system based on low-rank and sparse priors, characterized by: Used to perform a universal image fusion method based on low rank and sparse prior as claimed in any one of claims 1 to 6, comprising: A low-rank decomposition unit, wherein the low-rank decomposition unit extracts common features of the two source images through a common feature encoder based on low-rank decomposition; A sparse representation unit, wherein the sparse representation unit extracts sparse unique features of the two source images respectively through a unique feature encoder based on sparse representation; A fusion unit, wherein the fusion unit fuses the sparse unique features of the two source images through a serial adaptive fusion module, and fuses them with the common features of the two source images to obtain a fusion feature; The reconstruction unit inputs the fused features into a feature reconstruction module to obtain a fused image.

Images