A remote sensing image pan-sharpening method and system based on kernel guidance

Abstract

The present invention proposes a kernel-guided panchromatic sharpening method and system for remote sensing images to solve the generalization problem that a single deep learning panchromatic sharpening model trained with a single fixed blur kernel may suffer from distortion when encountering mismatched degradation. This method designs a fusion sub-network with strong interpretability to inject the details of the panchromatic image into the multispectral image through a learnable injection matrix. The prior degradation information generated by the kernel estimation sub-network can represent the blur degree of the multispectral image and is used to update the learnable injection matrix to control the injection of details and adapt to different types of degradation. In addition, the channel transformer is used as a residual connection in the fusion sub-network, which helps to retain important features in the deep network. Through the combination of kernel estimation and fusion sub-networks, the network's adaptability to various unknown degradations is enhanced, so the proposed method has great advantages in complex degradation situations.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image fusion, and relates to a remote sensing image full-color sharpening method and system based on kernel guidance, which is suitable for solving application scenarios of complex degraded scenes in the real world. Background Art

[0002] Multispectral image pan-sharpening aims to fuse a low-spatial-resolution multispectral image with a high-spatial-resolution panchromatic image of the same scene. This creates a high-quality multispectral image that is spatially consistent with the panchromatic image and spectrally consistent with the multispectral image, yielding a single image containing more information. High-quality pan-sharpened images facilitate the understanding and interpretation of remote sensing scenes, significantly improving recognition accuracy in applications such as change detection, object recognition, and agricultural analysis.

[0003] Currently, there are a variety of multispectral image pan-sharpening techniques, including component substitution, multiresolution analysis, variational optimization-based methods, and deep learning-based methods. Component substitution and multiresolution analysis are used to inject details from panchromatic images into multispectral images. However, due to the incomplete separation of spectral and spatial components, spatial or spectral distortions often result, potentially causing spatial or spectral distortion. In contrast, variational-based methods perform better in balancing spatial and spectral information. However, the complex a priori design of these methods and the time-consuming optimization process limit their practical application.

[0004] With the continuous launch of remote sensing satellites and the increasing amount of remote sensing data, deep learning-based methods have become popular. Some researchers have begun using large amounts of data to drive model training, and then using the trained models to obtain pan-sharpened images. Deep learning methods can map the complex nonlinear relationships between the fused image and the reference image, which can achieve good pan-sharpening results. However, the inconsistency between the training domain and the test domain often leads to model generalization issues.

[0005] In recent years, researchers have focused their attention on deep learning-based multispectral image pan-sharpening methods due to the advantages of convolutional neural networks in learning image features from large-scale data. Existing deep learning-based multispectral image pan-sharpening methods typically rely on the Wald protocol to create simulated datasets for training. This protocol specifies a specific degradation process that utilizes only a single fixed isotropic Gaussian blur kernel. Based on this fixed blur level, spatial details from the panchromatic image are then injected into the low-resolution multispectral image to obtain a high-resolution multispectral image. However, the real world can exhibit a variety of degradation types, resulting in inappropriate injection of details when encountering complex, unknown blur kernels, resulting in distortion or artifacts. Consequently, the performance of existing deep learning-based methods can significantly degrade when applied to data with mismatched degradation processes. Therefore, designing a multispectral image pan-sharpening architecture that can effectively fuse multispectral and panchromatic images in unknown degraded environments is a key challenge in remote sensing image fusion. Summary of the Invention

[0006] In order to solve the above problems in the prior art, the present invention provides a remote sensing image full color sharpening method based on kernel guidance.

[0007] The present invention provides a remote sensing image pan-sharpening method based on kernel guidance, comprising the following steps:

[0008] Step 1: Based on the blur degree of the blur kernel, a fusion sub-network is constructed to fuse the multispectral and panchromatic images;

[0009] The specific processing process of the fusion sub-network is as follows:

[0010] Step 1.1, upsample the multispectral image to match the size of the panchromatic image;

[0011] Step 1.2, convert the upsampled multispectral image into the feature domain through the convolution layer;

[0012] In step 1.3, N1 detail injection Transformer modules are used to inject the detail features of the panchromatic image into the multispectral image.

[0013] In step 1.4, the output of the detail injection transformer module is restored from the feature domain to the image domain through a convolutional layer;

[0014] Step 2: Based on the relationship between the fused image of the fusion subnetwork and the input low-resolution multispectral image, a kernel estimation subnetwork is constructed to estimate the blur kernel;

[0015] The specific processing process of the kernel estimation subnetwork is as follows:

[0016] In step 2.1, the fused image is first downsampled to a feature image of the same size as the low-resolution multispectral image, denoted as f1;

[0017] Step 2.2, the low-resolution multispectral image passes through a convolutional layer to obtain a feature map f2;

[0018] Step 2.3, f1 and f2 are extracted through N3 dual-path units, namely DP units, to obtain the fuzzy feature f 1_out and f 2_out , and serves as the input of the next DP unit;

[0019] In step 2.4, the final fuzzy features are converted into fuzzy kernels through several network layers.

[0020] Step 3: construct an image fusion model, which includes a fusion subnetwork and a kernel estimation subnetwork, and uses a target loss function that combines kernel loss and reconstruction loss to drive the training of the image fusion model;

[0021] In step 4, the image fusion model is trained using simulation data, and the trained model is tested on the test set. The resulting fused image achieves full color sharpening of the multispectral image.

[0022] Furthermore, the specific processing process of the fusion sub-network also includes:

[0023] In step 1.5, residual connections are added between the input and output of the fusion sub-network to connect the upsampled multispectral image to the output to further preserve the spectral information of the multispectral image.

[0024] Furthermore, in step 1.3, each detail injection Transformer module contains N2 detail injection units, namely DI units, which are used to inject appropriate details into the multispectral image. Residual connections are introduced at the input and output ends of the detail injection Transformer module, and a Transformer block with an embedded channel, namely a CTB module, is added to the residual connection branch.

[0025] Furthermore, the processing of the DI unit is as follows;

[0026] The panchromatic image and the randomly initialized blur kernel are used as the injection matrix, and the details of the panchromatic image are injected into the multispectral image using a learnable injection matrix. Since different blur levels will affect the amount of details injected into the multispectral image, the estimated blur kernel is subsequently used to update the blur kernel parameters in the learnable injection matrix. Specifically, the blur kernel learns an injection gain map through an injection gain learning module. The injection gain map represents the injection gain at each position in the multispectral image. The injection gain learning module contains four layers. Assuming that the blur kernel input to the injection gain learning module is k and the output result is a k-map, the injection gain learning module is expressed as:

[0027] k-map=Sigmod(Conv(Relu(Conv(k))))

[0028] Where Conv() represents 3×3 convolution, Sigmod() represents Sigmod activation function, and Relu() represents Relu activation function;

[0029] The full-color image is converted to the feature domain through a convolutional layer, and then the full-color image transferred to the feature domain is multiplied by the injection gain map to adjust the details; in addition, a residual connection is used between the original full-color image and the adjusted full-color image to finally obtain the details to be injected, and then all the details are injected into the multispectral image by summing the details to be injected with the output features of each layer in the detail-injected Transformer module.

[0030] Furthermore, the processing of the CTB module is as follows;

[0031] Assume the input is First, transform the input dimension to get Where H, W, and C represent the height, width, and number of channels of the input features respectively; then, the three learnable matrices W Q 、W K 、W V Multiply with the input X to get the query vector Q, key vector K, and value vector V,

[0032] And divided into multiple attention heads:

[0033] Q=[Q1,Q2,…,Q h ]

[0034] k=[k1,k2,…,K h ]

[0035] V=[V1,V2,…,V h ]

[0036] in, C = h × d. h is the number of attention heads, and d is the number of feature channels in each attention head.

[0037] Then, on each attention head, calculate Q i and K i The spatial similarity between d features at each attention head:

[0038]

[0039] in, S i represents the self-similarity between the d features in the i-th attention head, σ i represents the learnable parameters used to adjust the results;

[0040] Then, S i Normalized by the Softmax function and compared with V i Multiply and get the i-th attention head The output of each attention head is then concatenated and collapsed into a feature map through a linear transformation. Finally, CTB uses convolutional layers to transform the input features X in Processing is performed to construct position encoding to mark the relative position information between feature channels; the output of channel attention Combined with position encoding, the final output of channel multi-head self-attention C-MSA is obtained, and residual links are introduced at the input and output of C-MSA; then, the final output of CTB is obtained through a feed-forward layer consisting of layer normalization and convolutional layers, and residual links are added at the input and output of the feed-forward layer.

[0041] Furthermore, in step 2.3, the specific implementation of the DP unit is as follows;

[0042] In the DP unit, f1 and f2 are first connected and an injection gain G is calculated; the injection gain is multiplied by f1 and f2 to focus on the important features of f1 and f2 and compress unimportant features, and then added to the original f2 to form a new feature map. This process is expressed as:

[0043] G = Sigmoid(func(Cat(f1,f2)))

[0044] f 1_out =func(f1)+f1

[0045] f 2_out =G·func(f1)+G·func(f2)+f2

[0046] Among them, f 1_out and f2_out The features f1 and f2 representing the input are processed by the DP unit, func(·) is the Conv-ReLu-Conv operation, and G is the injection gain.

[0047] Furthermore, in step 2.4, the network layer is implemented through convolution-pooling-convolution-Softmax.

[0048] Furthermore, the objective loss function constructed in step 3 is as follows;

[0049] L=L img +λL ker

[0050] Among them, λ is a parameter used to balance the importance of these two loss terms; L img is the reconstruction loss between the fused image and the corresponding reference image; L ker is the loss between the estimated kernel and the reference kernel to help the network estimate the correct kernel; the expressions of these two losses are as follows:

[0051] L img =‖FR‖1

[0052] L ker =‖kk r ‖1

[0053] Among them, F and R are the fused image and the corresponding reference image, k and k r are the estimated blur kernel and reference kernel respectively.

[0054] Furthermore, step 4 also includes comparing the test results with the existing algorithm through objective evaluation indicators, and the objective evaluation indicators include peak signal-to-noise ratio and spectral angle mapping.

[0055] The present invention also provides a remote sensing image full-color sharpening system based on kernel guidance, comprising a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a remote sensing image full-color sharpening method based on kernel guidance as described in the above technical solution.

[0056] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0057] The present invention embeds kernel estimation into a fusion network to utilize prior information about the kernel. The network consists of two subnetworks: a kernel estimation subnetwork and a fusion subnetwork. The fusion subnetwork restores high-resolution multispectral images based on the blur kernel estimated by the kernel estimation subnetwork. Specifically, a channel transformer is used in the fusion subnetwork to focus on important features. The kernel estimation subnetwork is implemented via a dual-path network. The kernel estimation subnetwork uses the fused image obtained by the fusion subnetwork and the low-resolution multispectral image to estimate the blur kernel. The estimated blur kernel is used to update the learnable matrix, thereby motivating the fusion subnetwork to achieve good fusion results. Therefore, the proposed method has strong adaptability to complex degraded scenes, and the fused image is clearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is an overall framework diagram of a kernel-guided remote sensing image full-color sharpening method according to an embodiment.

[0059] Figure 2 This is a network architecture diagram of the channel Transformer in the embodiment.

[0060] Figure 3 : It is a test result diagram of the simulation data of the embodiment, where (a) is the result of FusionNet, (b) is the result of GPPNN, (c) is the result of ADKNet, (d) is the result of the method proposed in the present invention, (e) is the reference image, (f) is the estimated kernel, and (g) is the reference kernel. DETAILED DESCRIPTION

[0061] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0062] This invention primarily addresses the complexities of real-world degradation scenarios, which can lead to inappropriately injected detail when encountering complex, unknown blur kernels. We utilize a fusion subnetwork to adjust the details injected from the panchromatic image based on the estimated blur kernel from the kernel estimation subnetwork, yielding a fused image. Specifically, a channel transformer is employed within the fusion subnetwork to focus on important features. The kernel estimation subnetwork learns the blur relationship between the fused image obtained by the fusion subnetwork and a low-resolution multispectral image, thereby estimating the blur kernel. This results in a multispectral fused image with rich spatial detail and minimal spectral distortion in the presence of complex degradation.

[0063] Attachment Figure 1 This is the overall framework diagram of the kernel-guided remote sensing image pan-sharpening method of the embodiment, Figure 2This is a network architecture diagram of the channel transformer of the embodiment. This embodiment provides a kernel-guided remote sensing image full-color sharpening method to solve the degradation mismatch problem caused by the complexity of real-world degradation scenes. Specifically, it includes the following steps:

[0064] Step 1: Based on the blur degree of the blur kernel, the multispectral and panchromatic images are fused through the fusion sub-network. The specific implementation includes the following sub-steps:

[0065] Step 1.1, upsample the multispectral image to match the size of the panchromatic image;

[0066] Step 1.2, convert the upsampled multispectral image into the feature domain through the convolution kernel;

[0067] In step 1.3, N1 detail injection transformer modules (DITB modules) are used to inject the detail features of the panchromatic image into the multispectral image. Each DITB contains N2 detail injection (DI) units, which are used to inject appropriate details into the multispectral image. In order to improve the ability of the deep network to retain important information, residual connections are introduced at the input and output of the DITB, and a channel transformer block (CTB) with embedding is added to the residual connection branch.

[0068] In the DI unit, the full-color image and the randomly initialized blur kernel are first used as the injection matrix, and the learnable injection matrix is ​​used to inject the details of the full-color image into the multispectral image. And because different blur levels will affect the amount of details injected into the multispectral image, the blur kernel estimated in step 2 is further used to update the blur kernel parameters in the learnable injection matrix. Specifically, the blur kernel learns the injection gain map through the injection gain learning module, which represents the injection gain at each position in the multispectral image. Specifically, the above-mentioned injection gain learning module contains four layers. Assuming that the blur kernel input to the module is k and the output result is k-map, the injection gain learning module can be expressed as:

[0069] k-map=Sigmod(Conv(Relu(Conv(k))))

[0070] Among them, Conv() represents 3×3 convolution, Sigmod() represents Sigmod activation function, and its expression is as follows:

[0071]

[0072] Relu() represents the Relu activation function, and its mathematical expression is as follows:

[0073] Relu(x)=max(0,x)

[0074] The panchromatic image is converted to the feature domain through a Conv layer. The converted panchromatic image is then multiplied by the gain map to adjust the details. Furthermore, a residual connection is used between the original and adjusted panchromatic images to obtain the details to be injected. All details are then injected into the multispectral image by summing the details to be injected with the output features of each layer in the DITB module.

[0075] As attached Figure 2 As shown, in the CTB module, assuming the input is Where H, W, and C represent the height, width, and number of channels of the input features respectively. CTB first transforms the input dimension to obtain Then, three matrices W that can be learned iteratively through the network Q 、W K 、W V Multiply with the input X to get the query vector Q, key vector K, and value vector V, And divided into multiple attention heads:

[0076] Q=[Q1,Q2,…,Q h ]

[0077] k=[k1,k2,…,K h ]

[0078] V=[V1,V2,…,V h ]

[0079] in, C = h × d. h is the number of attention heads, and d is the number of feature channels in each attention head.

[0080] Then, on each attention head, calculate Q i and K i The spatial similarity between d features at each attention head:

[0081]

[0082] in, S i represents the self-similarity between the d features in the i-th attention head, σ i represents a learnable parameter used to adjust the result.

[0083] Then, S i Normalized by the Softmax function and compared with V i Multiply and get the i-th attention head Then, the output vectors of each attention head are concatenated and folded into a feature map through linear transformation Finally, CTB uses convolutional layers to transform the input features X in Processing is performed to construct position encoding to mark the relative position information between feature channels. Output of channel attention Combined with positional encoding, the final output of C-MSA is obtained, and residual links are introduced at the input and output of C-MSA. Then, the final output of CTB is obtained through a feed-forward layer consisting of layer normalization and convolutional layers. Residual links are added at the input and output of the feed-forward layer.

[0084] In step 1.4, the output of DITB passes through a Conv layer with 4 channels to restore it from the feature domain to the image domain;

[0085] In step 1.5, residual connections are added between the input and output of the fusion sub-network to connect the upsampled multispectral image to the output to further preserve the spectral information of the multispectral image.

[0086] Step 2: Based on the relationship between the fused image of the fusion subnetwork and the input low-resolution image, the kernel estimation subnetwork estimates the blur kernel. The specific implementation includes the following substeps:

[0087] Step 2.1, in the kernel estimation subnetwork, the fused image is first downsampled to a feature image of the same size as the low-resolution multispectral image, denoted as f1;

[0088] Step 2.2, the low-resolution multispectral image passes through a convolutional layer to obtain a feature map f2;

[0089] Step 2.3, f1 and f2 are then passed through N3 dual-path units (DP) to extract the fuzzy feature f 1_out and f 2_out , and serves as the input of the next DP layer;

[0090] In DP, f1 and f2 are first concatenated and an injection gain is calculated. The injection gain is multiplied by f1 and f2 to focus on the important features of f1 and f2 and compress unimportant features. Then, they are added to the original f2 to form a new feature map. By integrating their information, the kernel estimation subnetwork can learn the relationship between them and effectively capture the fuzzy features. This process can be expressed as:

[0091] G = Sigmoid(func(Cat(f1,f2)))

[0092] f 1_out =func(f1)+f1

[0093] f 2_out =G·func(f1)+G·func(f2)+f2

[0094] Among them, f 1_out and f 2_out represents the output of input features f1 and f2 after DP processing, func(·) is the Conv-ReLu-Conv operation, and G is the injection gain.

[0095] In step 2.4, the blurred features are converted into blur kernels through four network layers, which are implemented by convolution-pooling-convolution-Softmax.

[0096] Step 3: Construct an objective function including kernel loss and reconstruction loss to drive the training of the image fusion model, which includes a fusion subnetwork and a kernel estimation subnetwork. The specific implementation includes the following substeps:

[0097] Step 3.1: Construct a loss function. Construct a loss function based on image reconstruction and kernel reconstruction:

[0098] L=L img +λL ker

[0099] Among them, λ is a parameter used to balance the importance of these two loss terms. img A reconstruction loss is constructed between the fused image and the corresponding reference image to ensure that the fusion result is close to the reference image; L ker Is the loss between the estimated kernel and the reference kernel to help BPNet estimate the correct kernel; they both use L1 loss to calculate the error. These two losses can be described as:

[0100] L img =‖FR‖1

[0101] L ker =‖kk r ‖1

[0102] Where F and R are the fused image and the corresponding reference image respectively. k and k r They are the blur kernel and reference kernel estimated in step 2. img and L ker The combination of can drive the model to solve image fusion and kernel estimation problems at the same time.

[0103] Step 3.2: Randomly select training data from the training set and input it into the network, complete one iteration, and adjust the network parameters.

[0104] In step 4, the image fusion model is trained using simulation data and tested on the test set using the trained model. The specific implementation includes the following sub-steps:

[0105] Step 4.1: The kernel-guided model described above was trained using image data from the Gaofen-2 satellite. In simulation experiments, the original multispectral image was blurred using a random kernel and then downsampled to generate a simulated image, while the original multispectral image was used as the reference image. Similarly, the panchromatic image was downsampled by a factor r to generate a simulated panchromatic image. Simulated data generation was performed using an isotropic Gaussian blur kernel of size 21×21, with kernel width uniformly sampled from [0.2, 4]. In each dataset, 95% of the data was randomly selected as training data, and the remaining data was used as validation data. Data augmentation was employed to increase data diversity through rotation and flipping. The size of the multispectral images in both training and validation data was 64×64×4, and the size of the panchromatic images was 256×256×1. All images were normalized to the pixel range [0, 1]. The network's initial learning rate was set to 5e-4 and decayed by 0.5 every 100 epochs. Furthermore, the Adam optimizer was used to update the network parameters. The batch size is set to 4 for all datasets and the networks are trained for approximately 800 epochs.

[0106] Step 4.2: Load the fusion model trained on the Gaofen-2 satellite and test it on the test set. Compare the test results with the comparison methods in terms of visual and objective evaluation indicators. The test image size is 256×256. To verify the effectiveness of the proposed method, we compare it with deep learning-based methods. Deep learning methods include FusionNet, GPPNN, and ADKNet. The visual comparison results are shown in the attached figure. Figure 3 The objective evaluation indicators are peak signal-to-noise ratio (PSNR) and spectral angle matching (SAM). PSNR measures the noise of the fused image, and SAM measures the spectral preservation of the fused image. The average results on the test set are shown in Table 1.

[0107] Table 1 Comparison of indicators of different methods for simulation data (ideal value: PSNR: +∞, SAM: 0)

[0108]

[0109]

[0110] It can be seen that the kernel estimation method proposed by us can promote the fusion result through the kernel prior, and can obtain a multispectral fusion image with clear details and good spectral preservation; and the correct kernel is estimated.

[0111] On the other hand, an embodiment of the present invention also provides a remote sensing image full-color sharpening system based on kernel guidance, including a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a remote sensing image full-color sharpening method based on kernel guidance as described in the above technical solution.

[0112] It should be understood that parts not elaborated in detail in this specification belong to the prior art.

[0113] It should be understood that the above description of the embodiments is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A remote sensing image pan-sharpening method based on kernel guidance, characterized in that: The steps include: Step 1: Based on the blur degree of the blur kernel, a fusion sub-network is constructed to fuse the multispectral and panchromatic images; The specific processing process of the fusion sub-network is as follows: Step 1.1, upsample the multispectral image to match the size of the panchromatic image; Step 1.2, convert the upsampled multispectral image into the feature domain through the convolution layer; Step 1.3, use A detail injection Transformer module injects detail features of the panchromatic image into the multispectral image; In step 1.4, the output of the detail injection transformer module is restored from the feature domain to the image domain through a convolutional layer; Step 2: Based on the relationship between the fused image of the fusion subnetwork and the input low-resolution multispectral image, a kernel estimation subnetwork is constructed to estimate the blur kernel; The specific processing process of the kernel estimation subnetwork is as follows: In step 2.1, the fused image is first downsampled to a feature image of the same size as the low-resolution multispectral image, represented as ; Step 2.2: The low-resolution multispectral image passes through a convolutional layer to obtain a feature map ; Step 2.3, and pass A dual-path unit, DP unit, extracts fuzzy features and , and serves as the input of the next DP unit; In step 2.4, the final fuzzy features are converted into fuzzy kernels through several network layers; Step 3: construct an image fusion model, which includes a fusion subnetwork and a kernel estimation subnetwork, and uses a target loss function that combines kernel loss and reconstruction loss to drive the training of the image fusion model; The objective loss function constructed in step 3 is as follows; in, is a parameter used to balance the importance of these two loss terms; is the reconstruction loss between the fused image and the corresponding reference image; is the loss between the estimated kernel and the reference kernel to help the network estimate the correct kernel; the expressions of these two losses are as follows: in, and are the fused image and the corresponding reference image, and are the estimated blur kernel and reference kernel respectively; In step 4, the image fusion model is trained using simulation data, and the trained model is tested on the test set. The resulting fused image achieves full color sharpening of the multispectral image.

2. The kernel-guided remote sensing image pan-sharpening method according to claim 1, wherein: The specific processing process of the fusion sub-network also includes: In step 1.5, residual connections are added between the input and output of the fusion sub-network to connect the upsampled multispectral image to the output to further preserve the spectral information of the multispectral image.

3. The kernel-guided remote sensing image pan-sharpening method according to claim 2, wherein: In step 1.3, each detail is injected into the Transformer module containing A detail injection unit, namely the DI unit, is used to inject appropriate details into the multispectral image. A residual connection is introduced between the input and output of the detail injection Transformer module, and a Transformer block with an embedded channel, namely the CTB module, is added to the residual connection branch.

4. The kernel-guided remote sensing image pan-sharpening method according to claim 3, wherein: The processing of the DI unit is as follows; The full-color image and the randomly initialized blur kernel are used as the injection matrix, and the details of the full-color image are injected into the multispectral image using a learnable injection matrix. Since different blur levels will affect the amount of details injected into the multispectral image, the estimated blur kernel is subsequently used to update the blur kernel parameters in the learnable injection matrix. Specifically, the blur kernel learns the injection gain map through the injection gain learning module, which represents the injection gain at each position of the multispectral image. The injection gain learning module contains four layers. Assume that the blur kernel input to the injection gain learning module is , the output result is , then the injection gain learning module is expressed as: in express convolution, express activation function, express Activation function; The full-color image is converted to the feature domain through a convolutional layer, and then the full-color image transferred to the feature domain is multiplied by the injection gain map to adjust the details; in addition, a residual connection is used between the original full-color image and the adjusted full-color image to finally obtain the details to be injected, and then all the details are injected into the multispectral image by summing the details to be injected with the output features of each layer in the detail-injected Transformer module.

5. The kernel-guided remote sensing image pan-sharpening method according to claim 3, wherein: The processing of the CTB module is as follows; Assume the input is , first transform the input dimension to get ,in Represent the height, width and number of channels of the input features respectively; then, three learnable matrices 、 、 With input Multiply to get the query vector , key vector , value vector , , and divided into multiple attention heads: in, , ; is the number of attention heads, is the number of feature channels in each attention head; Then, on each attention head, calculate and On each attention head Spatial similarity between features: in, , Indicates the Attention head The self-similarity between features, represents the learnable parameters used to adjust the results; Then, Normalized by the Softmax function and Multiply them together to get Attention head The output of each attention head is then concatenated and collapsed into a feature map through a linear transformation. ; Finally, CTB uses convolutional layers to transform input features Processing is performed to construct position encoding to mark the relative position information between feature channels; the output of channel attention Combined with position encoding, the final output of channel multi-head self-attention C-MSA is obtained, and residual links are introduced at the input and output of C-MSA; then, the final output of CTB is obtained through a feed-forward layer consisting of layer normalization and convolutional layers, and residual links are added at the input and output of the feed-forward layer.

6. The kernel-guided remote sensing image pan-sharpening method according to claim 1, wherein: In step 2.3, the specific implementation of the DP unit is as follows; In the DP unit, first and Connect them and calculate an injection gain ; Injection gain and and Multiply to focus on and The important features of Add together to form a new feature map. This process is expressed as: in, and Features representing the input and The output obtained after processing by the DP unit is is the Conv-ReLu-Conv operation, is the injection gain.

7. The kernel-guided remote sensing image pan-sharpening method according to claim 1, wherein: In step 2.4, the network layer is implemented by convolution-pooling-convolution-Softmax.

8. The kernel-guided remote sensing image pan-sharpening method according to claim 1, wherein: Step 4 also includes comparing the test results with existing algorithms through objective evaluation indicators, such as peak signal-to-noise ratio and spectral angle mapping.

9. A remote sensing image pan-sharpening system based on kernel guidance, characterized by: The invention comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a remote sensing image full-color sharpening method based on kernel guidance as claimed in any one of claims 1 to 8.

Images