A multi-modal correction panchromatic sharpening system and method based on a task allocation method

Abstract

The application relates to the technical field of image processing, and particularly discloses a multi-modal correction panchromatic sharpening system and method based on a task allocation method, which comprises three encoder-decoder branches, which are used for respectively calculating a spatial detail image, a supplementary detail image and a correction detail image according to a low-resolution multispectral image and a panchromatic image; and a feature fusion module, which is used for fusing the spatial detail image, the supplementary detail image, the correction detail image and the low-resolution multispectral image to obtain a high-resolution multispectral image. The application decomposes a fusion target into a plurality of subtasks with clear semantics through a task allocation mechanism, and introduces a modal correction attention module to realize physical consistent cross-modal feature alignment, so that the spatial resolution is improved while the original spectral characteristics are maximally reserved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a multimodal correction panchromatic sharpening system and method based on a task allocation method. Background Technology

[0002] Multispectral remote sensing images are widely used in urban planning, environmental monitoring, precision agriculture, and other fields due to their excellent spectral resolution. However, limited by the design constraints of satellite sensors, the spatial resolution of multispectral images is usually low, making it difficult to meet the needs of refined ground feature identification. Therefore, panchromatic sharpening technology, which fuses high spatial resolution panchromatic images with low spatial resolution multispectral images to generate high spatial resolution multispectral images, has become a key step in remote sensing image processing.

[0003] Current mainstream panchromatic sharpening methods mainly include traditional computational models (such as GSA, GLP-REG, LRTCF) and deep learning models (such as DRPNN, MSDCNN, TFNet, etc.). Traditional methods are usually based on detail injection or component replacement strategies, rely on linear assumptions, and are difficult to adapt to complex terrain scenes. While existing deep learning methods can extract nonlinear features, they still have shortcomings in modeling the inherent differences between PAN (panchromatic image) and MS (multispectral image) modes. They often introduce spectral distortion while improving spatial details, or cause reconstruction bias by ignoring negative information in the residuals.

[0004] Despite some progress in full-color sharpening, existing technologies still have significant shortcomings in the following areas, limiting the quality and practicality of fused images:

[0005] 1. Insufficient modeling of modal differences: PAN images contain only intensity information, while MS images contain multi-channel spectral information, and there are fundamental differences between the two in terms of imaging mechanism and information dimension. Most existing methods directly stitch or add the two types of features without considering their physical inconsistencies, resulting in spectral distortion or color shift during the fusion process.

[0006] 2. Incomplete Residual Detail Reconstruction: Traditional detail injection methods typically employ non-negative activation functions such as ReLU, forcibly restricting the residuals to positive values, ignoring the negative components that may exist in the actual residuals (such as regions of decreased spectral reflectance), resulting in information loss. Although existing deep learning models can learn residuals end-to-end, a single network struggles to simultaneously optimize both spatial enhancement and spectral fidelity.

[0007] 3. Coarse feature fusion strategy: Most deep learning methods use simple concatenation or convolution to fuse features from different levels, lacking the ability to distinguish and model different semantic details (such as edges, textures, and spectral consistency), resulting in blurred spatial details or halo artifacts.

[0008] 4. Lack of physical imaging prior guidance: Most existing models are purely data-driven and do not incorporate physical models of remote sensing imaging systems (such as modulation transfer function MTF), which makes the feature alignment process lack physical rationality and affects the cross-sensor generalization ability. Summary of the Invention

[0009] This invention aims to address the problems of insufficient modal difference modeling, incomplete residual reconstruction, and coarse fusion strategies in existing panchromatic sharpening methods. To this end, this invention provides a multimodal correction panchromatic sharpening system and method based on a task allocation approach. The system decomposes the fusion objective into multiple semantically defined sub-tasks through a task allocation mechanism and introduces a modal correction attention module to achieve physically consistent cross-modal feature alignment, thereby improving spatial resolution while preserving the original spectral characteristics to the greatest extent possible.

[0010] This invention provides a multimodal correction panchromatic sharpening system based on a task allocation method, and the technical solution adopted is as follows: including:

[0011] The encoder-decoder has three branches: spatial detail branch, supplementary detail branch, and corrective detail branch.

[0012] The encoder-decoder branch includes a dual-stream coding layer that simultaneously encodes the panchromatic image and the low-resolution multispectral image. It also utilizes a modality correction attention module to correct features extracted from the panchromatic image and align them with the desired features from the high-resolution multispectral image.

[0013] The spatial detail branch, supplementary detail branch, and corrected detail branch are used to calculate the spatial detail image, supplementary detail image, and corrected detail image based on the low-resolution multispectral image and panchromatic image, respectively.

[0014] The feature fusion module is used to fuse spatial detail images, supplementary detail images, corrected detail images, and low-resolution multispectral images to obtain high-resolution multispectral images.

[0015] Furthermore, the spatial detail branch, supplementary detail branch, and corrected detail branch adopt the same model structure. During training, they are supervised by spatial information of the residual difference between high-resolution multispectral images and low-resolution multispectral images, residual supplementary information of spatial information, and residual difference between low-resolution multispectral images and high-resolution multispectral images, respectively.

[0016] Furthermore, the calculation process of the modality correction attention module includes:

[0017] The feature difference operation is performed on the panchromatic image features and multispectral image features obtained after encoding to obtain an attention map of cross-modal feature differences;

[0018] The modality-specific difference features are obtained by performing element-wise multiplication between the panchromatic image features and the attention map;

[0019] Subtracting modality-specific difference features from panchromatic image features yields features that are more closely aligned with multispectral modes.

[0020] Furthermore, the formula for calculating the attention map is:

[0021]

[0022] in, For attention maps, Features of a panchromatic image For multispectral image features, The modulation transfer function acts on To reduce its spatial resolution, and Reach the same level.

[0023] Furthermore, in the encoder, the encoding process includes feature extraction and downsampling;

[0024] The panchromatic image is processed by the decomposition module and 3×3 convolution, then input into the feature extraction module for feature extraction, and then downsampled.

[0025] The low-resolution multispectral image is pre-upsampled, then input into the feature extraction module for feature extraction, and then downsampled.

[0026] Furthermore, the feature extraction module includes a 5×5 convolutional layer, a multilayer perceptron, and layer normalization, and embeds residual connections.

[0027] Furthermore, in the encoder, feature extraction and downsampling are performed multiple times on the low-resolution multispectral image and the panchromatic image, respectively.

[0028] Furthermore, the decoder includes alternating feature extraction modules and upsampling layers.

[0029] Furthermore, the total loss function The calculation formula is:

[0030]

[0031] in, For the global loss function, For the spatial detail branch, To supplement the loss function of the detailed branches, To correct the loss function for the detail branches.

[0032] This invention also provides a multimodal correction panchromatic sharpening method based on a task allocation method, the technical solution of which includes the following steps:

[0033] S1: Acquire panchromatic and low-resolution multispectral images;

[0034] S2: Input the panchromatic image and the low-resolution multispectral image into the spatial detail branch, the supplementary detail branch, and the corrected detail branch respectively, and calculate the spatial detail image, the supplementary detail image, and the corrected detail image;

[0035] S3: Input the spatial detail image, supplementary detail image, and corrected detail image into the feature fusion module, and fuse them with the low-resolution multispectral image to obtain a high-resolution multispectral image.

[0036] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0037] 1. This invention effectively avoids information loss through a task decomposition mechanism. By explicitly modeling positive and negative residuals and further decomposing positive residuals into spatial and supplementary details, this invention can focus on learning different types of features, overcoming the spectral distortion problem caused by ignoring negative values ​​or mixed details in traditional methods.

[0038] 2. This invention designs a modality correction attention module to achieve physically consistent modality alignment. Furthermore, a modulation transfer function (MTF) is introduced into the modality correction attention module to simulate the optical response of a satellite imaging system, ensuring that PAN and MS features are compared at the same spatial scale. The generated attention map accurately reflects the true modal differences, significantly improving spectral fidelity.

[0039] 3. This invention employs multi-branch supervision to improve training stability. Each subtask is supervised by a corresponding detail image, resulting in a clearer gradient path and alleviating the gradient vanishing problem in end-to-end training. It is particularly suitable for learning from sparse detail images.

[0040] 4. The fusion results of this invention are more applicable to downstream tasks. Experiments show that the HRMS images generated by this invention perform better in downstream tasks such as classification, verifying that it effectively preserves discriminative spectral information while maintaining spatial details.

[0041] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 This is a system architecture diagram provided by the present invention.

[0044] Figure 2 This is a flowchart of the method provided by the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but should not be used to limit the scope of this invention.

[0046] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0047] The following is combined with Figure 1 and Figure 2 The present invention will be further described in detail below, including a multimodal correction panchromatic sharpening system and method based on a task allocation method:

[0048] Task Decomposition and Supervised Image Generation: First, the concept of task decomposition originates from the traditional detail injection method. High-resolution multispectral images can be obtained by injecting residual details from low-resolution multispectral images. Specifically, the more accurate the estimation of residual details between high-resolution and low-resolution multispectral images, the better the restoration effect of the high-resolution image. Typically, this residual information can be expressed through a variant of the residual formula, as follows:

[0049]

[0050] in, The residuals between high-resolution multispectral images (HRMS) and low-resolution multispectral images (LRMS) are... For high-resolution multispectral images, It is a low-resolution multispectral image.

[0051] However, the residuals between LRMS and HRMS images contain many negative values. In deep learning, activation functions such as ReLU and PReLU are commonly used to accelerate network training and prevent gradient explosion, but the LeakyReLU activation function may suppress negative values, leading to the loss of some residual details. To preserve these details as much as possible, this embodiment calculates both the positive and negative parts of the residuals. The new residual calculation formula can be expressed as follows:

[0052]

[0053]

[0054] in, For pixels of high-resolution multispectral images , For pixels of low-resolution multispectral images , For high-resolution multispectral images and low-resolution multispectral images at the pixel level The residual difference at a given point is only positive. Conversely, For low-resolution multispectral images and high-resolution multispectral images, at the pixel level The residual difference at that point is also limited to positive values.

[0055] therefore, It can be represented as minus The results are expressed as follows:

[0056]

[0057] in, The residual difference between high-resolution and low-resolution multispectral images. This represents the residual difference between the low-resolution and high-resolution multispectral images. (After decomposition) Ensure and Both matrices contain positive elements, thus mitigating the loss due to negative values.

[0058] Based on the above decomposition method, the residual information is further decomposed from a modal perspective. The residual information simultaneously contains spatial and spectral information provided by panchromatic and multispectral images, which can be used to supervise network training. However, since the residual contains multimodal information, directly fusing different modal features through a single network may lead to distortion. To address this issue, the modal information in the residual is separated, aiming to minimize cross-modal information within each separated part, thereby reducing the learning difficulty of the corresponding separation task.

[0059] Assume there are two main modes of residual information: spatial information from the panchromatic image and spectral information from the multispectral image. Theoretically, completely separating spectral and spatial information from the residuals is challenging. Therefore, this embodiment aims to maximize the extraction of spatial information from the residuals by decomposing the residuals into spatial and complementary residual components and minimizing the cross-modal information between the two parts. This goal is achieved by employing multi-scale analysis and gradient pyramid sharing methods to extract spatial details from the residuals. The specific formula is as follows:

[0060]

[0061] in, From Spatial information extracted from it, The gradient operator is used to extract spatial information. Let be the spatial Gaussian blur kernel for the i-th layer, used to calculate the statistical sum of spatial information across different scales. Therefore, residual supplementary information... It can be calculated using the following formula:

[0062] .

[0063] Using the above decomposition method It can be decomposed into four different modal images. Furthermore, the original... Linear calculations can be performed using the following inverse decomposition equations:

[0064] .

[0065] Using a task allocation decomposition formula, this embodiment decomposes the HRMS learning task into multiple modalities, allowing each branch to focus primarily on a single task, thereby reducing interference from cross-modal learning. Finally, the modal images are synthesized using inverse decomposition equations to obtain a fused image.

[0066] Based on the above task decomposition of residuals from high- and low-resolution multispectral images, in this embodiment, as follows: Figure 1 As shown, a multimodal correction pancolor sharpening system based on a task allocation method is provided, including: three encoder-decoder branches and a feature fusion module.

[0067] Three encoder-decoder branches are used to calculate spatial detail images, supplementary detail images, and corrected detail images based on the low-resolution multispectral image and the panchromatic image, respectively. In the feature fusion module, the three detail images are fused with the low-resolution multispectral image to obtain a high-resolution multispectral image. In this embodiment, the low-resolution multispectral image has been pre-upsampled.

[0068] Construction of a multi-branch encoder-decoder: Given the three decomposition terms involving spatial details, supplementary details, and correction details (corresponding to respectively) , and This embodiment proposes a general encoder-decoder framework to learn each subtask separately. The framework architecture includes a two-stream coding layer that can simultaneously encode panchromatic and low-resolution multispectral images.

[0069] like Figure 1 As shown, this embodiment includes three encoder-decoder branches, named Spatial Details Branch (Net-1), Supplementary Details Branch (Net-2), and Corrected Details Branch (Net-3), respectively. The structures of Net-2 and Net-3 are the same as those of Net-1.

[0070] The following uses Net-1 as an example to describe the structure of the encoder-decoder branch and its working process.

[0071] In the encoder structure, Net-1 integrates a decomposition module (TAB). This module first separates texture features from the panchromatic image, and then fuses them with the panchromatic image to generate latent modal features. The specific calculation process is shown in the formula:

[0072] .

[0073] Net-2 and Net-3 generate different latent modal features through different decomposition modules.

[0074] In the encoder, the encoding process includes feature extraction and downsampling. Net-1 uses a Feature Extraction Module (FEB) to extract features from both the low-resolution multispectral image and the panchromatic image. In this embodiment, the low-resolution multispectral image input to the FEB is pre-upsampled. The panchromatic image input to the FEB is first processed by a decomposition module and a 3×3 convolution. The FEB uses a large convolutional kernel with a 5×5 convolutional layer to process features, thereby expanding the receptive field of feature extraction. However, this operation may lead to loss of feature map granularity. Therefore, the FEB introduces a Multilayer Perceptron (MLP) and layer normalization techniques to improve feature granularity. To reduce the risk of gradient vanishing, a residual connection structure is also embedded in the FEB.

[0075] Subsequently, the feature map output by FEB is downsampled to obtain multi-scale features, which are then re-inputted into FEB for secondary extraction. Through three iterative downsampling processes, a more accurate multi-scale feature set is finally obtained.

[0076] During the downsampling process, a unique Modality Correction Attention Module (MCA) is integrated simultaneously to correct features acquired from the panchromatic image and align them with the desired features in the high-resolution multispectral image. MCA generates an attention map by calculating the modal differences between features in the low-resolution multispectral image and the panchromatic image, dynamically correcting the features of the panchromatic image to adapt to the multispectral modalities.

[0077] Significant modal differences exist between high-resolution multispectral images (HRMS) and panchromatic images (PAN). Furthermore, due to the use of a two-stream coding scheme, spatial details extracted from panchromatic images may lack necessary spectral information, making it difficult to provide the required features for HRMS image reconstruction. To address this issue, this embodiment proposes an attention-based MCA module. This module performs feature difference calculations between panchromatic image features obtained after downsampling spatial resolution and multispectral image features to obtain an attention map of cross-modal feature differences. The formula is as follows:

[0078]

[0079] in, For attention maps, Features of a panchromatic image For multispectral image features, The modulation transfer function acts on To reduce its spatial resolution, Spatial resolution and Reach the same level.

[0080] Subsequently, element-wise multiplication is performed between the panchromatic image features and the attention map to extract modality-specific difference features. These modality-specific difference features are then subtracted from the panchromatic image features to obtain features more closely aligned with the multispectral modalities. The formula is as follows:

[0081]

[0082] in, This is for element-wise multiplication.

[0083] Will and After stitching, it serves as the panchromatic image feature input for the next stage of feature extraction.

[0084] In the decoder structure, the same Feature Extraction Module (FEB) is used for feature extraction. The extracted features are then concatenated with the original-size encoding layer features, and deconvolution is applied to upsample the output features of the decoding layer, ultimately generating the output images for each branch. Corresponding to the encoding stage, in this embodiment, the decoder performs three upsampling operations; that is, the decoder includes four alternately arranged feature extraction modules and three upsampling layers. The Net-1 decoder ultimately outputs a spatial detail image.

[0085] Similarly, the low-resolution multispectral image and the panchromatic image are input into the supplementary detail branch to calculate the supplementary detail image. The low-resolution multispectral image and the panchromatic image are input into the correction detail branch to calculate the correction detail image.

[0086] The feature fusion module is used to fuse the spatial detail image, supplementary detail image, and corrected detail image with the low-resolution multispectral image to obtain a high-resolution multispectral image. That is, the low-resolution multispectral image is added to the spatial detail image and the supplementary detail image, and the corrected detail image is subtracted to obtain the high-resolution multispectral image.

[0087] Loss Function: Each branch of this system (Net-1, Net-2, and Net-3) is associated with a specific subtask and supervised by the corresponding detail images obtained through task assignment. The supplementary detail branch and the corrected detail branch adopt the same model structure and are supervised by the spatial information of the residual difference between high-resolution multispectral images and low-resolution multispectral images, the residual supplementation information of the spatial information, and the residual difference between low-resolution multispectral images and high-resolution multispectral images, respectively.

[0088] Specifically, Spatial Details Branch (SPAB) uses Supervision is performed. The loss function for the spatial detail branch. It can be defined as:

[0089]

[0090] in, It is an L1 norm. This is a branch for spatial details.

[0091] Similarly, supplemental detail branches (SUB) are used. To monitor and correct detail branches (CDB), use... Perform supervision. Add details to the loss function for each branch. and the loss function for correcting detail branches They can be defined as follows:

[0092]

[0093]

[0094] in, To supplement the detailed branches, To correct the detailed branches.

[0095] Subsequently, according to the original The solution formula linearly combines the outputs of each branch to generate the final fusion result, which can be supervised using high-resolution multispectral images.

[0096]

[0097] in, For the final fusion result, For the output of the spatial detail branch, To supplement the output of the detailed branches, To correct the output of the detailed branch.

[0098] It can be achieved through the global loss function To conduct supervision, the formula is:

[0099] .

[0100] In summary, the total loss function It can be defined as:

[0101] .

[0102] This embodiment performs an average quantitative evaluation of our system (MMCP) and existing state-of-the-art models on the Pléiades and WorldView-3 datasets. Existing state-of-the-art models include GSA, GLP-Reg, DRPNN, MSDCNN, PSDCRN, TFNet, FusionNet, TDNet, ALWFN, and LRTCF. Evaluation metrics include: PSNR (Peak Signal-to-Noise Ratio), UIQI (Universal Image Quality Index), SAM (Spectral Angle Mapper), ERGAS (Relative Dimensionless Global Synthesis Error), and QNR (QR-Free Quality Index). (Second-order local general image quality index) (Spectral distortion) (Spatial distortion). The average quantitative evaluation results on the Pléiades dataset are shown in Table 1, and the average quantitative evaluation results on the WorldView-3 dataset are shown in Table 2. This system significantly outperforms existing state-of-the-art models.

[0103] Table 1

[0104]

[0105] Table 2

[0106]

[0107] This embodiment also provides a multimodal correction pancolor sharpening method based on a task allocation method, implemented based on the aforementioned multimodal correction pancolor sharpening system based on a task allocation method, such as... Figure 2 As shown, the technical solution adopted is as follows: including the following steps:

[0108] S1: Acquire panchromatic and low-resolution multispectral images;

[0109] S2: Input the panchromatic image and the low-resolution multispectral image into the spatial detail branch, the supplementary detail branch, and the corrected detail branch respectively, and calculate the spatial detail image, the supplementary detail image, and the corrected detail image;

[0110] S3: Input the spatial detail image, supplementary detail image, and corrected detail image into the feature fusion module, and fuse them with the low-resolution multispectral image to obtain a high-resolution multispectral image.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal correction panchromatic sharpening system based on a task allocation method, characterized in that, include: The encoder-decoder has three branches: spatial detail branch, supplementary detail branch, and corrective detail branch. The encoder-decoder branch includes a dual-stream coding layer that simultaneously encodes the panchromatic image and the low-resolution multispectral image. It also utilizes a modality correction attention module to correct features extracted from the panchromatic image and align them with the desired features from the high-resolution multispectral image. The calculation process of the modality correction attention module includes: The feature difference operation is performed on the panchromatic image features and multispectral image features obtained after encoding to obtain an attention map of cross-modal feature differences; The modality-specific difference features are obtained by performing element-wise multiplication between the panchromatic image features and the attention map; Subtracting mode-specific difference features from panchromatic image features yields features more closely aligned with multispectral modes. The spatial detail branch, supplementary detail branch, and corrected detail branch are used to calculate the spatial detail image, supplementary detail image, and corrected detail image based on the low-resolution multispectral image and panchromatic image, respectively. The spatial detail branch, supplementary detail branch, and corrected detail branch adopt the same model structure. During training, they are supervised by spatial information of the residual difference between high-resolution multispectral images and low-resolution multispectral images, residual supplementary information of spatial information, and residual difference between low-resolution multispectral images and high-resolution multispectral images, respectively. The feature fusion module is used to fuse spatial detail images, supplementary detail images, corrected detail images, and low-resolution multispectral images to obtain high-resolution multispectral images.

2. The multimodal correction panchromatic sharpening system based on a task allocation method as described in claim 1, characterized in that, The formula for calculating the attention map is: in, For attention maps, Features of a panchromatic image For multispectral image features, This is the modulation transfer function.

3. The multimodal correction panchromatic sharpening system based on a task allocation method as described in claim 1, characterized in that, Encoding processing includes feature extraction and downsampling; The panchromatic image is processed by the decomposition module and 3×3 convolution, then input into the feature extraction module for feature extraction, and then downsampled. The low-resolution multispectral image is pre-upsampled, then input into the feature extraction module for feature extraction, and then downsampled.

4. The multimodal correction panchromatic sharpening system based on a task allocation method as described in claim 3, characterized in that, The feature extraction module includes a 5×5 convolutional layer, a multilayer perceptron, and layer normalization, and embeds residual connections.

5. The multimodal correction panchromatic sharpening system based on a task allocation method as described in claim 1, characterized in that, In the encoder, feature extraction and downsampling are performed multiple times on the low-resolution multispectral image and the panchromatic image, respectively.

6. The multimodal correction panchromatic sharpening system based on a task allocation method as described in claim 1, characterized in that, The decoder consists of alternating feature extraction modules and upsampling layers.

7. The multimodal correction panchromatic sharpening system based on a task allocation method as described in claim 1, characterized in that, Total loss function The calculation formula is: in, For the global loss function, For the spatial detail branch, To supplement the loss function of the detailed branches, To correct the loss function for the detail branches.

8. A multimodal correction panchromatic sharpening method based on task allocation, characterized in that, The implementation of a multimodal correction panchromatic sharpening system based on a task allocation method as described in any one of claims 1 to 7 includes the following steps: S1: Acquire panchromatic and low-resolution multispectral images; S2: Input the panchromatic image and the low-resolution multispectral image into the spatial detail branch, the supplementary detail branch, and the corrected detail branch respectively, and calculate the spatial detail image, the supplementary detail image, and the corrected detail image; S3: Input the spatial detail image, supplementary detail image, and corrected detail image into the feature fusion module, and fuse them with the low-resolution multispectral image to obtain a high-resolution multispectral image.

Images