Panchromatic image sharpening method based on degradation process consistency

Abstract

The invention relates to the technical field of image enhancement, and discloses a panchromatic image sharpening method based on degradation process consistency, which comprises the following steps: theoretically analyzing a consistency relationship of a degradation process, simultaneously constructing a spectrum degradation network and a spatial resolution degradation network, introducing an operator learning method in the construction of a spectrum model, and constructing a panchromatic image. Constructing an accurate mapping function from a multispectral image to a panchromatic image, and decoupling spectral features and texture features; then, through joint training, an operator and a convolutional network, respectively learning a space degradation process and a spectrum degradation process under full resolution; and the consistency constraint is constructed by introducing the images, so that the panchromatic image sharpening network can be trained under the original full resolution. The unsupervised training constructed by the invention can be applied to the existing panchromatic image method, the availability of the panchromatic image method under the original resolution scale is improved, and the quality of the generated high-resolution multispectral image is improved.

Description

Technical Field

[0001] The invention relates to the technical field of image enhancement, and in particular to a full-color image sharpening method based on consistency of degradation process. Background Art

[0002] Satellite imaging systems typically feature multispectral sensors that allow observations across multiple wavelengths. However, due to mechanical limitations of sensor design, achieving high spatial and spectral resolutions while maintaining sufficient signal-to-noise ratio is a technical challenge. Satellites simultaneously acquire high-resolution panchromatic images and low-resolution multispectral image capture features. To overcome this limitation, panchromatic image sharpening techniques have been proposed in the prior art, with the goal of aligning and fusing the features of these two images to generate high-resolution multispectral composite images, which is convenient for downstream tasks in areas such as agriculture and environmental monitoring.

[0003] There are now a variety of panchromatic image sharpening techniques, including traditional model methods that rely on hand-crafted priors, and modern methods based on deep learning. Although deep learning methods are superior to hand-crafted models in representation and generalization capabilities, existing deep learning panchromatic image sharpening methods still face the challenge of lacking real high-resolution multispectral images as supervisory information. Figure 1 As shown in Figure 1, in order to solve this problem, researchers usually use techniques such as bicubic downsampling of the original high-resolution multispectral images to generate panchromatic and multispectral image pairs with reduced resolution scales as pseudo-true values ​​for training the network.

[0004] In fact, the above-mentioned method of synthesizing data is based on the spatial resolution degradation process set up artificially by experience, which is not accurate enough compared to the complex imaging mechanism in reality. At the same time, the network trained on the reduced resolution scale also has a scale leap when generalized to the original scene, and cannot guarantee high-quality images at full resolution. The existing full-color image sharpening method has no fundamental defects in network design. The problem mainly lies in the use of imprecise synthetic data training methods, which introduces scale inconsistency and affects the quality and applicability of the final image. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a panchromatic image sharpening method based on the consistency of the degradation process. This method does not rely on pseudo-true values, but uses the invariance of the degradation process to model a true and accurate spectral degradation network and a spatial degradation network, and uses these two models to construct a consistent loss function on the original scale for network training to avoid the problem of scale crossing.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A panchromatic image sharpening method based on consistency of degradation process is proposed. The consistency characteristics of spatial resolution degradation process and spectral resolution degradation process in different processes are used to complete modeling, so as to replace the missing true value for supervision as a constraint trained at the original scale. The input high-resolution panchromatic image and low-resolution multispectral image are output as a high-resolution multispectral image. The method specifically includes the following steps:

[0008] Step 1: construct a spectral degradation network and a spatial degradation network, input the high-resolution panchromatic image P and the low-resolution multispectral image M into the spatial degradation network and the spectral degradation network respectively, and both the spatial degradation network and the spectral degradation network output a low-resolution panchromatic image LP with consistent spatial resolution and spectral band;

[0009] Step 2: Joint training of degradation networks: Based on the low-resolution panchromatic image LP output by the spatial degradation network and the spectral degradation network, the joint training of the spatial degradation network and the spectral degradation network is guided by consistency constraints. A pre-trained panchromatic image sharpening network is added to the joint training to guide the initialization of the spatial degradation network and the spectral degradation network.

[0010] Step 3: Panchromatic image sharpening network training: The panchromatic image sharpening network extracts the information of the high-resolution panchromatic image P and the low-resolution multispectral image M, and fuses them to generate a high-resolution multispectral image; the generated high-resolution multispectral image is input into the spatial degradation network and spectral degradation network that have completed joint training, and the loss function of the panchromatic image sharpening network is constructed based on the high-resolution panchromatic image P and the low-resolution multispectral image M. Guides the training of a panchromatic image sharpening network.

[0011] Furthermore, step one specifically includes:

[0012] The spectral degradation network uses operator learning methods to model the spectral resolution degradation process, and the spatial degradation network uses a convolutional combination network to fit the spatial resolution degradation process;

[0013] LP = Fd(M) = Ds(P);

[0014] The low-resolution multispectral image M and the high-resolution panchromatic image P are respectively subjected to the spectral degradation network Fd and the spatial degradation network Ds to obtain a consistent low-resolution panchromatic image LP.

[0015] Furthermore, step 2 specifically includes:

[0016] The low-resolution multispectral image M and the high-resolution panchromatic image P are respectively input into the spectral degradation network and the spatial degradation network for downsampling of spectral information and spatial information to form two branches of joint training; the consistency constraint of the output images of the two branches of joint training is established, thereby obtaining a joint learning framework that does not require true value supervision, and the consistency loss function for:

[0017]

[0018] ‖·‖1 represents the absolute mean error, Fd(·) represents the spectral degradation network, and Ds(·) represents the spatial degradation network. In order to introduce constraints without the true value, a trained panchromatic image sharpening network Pr is added. The high-resolution multispectral image HM predicted by the panchromatic image sharpening network Pr can be obtained by inputting the low-resolution multispectral image M and the high-resolution panchromatic image P, and the following loss function is introduced:

[0019]

[0020] represents the panchromatic loss function, represents the multispectral loss function;

[0021] The above loss functions are combined into the total loss function for joint training of the degradation network

[0022]

[0023] α1, β1, and β2 are three adjustable hyperparameters.

[0024] Furthermore, step three specifically includes:

[0025] The high-resolution panchromatic image P and the low-resolution multispectral image M are input into the panchromatic image sharpening network to generate the predicted high-resolution multispectral image HM pre , using the already trained spectral degradation network and spatial degradation network to HM pre Perform degradation operation to obtain the predicted high-resolution full-color image P pre and low-resolution multispectral image M pre :

[0026] M pre =Ds(HM pre );

[0027] P pre =Fd(HM pre );

[0028] Building a loss function for the panchromatic image sharpening network

[0029]

[0030] Among them, α and β are weight factors.

[0031] Compared with the prior art, the beneficial technical effects of the present invention are:

[0032] 1. This invention constructs an unsupervised optimization framework at the original scale for the first time from a new perspective of analyzing the consistency of the degradation process. By exploring the characteristics of the spatial resolution degradation process and the spectral resolution degradation process, the network is established to fit the network model of the two degradation processes using their same mapping properties in different processes, and then the trained degradation model is used to establish constraints for the training of the full-color image sharpening network. This invention can be applied to any proposed full-color image sharpening network to improve its generation capability at full resolution.

[0033] 2. This invention proposes a method for modeling the spectral resolution degradation process using an operator learning method. Compared with the existing linear combination method, the operator learning method can effectively model nonlinear relationships, and the form of the modeling operator can approach the essence of the imaging principle, which is easy to generalize to multiple data sets and multiple tasks.

[0034] 3. The present invention actually gets rid of the limitation of relying on high-resolution panchromatic images and low-resolution multispectral images to synthesize pseudo-true values, and can improve the generation effect of the network at the original scale without adding any network parameters. The present invention has shown excellent performance on several data sets, and has achieved significant improvement in generation quality when applied to existing panchromatic image sharpening algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A comparison diagram of a full-color image sharpening method framework provided according to an embodiment of the present invention and a traditional method framework;

[0036] Figure 2 A schematic diagram of a framework of a full-color image sharpening method provided according to an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of image degradation relationship at the original scale provided according to an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of specific structures of two degenerate networks provided according to an embodiment of the present invention;

[0039] Figure 5 It is a schematic diagram of a specific fitting effect of a spectral degradation network provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0041] The present invention proposes a panchromatic image sharpening method based on the consistency of the degradation process, in order to use the consistency characteristics of the spatial resolution degradation process and the spectral degradation process in different processes to complete the modeling, so as to replace the missing true value as a constraint for training the network at the original scale for supervision. The training framework proposed by the present invention is simple and effective. It only needs to jointly train the spatial degradation network and the spectral degradation network proposed by the present invention on the used data set, and then place the existing panchromatic image sharpening network in the training framework of the present invention for training at the original resolution scale, so as to improve its generation ability in actual application scenarios.

[0042] like Figure 1 As shown, the present invention is applied to the field of image enhancement technology, exploring the consistency of the degradation process of remote sensing images for more accurate modeling, and using it as a constraint to avoid the adverse effects of training at a reduced resolution scale. Based on a new perspective of exploring the relationship between degradation processes to solve the task of panchromatic image sharpening, the present invention develops an unsupervised training framework.

[0043] Firstly, in order to simulate the spectral degradation process, the present invention constructs an operator learning model that is independent of resolution to characterize the pixel-level spectral mapping relationship between multispectral and panchromatic images, and establishes an improved spectral observation model.

[0044] In addition, the present invention constructs a convolutional layer-based downsampling network to replace the traditional bicubic interpolation to learn resolution-related spatial degradation.

[0045] The above two degradation networks are jointly trained at the original full resolution. The low-resolution panchromatic image output generated by the spatial degradation network should match the panchromatic image output obtained by the multispectral image through the spectral degradation model.

[0046] After the degradation networks are trained, their outputs can provide consistency constraints during the optimization of the full-resolution pan-sharpening network, avoiding the problem of cross-scale variations.

[0047] The present invention comprises the following steps:

[0048] S1: Perform consistency analysis of the degradation process at the original resolution scale. The present invention theoretically analyzes the consistency of the degradation process to establish a joint training paradigm at the original resolution scale, specifically including:

[0049] The present invention gives Figure 3Two conclusions are proposed for remote sensing image transformation in: 1. Scale invariance: It is assumed that the spectral degradation process and the spatial degradation process are invariant under a fixed scale. This means that regardless of the resolution of the image, the spectral and spatial degradation processes follow the same physical laws. 2. Interchangeability: It is assumed that the spectral degradation process and the spatial degradation process can be exchanged under certain conditions without affecting the final degradation result. This provides a theoretical basis for jointly training the degradation network.

[0050] The following is a proof of the above two conclusions. First, referring to the existing descriptions of the two degradation processes, the spatial downsampling process is usually adjusted using the bicubic operator. During the processing of this operator, the channels of the image are independent of each other. The spectral downsampling process usually uses a weighted linear combination between pixels. It is easy to find that this setting is consistent with the above conclusions.

[0051] Figure 3 The degradation process at full resolution scale is summarized:

[0052] M = Ds1 (MS), P = Fd1 (MS);

[0053] LP = Fd2(M), LP = Ds2(P);

[0054] HM is a high-resolution multispectral image, M is a low-resolution multispectral image, P is a high-resolution panchromatic image, LP is a low-resolution panchromatic image, Ds1 and Ds2 are two spatial resolution degradation processes, and Fd1 and Fd2 are two spectral resolution degradation processes.

[0055] It is not difficult to get the above formula:

[0056] Fd2(Ds1(MS))=LP=Ds2(Fd1(MS));

[0057] By analyzing the imaging mechanism of the element and combining the general theory of spatial resolution degradation, we can get the interchangeability in the conclusion. Combining the above formulas, we can get the conclusion:

[0058] Ds1(·)=Ds2(·);

[0059] Fd1(·)=Fd2(·);

[0060] That is the consistency of the degradation process in the conclusion, which is the basis for the subsequent experimental design of the present invention.

[0061] S2: Two degradation process modules are designed, which can respectively model the spatial resolution degradation process and the spectral degradation process at the original resolution scale. In order to avoid the errors that may be caused by cross-scale training, a joint training paradigm is established at the original resolution scale in combination with the consistency analysis of the degradation process; specifically, the following steps are included:

[0062] S2.1: Design two degradation process modules, which respectively fit the spatial resolution degradation process and the spectral degradation process. The specific implementation is:

[0063] The spectral degradation process refers to the mapping of multispectral images to panchromatic images at the same resolution. Based on the imaging mechanism of the photosensitive element, existing methods set it as a linear weighted form. This model is a reasonable guess, but the errors that have been found prove that they cannot fully capture the nonlinear transformation between multispectral and panchromatic images at fine granularity. In order to better fit, an operator learning method is introduced.

[0064] The operator learning method has shown strong performance in fitting nonlinear functions. The present invention chooses to use this to model the spectral conversion process with complex mechanisms. In order to decouple the spectral features from the spatial relationship, the image is decomposed into pixels and input into the DeepONet framework. The parameters are set to 3 layers of FNN to train a process from 1×4 pixel values ​​to 1×1 single pixel values. The operator learning method uses discrete data to try to fit continuous operators. This form of training can train a network with a simple structure to obtain better results. After the training is completed, the neuron parameter values ​​in the FNN are copied to the 1×1 convolution kernel, thereby directly converting the pixel-level mapping into a network structure, which serves as the spectral degradation network used to fit the spectral degradation relationship in the present invention. The spectral degradation modeling was tested using the above network, and the experimental results are as follows. Figure 4 As shown in Figure 2. The purpose of this experiment is to fit the pixel intensity value corresponding to the panchromatic image using the spectral corresponding value of each pixel in the multispectral image. It can be seen that the predicted panchromatic image pixel value is very close to the actual value, which shows the effectiveness of modeling using the operator learning method.

[0065] There are many excellent methods for the spatial degradation process in the development of image processing technology. Ordinary convolution kernels can directly process images for further training. Therefore, for the spatial degradation process, the constructed degradation network consists of several simple convolutional networks, including convolution operations using point spread functions and spatial downsampling operations. The specific network structure is described in Figure 3 It is also displayed in.

[0066] The above two processes can be described as: LP = Fd(M) = Ds(P), which means that the original image pair, the low-resolution multispectral image M and the high-resolution panchromatic image P, will obtain a consistent low-resolution panchromatic image LP after the degradation network respectively.

[0067] S2.2: Design a joint training process for two degradation process modules, hoping to use the consistency relationship of the degradation process to establish a training process at the original resolution scale. At the same time, the specific implementation is:

[0068] The low-resolution multispectral image M and the high-resolution panchromatic image P are respectively input into the spectral degradation network and the spatial degradation network for downsampling of spectral information and spatial information, forming two branches for joint training. According to the consistency analysis in S1, these two processes should output the same low-resolution panchromatic image. Based on this theory, the consistency constraint of the output images of the two branches can be established, thereby obtaining a joint learning framework that does not require true value supervision, and the loss function is:

[0069]

[0070] ‖·‖1 represents the absolute mean error, Fd(·) represents the output result of the input image after passing through the spectral degradation network, and Ds(·) represents the output result of the input image after passing through the spatial degradation network.

[0071] However, such constraints are not strict enough for the training of the two degradation process networks without initialization, and the network can easily collapse when the expected results are not obtained. Therefore, it is necessary to design some other constraints in the loss function to prevent the degradation of the generated image. In order to introduce stronger constraints without the premise of true value, a trained unsupervised model Pr is selected to be added. The input image pair can obtain the high-resolution multispectral image HM predicted by the unsupervised model. Using this picture, the following two new constraints can be introduced.

[0072]

[0073] Where Pr(·) represents the output of the pre-trained unsupervised full-color image sharpening network for the image pair.

[0074] In a preferred embodiment, the full-color image sharpening network adopts the P2Sharpen network.

[0075] The combination of the above three constraints can effectively initialize the two degradation module networks, which together constitute the loss function of the first stage joint training:

[0076]

[0077] α1, β1, and β2 are three adjustable hyperparameters. When the network parameters are close to stability, the values ​​of β1 and β2 are reduced to minimize the influence of the pre-trained network on the network learning. After joint training, the degradation network model can effectively fit the spectral degradation process and the spatial degradation process respectively.

[0078] S3: Design an unsupervised training framework for panchromatic image sharpening. In the design of step S2, a degradation process fitting module is obtained through a series of designs and pre-training. A framework is designed to use these two modules to establish constraints on the generated image at the original resolution scale for supervised training; specifically, it includes:

[0079] Set the panchromatic sharpening network you want to train, whose input is a low-resolution multispectral image M and a high-resolution panchromatic image P, and the output is the predicted high-resolution multispectral image HM pre , modeled as:

[0080] HM pre = PanNet(M,P);

[0081] In the traditional supervision method, the HM image needs to be compared, which is unreasonable in the absence of true value supervision. In this case, the degradation model trained in S2 of the present invention is used to compare the MS pre Degrade again to get the predicted low-resolution multispectral image M pre and high-resolution full-color image P pre :

[0082] M pre =Ds(HM pre );

[0083] P pre =Fd(HM pre );

[0084] The corresponding truth values ​​of these two are available, so a clear learning goal can be established at this time:

[0085]

[0086] In the complete training, the above two learning objectives need to be integrated and organized into:

[0087]

[0088] Where α and β are weight factors, which are set to 0.5 based on experience. At this point, the present invention constructs an unsupervised training framework for full-color image sharpening at the original resolution scale. From the above analysis and design, it is easy to conclude that the framework of the present invention has nothing to do with the full-color sharpening network itself. This is emphasized because the present invention believes that the reason why the existing methods are not satisfactory in actual application scenarios is that the existing methods use a strategy of supervised learning at reduced resolution, which has nothing to do with the network design itself.

[0089] S4: Design an existing method to add to the designed training framework and perform training at the original resolution scale to verify that the proposed training framework can improve the network's ability to generate high-quality images. At the same time, design experiments to verify the universality of the invented method for different networks, which specifically includes the following steps:

[0090] S4.1: The unsupervised training framework proposed in the present invention tests the performance improvement after inserting a variety of different full-color image sharpening networks. The network is trained and evaluated on two satellite image datasets, including WorldView II and GaoFen-2. In order to prove the effectiveness of the present invention, two experiments are set up. The original images are directly cropped and aligned and used as datasets at full resolution for testing. The WorldView II dataset includes 2779 training sample pairs and 198 test sample pairs, and the GaoFen-2 dataset includes 2792 training sample pairs and 200 test sample pairs. At the same time, in order to achieve more accurate quantitative evaluation and analyze the reasons why the existing methods have poor generation capabilities at the original resolution scale, the dataset is generated at a reduced resolution scale in accordance with the Wald protocol.

[0091] S4.2: In the second stage of training, they are optimized by the Adam optimizer over 500 epochs with a batch size of 16. The learning rate is initialized to 1×10 -5 , and decayed by 0.5 when it reaches 200 times. In this process, the present invention selected several methods for experiments to verify the versatility and effectiveness of the method of the present invention, including PNN, PANNET, HMDCNN, SRPPNN, SFIINET and GPPNN. The present invention performs training settings according to the original settings of the above method, including the update of the learning rate, the number of training epochs and the patch size. The entire experiment was carried out on an NVIDIA GeForce RTX 3090 graphics card. In the experiment at full resolution, since there is no reference image, the reference-free spectral distortion index QNR and the spectral distortion index D are used. λ and spatial distortion index D s In the downscaling experiment, the peak signal-to-noise ratio (PSNR), the structural similarity parameter (SSIM), the error relative global synthesis dimension (ERGAS) and the spectral angle mapper (SAM) are used as evaluation indicators.

[0092] In order to further prove the theory of the degradation process characteristics of the inventive method for the full color sharpening task, this embodiment also attempts to change the downsampling method of the traditional generation of downscaled resolution data, and cross-compares different generated data with two training methods. The experimental results show that due to the influence of the spatial downsampling prior on the traditional training method, when the data set generation mode changes, it will be greatly affected, while the framework proposed by the present invention will not be greatly affected, which strongly proves that the existing training method will learn the prior in the form of bicubic downsampling, which is not conducive to the generation of actual tasks.

[0093] In summary, the present invention explores for the first time an unsupervised training framework that uses the consistency of the degradation process to completely eliminate scale differences in the field of panchromatic image sharpening. The present invention introduces a framework that fully utilizes the relationship between existing raw data to establish consistency constraints at the full-resolution scale for the panchromatic image sharpening task. The first step is to introduce operator learning and other methods to model it after analyzing the properties of the degradation process, and jointly train the two models, while the second step is to provide supervision for the panchromatic image sharpening task by using the trained model, thereby achieving training at full resolution. The framework of the present invention is compatible with various existing panchromatic image sharpening methods and does not increase parameters or reduce the inference speed. A large number of experiments have shown that the framework of the present invention stably enhances the generation quality of high-resolution multispectral images at the original scale while maintaining network efficiency, which is crucial for applications in precision agriculture, infrastructure assessment and other fields.

[0094] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0095] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A panchromatic image sharpening method based on consistency of degradation process, characterized in that: The modeling is completed by using the consistency characteristics of the spatial resolution degradation process and the spectral resolution degradation process in different processes, so as to replace the missing true value as a constraint trained at the original scale for supervision, and the input high-resolution panchromatic image and low-resolution multispectral image are output as a high-resolution multispectral image, which specifically includes the following steps: Step 1: construct a spectral degradation network and a spatial degradation network, input the high-resolution panchromatic image P and the low-resolution multispectral image M into the spatial degradation network and the spectral degradation network respectively, and both the spatial degradation network and the spectral degradation network output a low-resolution panchromatic image LP with consistent spatial resolution and spectral band; Step 2: Joint training of degradation networks: Based on the low-resolution panchromatic image LP output by the spatial degradation network and the spectral degradation network, the joint training of the spatial degradation network and the spectral degradation network is guided by consistency constraints. A pre-trained panchromatic image sharpening network is added to the joint training to guide the initialization of the spatial degradation network and the spectral degradation network. Step 3: Panchromatic image sharpening network training: The panchromatic image sharpening network extracts the information of the high-resolution panchromatic image P and the low-resolution multispectral image M, and fuses them to generate a high-resolution multispectral image; the generated high-resolution multispectral image is input into the spatial degradation network and spectral degradation network that have completed joint training, and the loss function of the panchromatic image sharpening network is constructed based on the high-resolution panchromatic image P and the low-resolution multispectral image M. Guides the training of a panchromatic image sharpening network.

2. The method for panchromatic image sharpening based on consistency of degradation process according to claim 1, characterized in that: Step 1 specifically includes: The spectral degradation network uses operator learning methods to model the spectral resolution degradation process, and the spatial degradation network uses a convolutional combination network to fit the spatial resolution degradation process; LP = Fd(M) = Ds(P); The low-resolution multispectral image M and the high-resolution panchromatic image P are respectively subjected to the spectral degradation network Fd and the spatial degradation network Ds to obtain a consistent low-resolution panchromatic image LP.

3. The method for sharpening a full-color image based on consistency of a degradation process according to claim 1, characterized in that: Step 2 specifically includes: The low-resolution multispectral image M and the high-resolution panchromatic image P are respectively input into the spectral degradation network and the spatial degradation network for downsampling of spectral information and spatial information to form two branches of joint training; the consistency constraint of the output images of the two branches of joint training is established, thereby obtaining a joint learning framework that does not require true value supervision, and the consistency loss function for: ‖·‖1 represents the absolute mean error, Fd(·) represents the spectral degradation network, and Ds(·) represents the spatial degradation network. In order to introduce constraints without the true value, a trained panchromatic image sharpening network Pr is added. The high-resolution multispectral image HM predicted by the panchromatic image sharpening network Pr can be obtained by inputting the low-resolution multispectral image M and the high-resolution panchromatic image P, and the following loss function is introduced: represents the panchromatic loss function, represents the multispectral loss function; The above loss functions are combined into the total loss function for joint training of the degradation network α1, β1, and β2 are three adjustable hyperparameters.

4. The method for sharpening a full-color image based on consistency of a degradation process according to claim 1, characterized in that: Step three specifically includes: The high-resolution panchromatic image P and the low-resolution multispectral image M are input into the panchromatic image sharpening network to generate the predicted high-resolution multispectral image HM pre , using the already trained spectral degradation network and spatial degradation network to HM pre Perform degradation operation to obtain the predicted high-resolution full-color image P pre and low-resolution multispectral image M pre : M pre =Ds(HM pre ); P pre =Fd(HM pre ); Building a loss function for the panchromatic image sharpening network Among them, α and β are weight factors.

Images