A method, system, device and medium for full color sharpening of error decomposition network
Through the combination of error decomposition network and mutual information comparison loss function, the problem of non-overlapping spectral responses and error information in full-color sharpening technology is solved, the quality of remote sensing images is improved, and clearer textures and higher perception effects are achieved. .
Patent Information
- Application Number
- CN202410272717.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-11
- Publication Date
- 2024-12-31
- Estimated Expiration
- 2044-03-11
AI Technical Summary
The existing full-color sharpening technology has problems such as non-overlapping spectral responses and error information in full-color images, which affects the quality of remote sensing images.
By using the error decomposition network method, the full-color image is decomposed into its corresponding multispectral image and error map by constructing a decomposition network model, and input it with low spatial resolution multispectral image into the fusion network for full-color sharpening. , optimize the model using mutual information comparison loss function to reduce errors and preserve texture details.
The texture clarity and perceived quality of remote sensing images are improved, the spectral response non-overlapping and error information interferes with the image, and the fusion effect of the image is enhanced.
Smart Images
Figure CN118333896B_ABST
Abstract
Description
Background Art
[0002] Satellites, as a cutting-edge technology, provide a way to understand the Earth. They are widely used in civil and military fields, including environmental monitoring, change detection, and remote sensing image segmentation. However, satellite hardware itself has inherent limitations, making it impossible to obtain high spatial and spectral resolution images at the same time. The main purpose of panchromatic sharpening technology is to fuse high spatial resolution panchromatic images with low spatial resolution multispectral images.
[0003] In the past decade, researchers have proposed a large number of traditional panchromatic sharpening methods, including component replacement, multi-resolution analysis, and variational optimization. However, the performance of these methods is still difficult to meet application requirements. Convolutional neural networks have gained widespread attention in different fields of computer vision due to their powerful feature extraction and feature expression capabilities. Some scholars have improved the subjective and objective quality of fused images by using panchromatic sharpening technology based on convolutional neural networks.
[0004] However, these deep learning-based panchromatic sharpening algorithms generally fuse panchromatic images and multispectral images at the image or feature level, but ignore the unique imaging properties of satellite sensors, namely, there are obvious non-overlapping spectral responses between panchromatic and multispectral images, and the panchromatic image is not equal to the cumulative radiant energy of a single band of its corresponding multispectral image, which indicates that there is error information in the panchromatic image. Summary of the invention
[0005] The technical problem to be solved by the present invention is to address the deficiencies of the prior art, specifically the problems of the existing panchromatic sharpening technology, such as the non-overlapping spectral response and the error information in the panchromatic image, and specifically provide a panchromatic sharpening method, system, device and medium of an error decomposition network, as follows:
[0006] 1) In the first aspect, the present invention provides a panchromatic sharpening method of an error decomposition network, and the specific technical scheme is as follows:
[0007] S1, obtaining an initial satellite remote sensing image of a target area, wherein the initial satellite remote sensing image includes a panchromatic image and an original high spatial resolution multispectral image, and performing a downsampling operation on the original high spatial resolution multispectral image to obtain a low spatial resolution multispectral image;
[0008] S2, constructing a decomposition network model according to the spectral response theory of panchromatic sharpening and a preset error term of the panchromatic image, inputting the panchromatic image into the decomposition network model, and obtaining a pseudo multispectral image of the panchromatic image;
[0009] S3, performing an upsampling operation on the low spatial resolution multispectral image to obtain a multispectral image, and inputting the multispectral image and the pseudo multispectral image into a preset fusion network model in sequence to obtain a final fused remote sensing image of the target area.
[0010] The beneficial effects of the full color sharpening method of the error decomposition network provided by the present invention are as follows:
[0011] The decomposition network model can decompose the panchromatic image into its corresponding multispectral image and error map, and input the low spatial resolution multispectral image and pseudo multispectral image into the fusion network for panchromatic sharpening to obtain a fused image, thereby avoiding the interference of the spectral response non-overlapping of the panchromatic sharpening technology and the error information in the panchromatic image on the remote sensing image of the target area, making the remote sensing image texture clearer, thereby improving the perceived quality of the image.
[0012] Based on the above solution, the present invention can also be improved as follows.
[0013] Furthermore, the S2 further includes:
[0014] A decomposition network model is constructed according to the spectral response theory of panchromatic sharpening and the preset error term of the panchromatic image, and the panchromatic image is input into the decomposition network model to obtain the error of the panchromatic image.
[0015] Furthermore, the step S3 further includes:
[0016] S4, determining a loss function of the decomposition network model according to the remote sensing image, and optimizing the decomposition network model according to the loss function.
[0017] Further, the loss function includes: a reconstruction loss function and a mutual information comparison loss function;
[0018] The mutual information contrast loss function includes: a function for minimizing the mutual information between the pseudo multispectral image and the error and a function for maximizing the mutual information between the pseudo multispectral image and the final fused remote sensing image.
[0019] The beneficial effects of adopting the above scheme are as follows:
[0020] The invention consists of a decomposition network, a fusion network and a mutual information contrast loss function. The mutual information contrast loss function is constructed by supervised learning directly from the decomposition stage to ensure enhanced separation and minimize overlap during the decomposition process, while also retaining the basic information in the decomposition network and restoring spectral and texture details.
[0021] 2) In the second aspect, the present invention also provides a full color sharpening system of an error decomposition network, and the specific technical solution is as follows:
[0022] A computing module is used to obtain an initial satellite remote sensing image of a target area, wherein the initial satellite remote sensing image includes a panchromatic image and an original high spatial resolution multispectral image, and downsample the original high spatial resolution multispectral image to obtain a low spatial resolution multispectral image;
[0023] A decomposition module, used for constructing a decomposition network model according to the spectral response theory of panchromatic sharpening and a preset error term of the panchromatic image, inputting the panchromatic image into the decomposition network model, and obtaining a pseudo multispectral image of the panchromatic image;
[0024] The fusion module is used to upsample the low spatial resolution multispectral image to obtain a multispectral image, and input the multispectral image and the pseudo multispectral image into a preset fusion network model in sequence to obtain a final fused remote sensing image of the target area.
[0025] Based on the above solution, the present invention can also be improved as follows.
[0026] Furthermore, the decomposition module further includes:
[0027] A decomposition network model is constructed according to the spectral response theory of panchromatic sharpening and the preset error term of the panchromatic image, and the panchromatic image is input into the decomposition network model to obtain the error of the panchromatic image.
[0028] Furthermore, it also includes:
[0029] The optimization module is used to determine the loss function of the decomposition network model according to the remote sensing image, and optimize the decomposition network model according to the loss function.
[0030] Further, the loss function includes: a reconstruction loss function and a mutual information comparison loss function;
[0031] The mutual information contrast loss function includes: a function for minimizing the mutual information between the pseudo multispectral image and the error and a function for maximizing the mutual information between the pseudo multispectral image and the final fused remote sensing image.
[0032] 3) In a third aspect, the present invention further provides a computer device, comprising a processor, wherein the processor is coupled to a memory, wherein the memory stores at least one computer program, and wherein the at least one computer program is loaded and executed by the processor so that the computer device implements any of the above methods.
[0033] 4) In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein at least one computer program is stored in the computer-readable storage medium, and the at least one computer program is loaded and executed by a processor so that a computer implements any of the above methods.
[0034] It should be noted that the beneficial effects achieved by the technical solutions of the second to fourth aspects of the present invention and the corresponding possible implementation methods can be found in the above-mentioned technical effects of the first aspect and its corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0036] Figure 1 A schematic diagram of a process flow of an embodiment of the present invention;
[0037] Figure 2 A schematic diagram of a relative spectral response visualization diagram of an embodiment of the present invention;
[0038] Figure 3 A schematic diagram of the process of pan-sharpening of remote sensing images according to an embodiment of the present invention;
[0039] Figure 4 A schematic diagram of the structure of a mutual information comparison loss function according to an embodiment of the present invention;
[0040] Figure 5 A texture diagram of a decomposed version of a full-color image according to an embodiment of the present invention;
[0041] Figure 6 A schematic diagram of the error of a decomposed version of a full-color image according to an embodiment of the present invention;
[0042] Figure 7 Schematic diagram of the device structure of an embodiment of the present invention. DETAILED DESCRIPTION
[0043] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0044] like Figure 1 As shown, a full color sharpening method of an error decomposition network according to an embodiment of the present invention comprises the following steps:
[0045] S1, obtaining an initial satellite remote sensing image of a target area, wherein the initial satellite remote sensing image includes a panchromatic image and an original high spatial resolution multispectral image, and performing a downsampling operation on the original high spatial resolution multispectral image to obtain a low spatial resolution multispectral image;
[0046] S2, constructing a decomposition network model according to the spectral response theory of panchromatic sharpening and a preset error term of the panchromatic image, inputting the panchromatic image into the decomposition network model, and obtaining a pseudo multispectral image of the panchromatic image;
[0047] S3, performing an upsampling operation on the low spatial resolution multispectral image to obtain a multispectral image, and inputting the multispectral image and the pseudo multispectral image into a preset fusion network model in sequence to obtain a final fused remote sensing image of the target area.
[0048] The beneficial effects of the full color sharpening method of the error decomposition network provided by the present invention are as follows:
[0049] The decomposition network model can decompose the panchromatic image into its corresponding multispectral image and error map, and input the low spatial resolution multispectral image and pseudo multispectral image into the fusion network for panchromatic sharpening to obtain a fused image, thereby avoiding the interference of the spectral response non-overlapping of the panchromatic sharpening technology and the error information in the panchromatic image on the remote sensing image of the target area, making the remote sensing image texture clearer, thereby improving the perceived quality of the image.
[0050] like Figure 2 This is a visualization of the spectral response of the IKONOS sensor, where R, G, B and NIR represent the red, green, blue and near-infrared bands of the multispectral image, respectively, and PAN represents the panchromatic image. There is an obvious non-overlapping spectral response between the panchromatic image and the multispectral image, that is, the panchromatic image is not equal to the cumulative radiant energy of a single band of the corresponding multispectral image, which indicates that there is error information in the panchromatic image.
[0051] Spectral response is a key feature of satellite sensors, which effectively quantifies the relationship between the radiant energy and the radiant energy entering the pupil. In theory, the radiant energy of a panchromatic image is equal to the cumulative radiant energy of all bands of the associated multispectral image, which can be determined by the first formula, which is as follows:
[0052] L j =∫L(λ)R j (λ)dλ,
[0053] Where λ is the wavelength, L j represents the radiation energy of the jth band, R j (λ) represents the spectral response at wavelength λ.
[0054] Therefore, the panchromatic image can be approximately regarded as the linear relationship of multispectral bands, which can be determined by the second formula, which is as follows:
[0055]
[0056] Among them, a j represents the jth frequency band The weighting coefficient of Represents the original high spatial resolution multispectral image, where j represents the number of bands.
[0057] The environment used in this embodiment is as follows:
[0058] The server's CPU is Intel Xeon E5-2665, the GPU is NVIDIA GTX2080Ti, the operating system is Ubuntu 18.04, and the compilation environment is PyTorch 1.1.0, Python 3.5, CUDA9.0, and CUDNN7.1.
[0059] The present invention is an example of remote sensing image panchromatic sharpening, the process is as follows Figure 3 The details are as follows:
[0060] S1, obtaining an initial satellite remote sensing image of the target area, wherein the initial satellite remote sensing image includes a panchromatic image and an original high spatial resolution multispectral image, and performing a downsampling operation on the original high spatial resolution multispectral image to obtain a low spatial resolution multispectral image. Wherein:
[0061] The target areas can be: areas that require environmental monitoring, such as streets, communities and parks; areas that require change detection, such as plain airflow and river changes.
[0062] The full-color image can be obtained by shooting the target area with a camera or video camera equipped with a full-color lens module.
[0063] The original high spatial resolution multispectral image can be obtained by shooting the target area with a camera or video camera equipped with a multispectral lens module.
[0064] The full color image is represented as The original high-resolution multispectral image is represented as I P and I HRMS have the same spatial resolution, where H and W represent height and width, and C represents the number of bands of the image.
[0065] The downsampling operation is to downsample the original high spatial resolution multispectral image by 4 times to obtain a low spatial resolution multispectral image. Downsample by 4 times to get I LRMS and I HRMS has the same spectral resolution, but I HRMS The spatial resolution is I LRMS 4 times of that.
[0066] S2, constructing a decomposition network model according to the spectral response theory of panchromatic sharpening and the preset error term of the panchromatic image, inputting the panchromatic image into the decomposition network model, and obtaining a pseudo multispectral image of the panchromatic image and the error of the panchromatic image.
[0067] The spectral response theory of pan-sharpening is: That is, the panchromatic image can be obtained by linearly weighting the corresponding multispectral image;
[0068] The decomposition network model includes the third formula, which is as follows:
[0069]
[0070] Among them, I P represents a full-color image, represents the original high spatial resolution multispectral image of the jth band, Represents the error map of the j-th band.
[0071] The pseudo multispectral image of the panchromatic image is determined by the fourth formula, which is as follows:
[0072]
[0073] Among them, H unmix (I p ) represents the operation of decomposing the network, I MS′ It represents the pseudo multispectral image of size C×4H×4W obtained by the decomposition network, I Err Represents the error of the full-color image. It should be noted that I Err The error is discarded and no longer participates in the subsequent information fusion stage, but is needed in the subsequent optimization of the decomposition network model.
[0074] S3, performing an upsampling operation on the low spatial resolution multispectral image to obtain a multispectral image, and inputting the multispectral image and the pseudo multispectral image into a preset fusion network model in sequence to obtain a final fused remote sensing image of the target area.
[0075] The upsampling operation is: upsampling the low spatial resolution multispectral image by 4 times to obtain a multispectral image. LRMS Upsampling by 4 times yields an I with specifications of 4×4H×4W LRMS′ .
[0076] The preset fusion network model includes the fifth formula, which is as follows:
[0077]
[0078] Among them, Ips Represents the final fused remote sensing image of the target area, I MS′ represents pseudo multispectral image, I LRMS′ represents multispectral image, H fus (I MS’ , I LRMS’ ) represents the fusion network model composed of convolutional neural network. MS′ and I LRMS′ Perform fusion operation.
[0079] Furthermore, the step S3 further includes:
[0080] S4, determining a loss function of the decomposition network model according to the remote sensing image, and optimizing the decomposition network model according to the loss function.
[0081] Further, the loss function includes: a reconstruction loss function and a mutual information comparison loss function;
[0082] The mutual information contrast loss function includes: a function for minimizing the mutual information between the pseudo multispectral image and the error and a function for maximizing the mutual information between the pseudo multispectral image and the final fused remote sensing image.
[0083] The reconstruction loss function is determined by the sixth formula, which is as follows:
[0084]
[0085] in, Represents the reconstruction loss function, which is the mean absolute error function, I HRMS represents the original high-resolution multispectral image, I ps The final fused remote sensing image representing the target area.
[0086] The mutual information contrast loss function is determined by the seventh formula, which is as follows:
[0087]
[0088] in, is the weakly supervised contrast loss function, the first term in the seventh formula In order to minimize the pseudo multispectral image generated by the decomposition network model (I MS′ ) and error (I Err ), the other two items ( and ) In order to maximize the decomposition of the pseudo multispectral image generated by the network model and the original image (I HRMS and I PS). In other words, the first term in the seventh formula is to minimize the mutual information between the pseudo multispectral image and the error, and the second term and the third term in the seventh formula are to maximize the mutual information between the pseudo multispectral image and the panchromatic image.
[0089] The first term is to maximize the anchor point (I MS′ ) and negative samples (I Err ), and the latter two items are intended to reduce the distance between the anchor points (I MS′ ) and the distance between the positive samples (I HRMS and I PS ), is the mutual information loss of vectors v1 and v2 of the input formula, determined by the eighth formula, which is as follows:
[0090]
[0091] in, It is the Kullback-Leibler divergence calculation operation, which is used to measure the difference between two probability distributions. H(v1) represents the binary cross entropy of vector v1, H(v2) represents the binary cross entropy of vector v2, H(v1, v2) represents the binary cross entropy of vectors v1 and v2, and I1 represents the probability distribution.
[0092] The structure of the mutual information contrast loss function is as follows: Figure 4 shown.
[0093] The anchor points and samples (positive and negative samples) are input into the convolutional neural network. The network mainly includes three convolutional layers, which mainly convert the input image into a feature vector to obtain the latent vector (f lv ). In order to avoid the non-differentiability of the intermediate nodes of gradient propagation, the following formula is used to generate the resampled potential vector z, and finally the potential vector z corresponding to the input sample is calculated to express the mutual information contrast loss value. Determined by the ninth formula, the ninth formula is as follows:
[0094] μ=tanh(fc u (f lv )),
[0095] σ=tanh(fc σ (f lv )),
[0096] z=μ+σ∈,
[0097] Among them, f lv represents the hidden vector, fc(.) represents the fully connected layer function, fc u (f lv ) indicates the lvPerform full-connected layer function calculation, tanh(.) represents the hyperbolic tangent activation function, tanh(fc u (f lv )) indicates fc u (f lv ) performs hyperbolic tangent activation function calculation, μ represents f lv The mean of f lv The variance of , ∈ is the sampling data of N~(0,1).
[0098] In summary, the loss function is determined by the tenth formula, which is as follows:
[0099]
[0100] in, represents the reconstruction loss function, L con Represents the mutual information contrast loss function. The proposed method guides the optimization of the model by minimizing the loss function, thereby more accurately decomposing the panchromatic image, reducing the pseudo-multispectral image, and fusing the multispectral image to obtain rich texture information.
[0101] like Figure 5 and Figure 6 is a decomposed version of the full-color image. Figure 5 Used to assist texture generation, contains rich texture information and has significant differences between them; Figure 6 For error information.
[0102] The present invention also provides a full color sharpening system of an error decomposition network, and the specific technical scheme is as follows:
[0103] A computing module is used to obtain an initial satellite remote sensing image of a target area, wherein the initial satellite remote sensing image includes a panchromatic image and an original high spatial resolution multispectral image, and downsample the original high spatial resolution multispectral image to obtain a low spatial resolution multispectral image;
[0104] A decomposition module, used for constructing a decomposition network model according to the spectral response theory of panchromatic sharpening and a preset error term of the panchromatic image, inputting the panchromatic image into the decomposition network model, and obtaining a pseudo multispectral image of the panchromatic image;
[0105] The fusion module is used to upsample the low spatial resolution multispectral image to obtain a multispectral image, and input the multispectral image and the pseudo multispectral image into a preset fusion network model in sequence to obtain a final fused remote sensing image of the target area.
[0106] Based on the above solution, the present invention can also be improved as follows.
[0107] Furthermore, the decomposition module further includes:
[0108] A decomposition network model is constructed according to the spectral response theory of panchromatic sharpening and the preset error term of the panchromatic image, and the panchromatic image is input into the decomposition network model to obtain the error of the panchromatic image.
[0109] Furthermore, it also includes:
[0110] The optimization module is used to determine the loss function of the decomposition network model according to the remote sensing image, and optimize the decomposition network model according to the loss function.
[0111] Further, the loss function includes: a reconstruction loss function and a mutual information comparison loss function;
[0112] The mutual information contrast loss function includes: a function for minimizing the mutual information between the pseudo multispectral image and the error and a function for maximizing the mutual information between the pseudo multispectral image and the final fused remote sensing image.
[0113] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given by the present invention. Those skilled in the art may adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.
[0114] It should be noted that the beneficial effects of the full-color sharpening system of the error decomposition network provided in the above embodiment are the same as the beneficial effects of the full-color sharpening method of the error decomposition network provided in the above embodiment, which will not be repeated here. In addition, when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to actual conditions to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0115] like Figure 7 As shown, a computer device 300 according to an embodiment of the present invention includes a processor 320, the processor 320 is coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the computer device 300 implements any of the above methods, specifically:
[0116] The computer device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors 320 (Central Processing Units, CPU) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the computer device 300 implements a full color sharpening method of an error decomposition network provided in the above embodiment. Of course, the computer device 300 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the computer device 300 may also include other components for implementing device functions, which will not be described in detail here.
[0117] A computer-readable storage medium according to an embodiment of the present invention stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that a computer implements any one of the above methods.
[0118] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.
[0119] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned full color sharpening methods of the error decomposition network.
[0120] Those skilled in the art know that the present invention can be implemented as a system, method or computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, namely: it can be complete hardware, it can be complete software (including firmware, resident software, microcode, etc.), or it can be a combination of hardware and software, which is generally referred to as "circuit", "module" or "system" herein. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, and the computer-readable medium contains computer-readable program code.
[0121] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.
[0122] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A panchromatic sharpening method using an error decomposition network, characterized in that: include: S1, obtaining an initial satellite remote sensing image of a target area, wherein the initial satellite remote sensing image includes a panchromatic image and an original high spatial resolution multispectral image, and performing a downsampling operation on the original high spatial resolution multispectral image to obtain a low spatial resolution multispectral image; S2, constructing a decomposition network model according to the spectral response theory of panchromatic sharpening and a preset error term of the panchromatic image, inputting the panchromatic image into the decomposition network model, and obtaining a pseudo multispectral image of the panchromatic image; S3, performing an upsampling operation on the low spatial resolution multispectral image to obtain a multispectral image, and inputting the multispectral image and the pseudo multispectral image into a preset fusion network model in sequence to obtain a final fused remote sensing image of the target area; Wherein, the S2 further includes: Constructing a decomposition network model according to the spectral response theory of panchromatic sharpening and the preset error term of the panchromatic image, inputting the panchromatic image into the decomposition network model, and obtaining the error of the panchromatic image; The S3 also includes: S4, determining a loss function of the decomposition network model according to the remote sensing image, and optimizing the decomposition network model according to the loss function; The loss functions include: reconstruction loss function and mutual information comparison loss function; The mutual information contrast loss function includes: a function for minimizing the mutual information between the pseudo multispectral image and the error and a function for maximizing the mutual information between the pseudo multispectral image and the final fused remote sensing image; The reconstruction loss function is: L r =||I HRMS -I PS ||1; Among them, L r represents the reconstruction loss function, I HRMS represents the original high-resolution multispectral image, I ps The final fused remote sensing image representing the target area; The mutual information contrast loss function is: in, is the weakly supervised contrast loss function, Minimizing the mutual information between the pseudo multispectral image and the error, as well as Used to maximize the mutual information between the pseudo multispectral image and the final fused remote sensing image; The loss function is: in, represents the reconstruction loss function, L con represents the mutual information contrast loss function.
2. A panchromatic sharpening system of an error decomposition network, characterized in that: include: A computing module is used to obtain an initial satellite remote sensing image of a target area, wherein the initial satellite remote sensing image includes a panchromatic image and an original high spatial resolution multispectral image, and downsample the original high spatial resolution multispectral image to obtain a low spatial resolution multispectral image; A decomposition module, used for constructing a decomposition network model according to the spectral response theory of panchromatic sharpening and a preset error term of the panchromatic image, inputting the panchromatic image into the decomposition network model, and obtaining a pseudo multispectral image of the panchromatic image; A fusion module is used to perform an upsampling operation on the low spatial resolution multispectral image to obtain a multispectral image, and input the multispectral image and the pseudo multispectral image into a preset fusion network model in sequence to obtain a final fused remote sensing image of the target area; Wherein, the decomposition module also includes: Constructing a decomposition network model according to the spectral response theory of panchromatic sharpening and the preset error term of the panchromatic image, inputting the panchromatic image into the decomposition network model, and obtaining the error of the panchromatic image; Also includes: An optimization module, used to determine the loss function of the decomposition network model according to the remote sensing image, and optimize the decomposition network model according to the loss function; The loss functions include: reconstruction loss function and mutual information comparison loss function; The mutual information contrast loss function includes: a function for minimizing the mutual information between the pseudo multispectral image and the error and a function for maximizing the mutual information between the pseudo multispectral image and the final fused remote sensing image; The reconstruction loss function is: in, represents the reconstruction loss function, I HRMS represents the original high-resolution multispectral image, I ps The final fused remote sensing image representing the target area; The mutual information contrast loss function is: in, is the weakly supervised contrast loss function, Minimizing the mutual information between the pseudo multispectral image and the error, as well as Used to maximize the mutual information between the pseudo multispectral image and the final fused remote sensing image; The loss function is: in, represents the reconstruction loss function, L con represents the mutual information contrast loss function.
3. A computer device, characterized in that: The computer device comprises a processor, the processor is coupled to a memory, at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the computer device implements the method according to claim 1.
4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that the computer implements the method according to claim 1.