Optical remote sensing satellite image fusion method, computer device and readable storage medium

Abstract

The application provides an optical remote sensing satellite image fusion method, computer equipment and a readable storage medium. The method comprises the following steps: taking a first to-be-fused image as input information of a predetermined resolution optimization model, outputting a first intermediate image through the resolution optimization model; determining a second intermediate image based on image features of each waveband of the first intermediate image and weights of each waveband; adjusting brightness and contrast of a second to-be-fused image to be consistent with brightness and contrast of the second intermediate image, obtaining a third intermediate image; determining a difference value image of the third intermediate image and the second intermediate image as a fourth intermediate image; for each waveband, determining a predetermined proportion feature of the image features of the waveband of the fourth intermediate image, and fusing the image features of the waveband of the first intermediate image and the predetermined proportion feature, obtaining a target fusion image obtained by fusing the fourth intermediate image and the first intermediate image. The scheme can improve the accuracy of the optical remote sensing satellite image fusion result.

Description

Technical Field

[0001] This application relates to the field of satellite remote sensing and image processing technology, and in particular to an optical remote sensing satellite image fusion method, computer equipment, and readable storage medium. Background Technology

[0002] Optical remote sensing satellites are important tools for Earth observation. However, due to the physical limitations of optical systems and complex practical operating conditions, it is difficult to simultaneously achieve both spatial and spectral resolution in satellite imaging. Specifically, to obtain more refined spectral information for accurate identification of ground features, multispectral or hyperspectral imaging can be used. These types of imaging have larger pixel sizes, resulting in blurred spatial details within the image. Panchromatic imaging, on the other hand, has smaller pixels and can acquire high-resolution textures, but it lacks fine spectral features.

[0003] Therefore, how to process images acquired by optical remote sensing satellites to achieve both detailed spectral features and high-resolution textures has become a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This application provides an optical remote sensing satellite image fusion method, computer device, and readable storage medium, aiming to solve the technical problem in related technologies that optical remote sensing satellite imaging is difficult to simultaneously take into account fine spectral features and high-resolution textures, resulting in image quality that cannot meet actual operational needs.

[0005] In a first aspect, embodiments of this application provide an optical remote sensing satellite image fusion method, including:

[0006] Acquire a first image to be fused and a second image to be fused, wherein the first image to be fused is a low-resolution multispectral image or a low-resolution hyperspectral image acquired by an optical remote sensing satellite, and the second image to be fused is a high-resolution panchromatic image acquired by the optical remote sensing satellite;

[0007] Using the first image to be fused as input information to a predetermined resolution optimization model, the resolution optimization model outputs a first intermediate image, wherein the resolution optimization model is used to perform spectral restoration processing on the first image to be fused, and the first intermediate image is an intermediate multispectral image or an intermediate hyperspectral image with the same resolution as the second image to be fused.

[0008] Based on the image features of the first intermediate image in each band and the weight of each band, a second intermediate image is determined, wherein the second intermediate image is used to reflect the brightness level of the first image to be fused after it has been adjusted to the resolution level of the second image to be fused.

[0009] The brightness and contrast of the second image to be fused are adjusted to match the brightness and contrast of the second intermediate image to obtain the third intermediate image;

[0010] The difference map between the third intermediate image and the second intermediate image is determined as the fourth intermediate image, wherein the fourth intermediate image is used to represent the high-frequency texture features of the second image to be fused.

[0011] For each band, a predetermined proportion feature of the image features of the fourth intermediate image in that band is determined, and the image features of the first intermediate image in that band are fused with the predetermined proportion feature to obtain a target fused image obtained by fusing the fourth intermediate image with the first intermediate image.

[0012] Optionally, in one embodiment of this application, before outputting a first intermediate image through the resolution optimization model using the first image to be fused as input information of a predetermined resolution optimization model, the method further includes:

[0013] Determine the data sources of the first image to be fused and the second image to be fused;

[0014] If the data source is a first designated source, proceed to the step of using the first image to be fused as input information for a predetermined resolution optimization model, and outputting a first intermediate image through the resolution optimization model;

[0015] If the data source is a second designated source, the first image to be fused and the second image to be fused are respectively subjected to radiometric calibration processing, geometric correction processing, atmospheric correction processing and geographic projection processing.

[0016] Optionally, in one embodiment of this application, before outputting a first intermediate image through the resolution optimization model using the first image to be fused as input information of a predetermined resolution optimization model, the method further includes:

[0017] Pixel registration processing based on imaging geometry is performed on the first image to be fused and the second image to be fused, so that the image alignment difference between the first image to be fused and the second image to be fused is less than a predetermined pixel value.

[0018] Optionally, in one embodiment of this application, before outputting a first intermediate image through the resolution optimization model using the first image to be fused as input information of a predetermined resolution optimization model, the method further includes:

[0019] According to a predetermined cutting size, the first image to be fused and the second image to be fused are cut to obtain a plurality of first sub-images after the first image to be fused is cut and a plurality of second sub-images after the second image to be fused is cut, wherein the area overlap of adjacent sub-images after cutting reaches a specified area threshold.

[0020] The step of using the first image to be fused as input information to a predetermined resolution optimization model, and outputting a first intermediate image through the resolution optimization model, includes:

[0021] For each of the multiple first sub-images, the first sub-image is used as the input information of a predetermined resolution optimization model, and the first intermediate image corresponding to the first sub-image is output through the resolution optimization model.

[0022] After obtaining the target fused image obtained by fusing the fourth intermediate image and the first intermediate image, the method further includes:

[0023] Based on the position information of each first sub-image in the first image to be fused, or based on the relative positional relationship of each first sub-image in the first image to be fused, the target fusion images corresponding to all the first sub-images are stitched together into a stitched fusion image.

[0024] In one embodiment of this application, optionally, determining the second intermediate image based on the image features of the first intermediate image in each band and the weight of each band includes:

[0025] Based on the weight of each band, the image features of the first intermediate image in each band are weighted and summed to obtain the second intermediate image, wherein...

[0026] For each band, the weights of the bands are determined in the following ways:

[0027] The weights of the multispectral bands are determined based on a predetermined spectral response function.

[0028] Alternatively, the weights of the multispectral bands can be determined based on the correlation between the panchromatic band and the multispectral band.

[0029] Optionally, in one embodiment of this application, adjusting the brightness and contrast of the second image to be fused to match the brightness and contrast of the second intermediate image includes:

[0030] The second image to be fused is converted into a third intermediate image with the same brightness and contrast as the second intermediate image by linear fitting.

[0031] ,

[0032] This refers to the third intermediate image. Let a represent the second image to be fused, and b be the fitting coefficients estimated by the least squares method.

[0033] Optionally, in one embodiment of this application, the resolution optimization model is a deep learning network, and the method for training the resolution optimization model includes:

[0034] Acquire a first historical image to be fused, a second historical image to be fused, and a historical target fused image, wherein the first historical image to be fused is a low-resolution multispectral image or a low-resolution hyperspectral image acquired by an optical remote sensing satellite in the historical fusion process, the second historical image to be fused is a high-resolution panchromatic image acquired by the optical remote sensing satellite in the historical fusion process, and the historical target fused image is the fusion result of the first historical image to be fused and the second historical image to be fused;

[0035] Using the first historical image to be fused as input sample, an initial resolution optimization model is input, and the resolution optimization model is iteratively trained until the resolution difference between the intermediate multispectral image or intermediate hyperspectral image output by the resolution optimization model and the second historical image to be fused meets the constraint range of the first loss function, wherein the first loss function is:

[0036] ,

[0037] Let the first loss function be... , and These represent the pixel-level mean square error, spectral angle mapping loss, and total variation loss in the current iteration, respectively. , and These represent the contributions of the pixel-level mean square error, the spectral angle mapping loss, and the total variation loss to remote sensing image fusion, respectively.

[0038] Optionally, in one embodiment of this application, the resolution optimization model is a deep learning network, and the method for training the resolution optimization model includes:

[0039] Acquire a first historical image to be fused, a second historical image to be fused, and a historical target fused image, wherein the first historical image to be fused is a low-resolution multispectral image or a low-resolution hyperspectral image acquired by an optical remote sensing satellite in the historical fusion process, the second historical image to be fused is a high-resolution panchromatic image acquired by the optical remote sensing satellite in the historical fusion process, and the historical target fused image is the fusion result of the first historical image to be fused and the second historical image to be fused;

[0040] Using the first historical image to be fused as input sample, an initial resolution optimization model is input. The spectral reconstruction module and detail injection module included in the resolution optimization model are jointly trained under the same optimization framework. During training, a joint loss function that simultaneously affects the intermediate spectral reconstruction results and the final fusion result is constructed to iteratively optimize the resolution optimization model until the spectral consistency of the intermediate multispectral / hyperspectral images output by the spectral reconstruction module and the quality of the final fused image jointly generated by the detail injection module and the spectral reconstruction module reach a predetermined optimization objective. The predetermined optimization objective is constrained by a second loss function, which is obtained by combining the first and third loss functions.

[0041] ,

[0042] This represents the second loss function. Let the first loss function be... For the third loss function, ∈[0, 1], used to reflect the importance of the first loss function in the second loss function. The larger the value, the more the second loss function focuses on the first optimization objective relative to the second optimization objective. The smaller the value, the more the second loss function emphasizes the second optimization objective relative to the first optimization objective, where the first optimization objective is the intermediate spectral reconstruction quality, and the second optimization objective is the final fused image quality;

[0043] ,

[0044] Let the first loss function be... , and These represent the pixel-level mean square error, spectral angle mapping loss, and total variation loss in the current iteration, respectively. , and These represent the contributions of the pixel-level mean square error, the spectral angle mapping loss, and the total variation loss to remote sensing image fusion, respectively.

[0045] ,

[0046] This represents the third loss function. This represents the predicted value for the first intermediate image in the current iteration. This represents the actual value of the first intermediate image. Indicates spectral uniformity loss, Represents structural similarity loss. This represents the injection regularization loss. This refers to the fourth intermediate image. This refers to the third intermediate image. This represents the second intermediate image. , , These represent the contribution levels of the spectral consistency loss, the structural similarity loss, and the injection regularization loss in the current iteration process, respectively.

[0047] In a second aspect, embodiments of this application provide a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in the first aspect above.

[0048] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions for performing the method described in the first aspect above.

[0049] The above technical solution addresses the technical problem in related technologies where optical remote sensing satellite imaging struggles to simultaneously capture detailed spectral features and high-resolution textures, resulting in image quality that fails to meet practical operational needs. It employs a resolution optimization model to spatially enhance low-resolution multispectral or hyperspectral images, making their spatial resolution consistent with high-resolution panchromatic images. Furthermore, it effectively extracts high-frequency texture features from the high-resolution panchromatic images and accurately injects these realistic high-frequency texture features into the enhanced spectral image. This approach balances detailed spectral features and high-resolution textures, avoiding the negative phenomena such as spectral distortion and texture illusions common in related technologies. While ensuring spectral physical consistency, it enhances the texture realism of the image and improves the accuracy of optical remote sensing satellite image fusion results. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 A flowchart of an optical remote sensing satellite image fusion method according to an embodiment of this application is shown;

[0052] Figure 2 A flowchart of a joint super-resolution system for remote sensing images according to an embodiment of this application is shown;

[0053] Figure 3 A flowchart of the input image preprocessing module according to one embodiment of this application is shown;

[0054] Figure 4 A flowchart illustrating the workflow of a deep learning spectral consistency super-resolution module according to an embodiment of this application is shown.

[0055] Figure 5 A flowchart of panchromatic multi-resolution analysis according to one embodiment of this application is shown;

[0056] Figure 6 A flowchart of the joint optimization module according to one embodiment of this application is shown;

[0057] Figure 7 A block diagram of a computer device according to one embodiment of this application is shown;

[0058] Figure 8 A block diagram of a computer device according to another embodiment of this application is shown. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Figure 1 A flowchart of an optical remote sensing satellite image fusion method according to an embodiment of this application is shown.

[0061] like Figure 1 As shown, the flow of an optical remote sensing satellite image fusion method according to an embodiment of this application includes:

[0062] Step 102: Obtain the first image to be fused and the second image to be fused.

[0063] The first and second images to be fused are ground feature images synchronously acquired by an optical remote sensing satellite at the same or near-term intervals for the same location. The first image to be fused is a low-resolution multispectral or high-resolution hyperspectral image acquired by the optical remote sensing satellite, containing rich spectral features that can be used for ground feature classification and quantitative inversion. The second image to be fused is a high-resolution panchromatic image acquired by the same optical remote sensing satellite, recording fine spatial textures to delineate the outlines and geometric details of ground features. In essence, the two images complement each other in terms of information display. Fusing them yields a high-quality image that balances fine spectral features and high-resolution textures, thus allowing for subsequent fusion steps.

[0064] Step 104: Using the first image to be fused as input information for a predetermined resolution optimization model, output a first intermediate image through the resolution optimization model.

[0065] The resolution optimization model is used to perform spectral restoration processing on the first image to be fused. In other words, the resolution optimization model improves the spatial resolution of the first image to be fused while maintaining spectral consistency, so as to restore its high-resolution spectral features. The first intermediate image is an intermediate multispectral image or intermediate hyperspectral image with the same resolution as the second image to be fused. More precisely, the first intermediate image is the spectral reconstruction result output by the resolution optimization model that is aligned with the resolution of the second image to be fused. Its high-frequency spatial details will be provided by the second image to be fused in subsequent steps.

[0066] In other words, the resolution optimization model is used to improve the resolution of the first low-resolution image to be fused, so that the resolution of the first image to be fused is consistent with that of the high-resolution second image to be fused, thus providing a data foundation for subsequent multi-resolution texture fusion.

[0067] Additionally, it should be noted that before step 104, the data sources of the first image to be fused and the second image to be fused can be determined; if the data source is a first designated source, the step of using the first image to be fused as input information for a predetermined resolution optimization model and outputting a first intermediate image through the resolution optimization model is initiated; if the data source is a second designated source, radiometric calibration, geometric correction, atmospheric correction, and geographic projection processing are performed on the first image to be fused and the second image to be fused, respectively.

[0068] Optionally, the data source refers to the processing stage that the images to be fused underwent after being acquired by the subject, or the product level of the images after being optimized by the subject. The first designated source refers to the level of remote sensing data products that have completed the radiometric calibration, geometric correction, atmospheric correction, and geographic projection processing. The second designated source refers to the level of data products that have not yet completed the above processing and only contain raw sensor signals or have only undergone preliminary radiometric correction. Determining whether to perform preprocessing such as radiometric calibration, geometric correction, atmospheric correction, and geographic projection processing on the images to be fused based on different data sources can ensure that the data to be fused itself performs well enough in terms of physical consistency, geometric accuracy, and spatial comparability, thereby providing a high-quality image data foundation for the subsequent fusion process.

[0069] In one possible design, before or after determining whether to perform preprocessing such as radiometric calibration, geometric correction, atmospheric correction, and geographic projection on the images to be fused based on different data sources, pixel registration processing based on imaging geometry can be performed on the first image to be fused and the second image to be fused, so that the image alignment difference between the first image to be fused and the second image to be fused is less than a predetermined pixel value.

[0070] The pixel registration process is achieved by calculating the spatial transformation model between the first image to be fused and the second image to be fused and resampling one of the images. This eliminates geometric misalignment caused by differences in sensor viewpoint, track or platform posture, and ensures that the image data in subsequent fusion operations achieves pixel-level spatial alignment, thereby avoiding abnormalities such as ghosting, blurring or texture misalignment in the fusion results.

[0071] Step 106: Determine the second intermediate image based on the image features of the first intermediate image in each band and the weight of each band.

[0072] The first intermediate image contains image features in each band, which are the pixel value matrix or feature map corresponding to that band after spatial resolution enhancement. The weight of each band represents the contribution of the spectral response characteristics of that band to the construction of an overall brightness image representing the overall brightness. Combining the two, the contribution quantization of that band to the final synthesized overall brightness image can be obtained. Then, the contribution quantizations of all bands to the final synthesized overall brightness image are superimposed to obtain a second intermediate image. This second intermediate image is used to reflect the brightness level of the first image to be fused after it has been adjusted to the resolution level of the second image to be fused.

[0073] In one possible design, based on the weight of each band, the image features of the first intermediate image in each band are weighted and summed to obtain the second intermediate image. The weighted value of the image features of the first intermediate image in each band is the quantized contribution of that band to the final synthesized overall brightness image. The sum obtained by weighted summation is the superposition of the quantized contributions of all bands to the final synthesized overall brightness image.

[0074] Therefore, the second intermediate image can effectively reflect the brightness level of the multispectral or hyperspectral image after resolution enhancement, providing a valid basis for the subsequent extraction of high-frequency texture features unique to the panchromatic image that are independent of brightness level.

[0075] In one possible design, the weight of each band is determined by: determining the weight of the multispectral band based on a predetermined spectral response function; or determining the weight of the multispectral band based on the correlation between the panchromatic band and the multispectral band. Here, the spectral response function is an inherent characteristic of the sensor, reflecting the spectral sensitivity distribution of the multispectral bands and the degree of overlap between the spectral ranges of the multispectral bands and the panchromatic band. The correlation between the panchromatic band and the multispectral band reflects the similarity of the brightness changes of ground features captured by both during imaging. Therefore, the determined band weights comprehensively consider the physical characteristics of the sensor and the actual features of the ground features, ensuring that the synthesized overall brightness image accurately reflects the brightness distribution of the actual ground features, thus providing a data foundation for subsequent extraction of high-frequency texture features.

[0076] Step 108: Adjust the brightness and contrast of the second image to be fused to match the brightness and contrast of the second intermediate image to obtain the third intermediate image.

[0077] Brightness and contrast reflect the global grayscale level of the second image to be fused and the difference span between the bright and dark areas in the second image to be fused, respectively. The brightness and contrast of the second image to be fused are adjusted to be consistent with the brightness and contrast of the second intermediate image, so that the overall brightness performance characteristics of the second image to be fused are consistent with the brightness performance characteristics of the second intermediate image, and a third intermediate image is obtained. In this way, the information extracted from the difference between the third intermediate image and the second intermediate image no longer includes the features caused by the difference in brightness level, but mainly includes high-frequency texture features caused by the difference in spatial resolution that are unrelated to the overall brightness level of the spectrum.

[0078] This allows us to separate the unique, spectral-independent fine spatial structure in high-resolution panchromatic images, providing a reliable basis for the subsequent steps of accurately injecting real high-resolution textures, i.e., high-frequency texture features, into multispectral or hyperspectral images.

[0079] In one possible design, the second image to be fused can be converted into a third intermediate image with the same brightness and contrast as the second intermediate image through linear fitting, wherein,

[0080] ,

[0081] This refers to the third intermediate image. Let a represent the second image to be fused, and b be the fitting coefficients estimated using the least squares method. Thus, the linear fitting method can be efficiently eliminated between the two images by a simple mathematical transformation using the least squares criterion, thus eliminating brightness and contrast deviations.

[0082] Step 110: Determine the difference map between the third intermediate image and the second intermediate image, and use it as the fourth intermediate image.

[0083] The difference between the third intermediate image and the second intermediate image is mainly due to the difference in spatial resolution, resulting in high-frequency texture features that are independent of the overall spectral brightness level. Therefore, the difference map between the two is used to represent the high-frequency texture features of the second image to be fused. In actual computation, the difference map not only contains high-frequency texture features but also often contains some noise or residuals. To address this, the difference map can be filtered or regularized to enhance its representation of the true spatial features.

[0084] Step 112: For each band, determine the predetermined proportion feature of the image features of the fourth intermediate image in the band, and fuse the image features of the first intermediate image in the band with the predetermined proportion feature to obtain the target fused image obtained by fusing the fourth intermediate image and the first intermediate image.

[0085] Different light bands have different predetermined proportions. The visible light band usually injects more details, while the near-infrared or short-wave infrared bands, due to their significant spectral differences from the panchromatic band, typically inject only a small amount of detail to avoid spectral distortion. Therefore, the predetermined proportion of image features of the fourth intermediate image in a single band—that is, the high-frequency texture features of the second image to be fused in that band—is fused with the image features of the first intermediate image in that band—that is, the spectral features of a high-resolution multispectral or hyperspectral image in that band—to obtain the target fused image.

[0086] It should be added that, before step 104, the first image to be fused and the second image to be fused can be cut according to a predetermined cutting size to obtain multiple first sub-images of the first image to be fused and multiple second sub-images of the second image to be fused. Since the first image to be fused and the second image to be fused are often large in size, the processing requires a lot of computation and consumes a lot of system resources. Therefore, the first image to be fused and the second image to be fused can each be cut into multiple sub-images. In this way, the large-scale overall computing task can be decomposed into multiple parallel sub-tasks, effectively utilizing parallel computing resources to improve the processing efficiency of image fusion.

[0087] In this process, the overlapping area of ​​adjacent sub-images after cutting reaches a specified area threshold. This can eliminate the large and discontinuous seams of sub-images caused by the boundary effect of splicing in the final splicing operation, and also avoid the situation where the sub-image connection is difficult to fit.

[0088] Simultaneously, in subsequent fusion processing steps, the operations on the first image to be fused and the second image to be fused are converted into operations on the sub-images obtained from their segmentation. After the sub-image fusion is completed, the target fused image of the fused sub-images is then stitched together. Optionally, step 104 involves using each of the multiple first sub-images as input information for a predetermined resolution optimization model, and outputting a first intermediate image corresponding to the first sub-image through the resolution optimization model. After step 112, the method further includes: stitching together the target fused images corresponding to all the first sub-images into a stitched fused image based on the position information of each first sub-image in the first image to be fused, or based on the relative positional relationship of each first sub-image in the first image to be fused.

[0089] In summary, the technical solution of this application enhances the spatial resolution of low-resolution multispectral or hyperspectral images through a resolution optimization model, making its spatial resolution consistent with that of high-resolution panchromatic images. It then effectively extracts the high-frequency texture features from the high-resolution panchromatic images and accurately injects these realistic high-frequency texture features into the enhanced spectral image. This approach balances fine spectral features and high-resolution texture, avoiding the negative phenomena such as spectral distortion and texture illusions that are common in related technologies. While ensuring spectral physical consistency, it enhances the texture realism of the image and improves the accuracy of optical remote sensing satellite image fusion results.

[0090] exist Figure 1Based on the embodiments, the resolution optimization model is implemented through a deep learning-based super-resolution method, or in other words, the resolution optimization model is implemented through a deep learning super-resolution network with spectral restoration as the core objective. The training of the resolution optimization model includes: acquiring a first historical image to be fused, a second historical image to be fused, and a historical target fusion image. The first historical image to be fused is a low-resolution multispectral image or a low-resolution hyperspectral image acquired by an optical remote sensing satellite during historical fusion processing; the second historical image to be fused is a high-resolution panchromatic image acquired by the same optical remote sensing satellite during historical fusion processing; and the historical target fusion image is the fusion result of the first historical image to be fused and the second historical image to be fused. Next, using the first historical image to be fused as an input sample, an initial resolution optimization model is input, and the resolution optimization model is iteratively trained until the resolution difference between the intermediate multispectral image or intermediate hyperspectral image output by the resolution optimization model and the second historical image to be fused meets the constraint range of a first loss function, wherein the first loss function is:

[0091] ,

[0092] Let the first loss function be... , and These represent the pixel-level mean square error, spectral angle mapping loss, and total variation loss in the current iteration, respectively. , and These represent the contributions of the pixel-level mean square error, the spectral angle mapping loss, and the total variation loss to remote sensing image fusion, respectively.

[0093] Specifically It is a pixel-level mean square error used to limit pixel value shift and maintain overall spectral intensity; This is the spectral angle mapping loss, used to maintain spectral shape consistency; This is the total variation loss, used to suppress over-sharpening and noise, and maintain spatial smoothness. This is achieved by appropriately setting the weights. , , The model can learn the spectral reconstruction process from low resolution to high resolution without over-generating fake textures, so that real details can be injected later in panchromatic mode.

[0094] By employing the above loss functions, precise and balanced constraints can be applied to pixel intensity, spectral morphology, and spatial smoothness during training. This ensures that the mapping learned by the model not only conforms to the spectral distribution of the original data but also possesses good visual fidelity and reliable noise resistance. Furthermore, these loss functions prevent the model from getting trapped in local optima due to a single optimization objective, such as over-sharpening or spectral distortion, thus improving the output stability of the deep learning model.

[0095] In another possible design, the resolution optimization model is trained by: acquiring a first historical image to be fused, a second historical image to be fused, and a historical target fusion image. The first historical image to be fused is a low-resolution multispectral or low-resolution hyperspectral image acquired by an optical remote sensing satellite during historical fusion processing; the second historical image to be fused is a high-resolution panchromatic image acquired by the same optical remote sensing satellite during historical fusion processing; and the historical target fusion image is the fusion result of the first and second historical images to be fused. Using the first historical image to be fused as an input sample, an initial resolution optimization model is input. The spectral reconstruction module and detail injection module included in the resolution optimization model are jointly trained within the same optimization framework. During training, a joint loss function is constructed that simultaneously affects the intermediate spectral reconstruction results and the final fusion result. The resolution optimization model is iteratively optimized until the spectral consistency of the intermediate multispectral / hyperspectral images output by the spectral reconstruction module and the quality of the final fusion image jointly generated by the detail injection module and the spectral reconstruction module reach a predetermined optimization objective. This predetermined optimization objective is constrained by a second loss function, which is obtained by combining a first and a third loss function.

[0096] ,

[0097] This represents the second loss function. Let the first loss function be... For the third loss function, ∈[0, 1], used to reflect the importance of the first loss function in the second loss function. The larger the value, the more the second loss function focuses on the first optimization objective relative to the second optimization objective. The smaller the value, the more the second loss function emphasizes the second optimization objective relative to the first optimization objective, where the first optimization objective is the intermediate spectral reconstruction quality, and the second optimization objective is the final fused image quality;

[0098] ,

[0099] Let the first loss function be... , and These represent the pixel-level mean square error, spectral angle mapping loss, and total variation loss in the current iteration, respectively. , and These represent the contributions of the pixel-level mean square error, the spectral angle mapping loss, and the total variation loss to remote sensing image fusion, respectively.

[0100] ,

[0101] This represents the third loss function. This represents the predicted value for the first intermediate image in the current iteration. This represents the actual value of the first intermediate image. Indicates spectral uniformity loss, Represents structural similarity loss. This represents the injection regularization loss. This refers to the fourth intermediate image. This refers to the third intermediate image. This represents the second intermediate image. , , These represent the contribution levels of the spectral consistency loss, the structural similarity loss, and the injection regularization loss in the current iteration process, respectively.

[0102] By employing the aforementioned joint loss function, the spectral reconstruction module and the detail injection module can be jointly trained within the same optimization framework. A joint loss function is constructed that simultaneously affects both intermediate and final results, merging the integrity constraints of spectral reconstruction with the reliability requirements of high-frequency texture injection. Through an adjustable weight γ, the spectral fidelity of the intermediate process and the overall quality of the final output are dynamically balanced during training. Therefore, this joint optimization framework not only avoids the error accumulation problem of traditional segmented fusion methods but also adaptively adjusts the optimization focus according to specific task requirements. This effectively prevents the model from falling into local optima due to over-sharpening or spectral distortion in pursuit of a single objective, thereby fundamentally improving the output quality of this deep learning model.

[0103] Figure 2 A flowchart of a joint super-resolution system for remote sensing images according to an embodiment of this application is shown.

[0104] like Figure 2As shown, in one embodiment of the remote sensing image joint super-resolution system of this application, there are an input image preprocessing module, a deep learning spectral consistency super-resolution module, a PAN (panchromatic image) multi-resolution detail analysis and injection module, and a joint optimization module connected in sequence. The overall workflow is executed in the order of the modules as follows.

[0105] First, the operations performed by the input image preprocessing module include: 1. Reading panchromatic and multispectral / hyperspectral inputs; 2. Radiometric calibration / atmospheric correction / geometric correction; 3. Image cropping, normalization, and reprojection; and finally, 4. Outputting the preprocessed image. (Low-resolution multispectral image) and PAN.

[0106] Next, the preprocessed data enters the deep learning spectral consistency super-resolution module, which performs the following operations: 1. multi-band joint input, 2. encoder extracting spectral-spatial features, and 3. decoder generating high resolution. 4. Loss Calculation 5. Output (High-resolution multispectral image).

[0107] Then, the output The preprocessed PAN is input together with the PAN multi-resolution detail analysis and injection module, which is specifically used for: 1. Luminance map Construction, 2.PAN and Scale matching, 3. Detail image D=PAN'- 4. Inject according to the band ratio α; 5. Output the fused product. (High-resolution multispectral images with high-frequency texture features).

[0108] Finally, the system enters the joint optimization module, which specifically includes: 1. Constructing the final loss. 2. Backpropagate to the deep learning module; 3. Optimize. 4. Optimal output of injected parameters. As the final product.

[0109] At this point, the system has completed the entire processing flow from the original input to the high-quality fused image.

[0110] Figure 3 A flowchart of the input image preprocessing module according to an embodiment of this application is shown.

[0111] like Figure 3 As shown, in the input image preprocessing module, its module input is... The entire preprocessing process, including PAN, sequentially includes the following steps.

[0112] First, perform data reading, which includes: 1. reading sensor L0 / L1 / L2 products, 2. obtaining metadata (rational polynomial coefficients RPC / projection information PRJ / gain).

[0113] Next, radiation and atmospheric calibration are performed, specifically including: 1. Loading the gain / bias file, 2. Reflectivity transformation (using the FLAASH / 6S model), 3. Outputting the surface reflectivity MS. ref .

[0114] Then, geometric correction and projection unification are performed, specifically including: 1. Geometric correction based on RPC, 2. Projection to a unified coordinate system (Universal Transverse Mercator Projection UTM / World Geodetic System WGS84), 3. Outputting the aligned MS coordinates. geo (Image of the alignment result).

[0115] Finally, image cropping and normalization are performed, specifically including: 1. cropping to a specified size block, 2. overlapping of adjacent sides of adjacent size blocks by a specified ratio, and 3. normalization to [0, 1].

[0116] After the above process, the final output of this module is: MS pre (Preprocessed multispectral data) and PAN pre (Preprocessed panchromatic image).

[0117] Figure 4 A flowchart illustrating the workflow of a deep learning spectral consistency super-resolution module according to an embodiment of this application is shown.

[0118] like Figure 4 As shown, in the deep learning spectral consistency super-resolution module, its input is: MS pre The entire processing flow includes the following steps in sequence.

[0119] First, perform multi-band joint input, which includes: 1. Input tensor dimension: H×W×B, 2. Position encoding, using Transformer.

[0120] Next, the data enters the encoder to extract spectral and spatial low-frequency features, specifically including: 1. determining the convolutional block / residual block, 2. performing multi-scale downsampling ( ), 3. Output Features .

[0121] Then, the core processing is performed through the backbone network (spectral reconstruction), specifically including: 1. constructing a multi-layer residual structure, 2. calling the spectral attention module, 3. preserving spectral shape constraints, and 4. outputting the results. .

[0122] Then, by upsampling The decoder reconstructs the features, specifically including: 1. transposed convolution, 2. Skip connections to ensure spatial consistency, and 3. outputting a resolution consistent with the PAN. .

[0123] Finally, the spectral consistency loss is calculated, and the specific formula is as follows: The final output of this module is a high-resolution multispectral image with spectral consistency. .

[0124] Figure 5 A flowchart of panchromatic multi-resolution analysis according to an embodiment of this application is shown.

[0125] like Figure 5 As shown, the module input for the PAN multi-resolution detail analysis and injection module is: With PAN pre The entire processing flow includes the following steps in sequence:

[0126] First, a brightness map is constructed, which includes the following formula:

[0127] ,

[0128] This represents the second intermediate image, where B is the number of bands. The weight of the i-th band is... The image features of the first intermediate image in the i-th band are... Derived from the spectral response function.

[0129] Next, PAN brightness scale matching is performed, specifically including the following formula:

[0130] ,

[0131] a and b are calculated using least squares fitting to ensure that PAN is consistent with... The purpose is to achieve low-frequency consistency.

[0132] Then, perform detailed image extraction, specifically using the formula: D=PAN'- This indicates that D contains PAN's unique high-frequency texture and does not contain global brightness deviation.

[0133] Next, multi-band injection is performed, specifically including the following formula:

[0134] ，

[0135] That is, the i-th band corresponds to , That is, the i-th band corresponds to , For the image features of the i-th band, the predetermined scale features of visible light Higher, near-infrared light The values ​​are relatively low, and these coefficients are obtained through model learning. .

[0136] Figure 6 A flowchart of the joint optimization module according to one embodiment of this application is shown.

[0137] like Figure 6 As shown, the input to the joint optimization module is: and (True high-resolution multispectral data, available during the training phase) The entire joint optimization process includes the following steps in sequence.

[0138] First, the final loss function is constructed, which includes: ,in This represents spectral error, including SAM (spectral angle plotting) and MSE (mean square error). To ensure structural and textural consistency, measures include SSIM (Structural Similarity Index) and ERGAS (Relative Global Dimensionless Error). To preserve loss at the edge, Used to limit over-injection of textures.

[0139] Next, backpropagation is performed to the deep learning super-resolution network to update the convolutional, attention, and Transformer parameters. Then, backpropagation is performed to the injection ratio. Automatically learn the optimal injection strength After that, iterative training proceeds to the convergence phase.

[0140] Ultimately, the module outputs the optimal model parameters, namely the network parameters and injection parameters. .

[0141] In any of the above embodiments, the SAM (Spectral Angle Error Index) of the output high-resolution multispectral image is reduced compared to super-resolution methods based solely on deep learning, while maintaining geometric details consistent with panchromatic images in high-frequency regions such as building edges and roads. The inference time of the entire super-resolution reconstruction process can be controlled within a short time range on GPUs (Graphics Processing Units) or spaceborne computing platforms, making it suitable for on-board processing or edge computing scenarios with limited computing resources.

[0142] In summary, this invention addresses multispectral or hyperspectral and panchromatic joint imaging scenarios, establishing a super-resolution reconstruction method that combines spectral consistency, spatial texture realism, and computational controllability. This method integrates the advantages of deep learning models in spectral reconstruction with the inherent advantages of panchromatic bands in texture sampling, forming a holistic technical path of "deep learning spectral reconstruction—multi-resolution texture injection—joint loss optimization." By uniformly optimizing the parameters of the two modules, this invention avoids the error accumulation problem caused by the independent spectral restoration and texture enhancement stages in traditional methods, resulting in a final high-resolution multispectral image that achieves an optimal balance in spectral consistency, texture accuracy, and geometric stability.

[0143] In another embodiment, this application provides a computer device, which may be a server, and its internal structure diagram may be as follows. Figure 7 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it can implement the methods described in any of the above embodiments.

[0144] In one embodiment, this application also provides a computer device, which can be a client, and its internal structure diagram can be as follows: Figure 8 As shown, the computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it can implement the methods described in any of the above embodiments.

[0145] The computer devices described in the embodiments of this application exist in various forms, and are particularly suitable for edge computing and on-board computing scenarios with limited computing resources and high requirements for real-time performance and autonomy, including but not limited to:

[0146] (1) Onboard computing platform: These devices are deployed on spacecraft such as optical remote sensing satellites. They have strict constraints on power consumption, size and reliability, and need to perform image fusion processing autonomously in orbit to realize the real-time or near-real-time downlink and application of observation data. For example, dedicated processing units or high-performance onboard computers integrated into satellite platforms.

[0147] (2) Edge computing nodes and terminals: These devices are deployed at the network edge or front end, possessing certain local computing capabilities. They can perform fusion processing near the data generation point to reduce data transmission latency and bandwidth dependence. These terminals include, but are not limited to: UAV-borne processors, vehicle-mounted computing units, ship information processing systems, portable field information terminals, and IoT gateways and edge servers.

[0148] (3) Mobile and portable smart devices: These devices can perform lightweight fusion tasks through built-in or external computing power, supporting rapid on-site analysis and decision-making. Such devices include, but are not limited to: ruggedized smartphones, tablets, dedicated handheld inspection terminals, and wearable devices and portable navigation and monitoring devices with integrated processing modules.

[0149] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0150] (5) Other electronic devices with data interaction functions.

[0151] Additionally, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which are used to perform the following steps:

[0152] Acquire a first image to be fused and a second image to be fused, wherein the first image to be fused is a low-resolution multispectral image or a low-resolution hyperspectral image acquired by an optical remote sensing satellite, and the second image to be fused is a high-resolution panchromatic image acquired by the optical remote sensing satellite;

[0153] Using the first image to be fused as input information to a predetermined resolution optimization model, the resolution optimization model outputs a first intermediate image, wherein the resolution optimization model is used to perform spectral restoration processing on the first image to be fused, and the first intermediate image is an intermediate multispectral image or an intermediate hyperspectral image with the same resolution as the second image to be fused.

[0154] Based on the image features of the first intermediate image in each band and the weight of each band, a second intermediate image is determined, wherein the second intermediate image is used to reflect the brightness level of the first image to be fused after it has been adjusted to the resolution level of the second image to be fused.

[0155] The brightness and contrast of the second image to be fused are adjusted to match the brightness and contrast of the second intermediate image to obtain the third intermediate image;

[0156] The difference map between the third intermediate image and the second intermediate image is determined as the fourth intermediate image, wherein the fourth intermediate image is used to represent the high-frequency texture features of the second image to be fused.

[0157] For each band, a predetermined proportion feature of the image features of the fourth intermediate image in that band is determined, and the image features of the first intermediate image in that band are fused with the predetermined proportion feature to obtain a target fused image obtained by fusing the fourth intermediate image with the first intermediate image.

[0158] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0159] The technical solution of this application has been described in detail above with reference to the accompanying drawings. This technical solution uses a resolution optimization model to spatially enhance the spatial resolution of low-resolution multispectral or hyperspectral images, making their spatial resolution consistent with that of high-resolution panchromatic images. Then, it effectively extracts the high-frequency texture features of the high-resolution panchromatic images and accurately injects these realistic high-frequency texture features into the enhanced spectral image. Therefore, it can balance fine spectral features and high-resolution texture, avoiding the problems of spectral distortion and texture illusion that are easily generated in related technologies. While ensuring spectral physical consistency, it enhances the texture realism of the image and improves the accuracy of the optical remote sensing satellite image fusion results.

[0160] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0161] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0163] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0164] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0165] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for fusion of optical remote sensing satellite images, characterized in that, include: Acquire a first image to be fused and a second image to be fused, wherein the first image to be fused is a low-resolution multispectral image or a low-resolution hyperspectral image acquired by an optical remote sensing satellite, and the second image to be fused is a high-resolution panchromatic image acquired by the optical remote sensing satellite; Using the first image to be fused as input information to a predetermined resolution optimization model, the resolution optimization model outputs a first intermediate image, wherein the resolution optimization model is used to perform spectral restoration processing on the first image to be fused, and the first intermediate image is an intermediate multispectral image or an intermediate hyperspectral image with the same resolution as the second image to be fused. Based on the image features of the first intermediate image in each band and the weight of each band, a second intermediate image is determined, wherein the second intermediate image is used to reflect the brightness level of the first image to be fused after it has been adjusted to the resolution level of the second image to be fused. The brightness and contrast of the second image to be fused are adjusted to match the brightness and contrast of the second intermediate image to obtain the third intermediate image; The difference map between the third intermediate image and the second intermediate image is determined as the fourth intermediate image, wherein the fourth intermediate image is used to represent the high-frequency texture features of the second image to be fused. For each band, a predetermined proportional feature of the image features of the fourth intermediate image in the band is determined, and the image features of the first intermediate image in the band are fused with the predetermined proportional feature to obtain a target fused image obtained by fusing the fourth intermediate image with the first intermediate image. The resolution optimization model is a deep learning network, and the methods for training the resolution optimization model include: Acquire a first historical image to be fused, a second historical image to be fused, and a historical target fused image, wherein the first historical image to be fused is a low-resolution multispectral image or a low-resolution hyperspectral image acquired by an optical remote sensing satellite in the historical fusion process, the second historical image to be fused is a high-resolution panchromatic image acquired by the optical remote sensing satellite in the historical fusion process, and the historical target fused image is the fusion result of the first historical image to be fused and the second historical image to be fused; Using the first historical image to be fused as input sample, an initial resolution optimization model is input. The spectral reconstruction module and detail injection module included in the resolution optimization model are jointly trained under the same optimization framework. During training, a joint loss function that simultaneously affects the intermediate spectral reconstruction results and the final fusion result is constructed to iteratively optimize the resolution optimization model until the spectral consistency of the intermediate multispectral / hyperspectral images output by the spectral reconstruction module and the quality of the final fused image jointly generated by the detail injection module and the spectral reconstruction module reach a predetermined optimization objective. The predetermined optimization objective is constrained by a second loss function, which is obtained by combining the first and third loss functions. ， This represents the second loss function. Let the first loss function be... For the third loss function, ∈[0, 1], used to reflect the importance of the first loss function in the second loss function. The larger the value, the more the second loss function focuses on the first optimization objective relative to the second optimization objective. The smaller the value, the more the second loss function emphasizes the second optimization objective relative to the first optimization objective, where the first optimization objective is the intermediate spectral reconstruction quality, and the second optimization objective is the final fused image quality; ， Let the first loss function be... , and These represent the pixel-level mean square error, spectral angle mapping loss, and total variation loss in the current iteration, respectively. , and These represent the contributions of the pixel-level mean square error, the spectral angle mapping loss, and the total variation loss to remote sensing image fusion, respectively. ， This represents the third loss function. This represents the predicted value for the first intermediate image in the current iteration. This represents the actual value of the first intermediate image. Indicates spectral uniformity loss, Represents structural similarity loss. This represents the injection regularization loss. This refers to the fourth intermediate image. This refers to the third intermediate image. This represents the second intermediate image. , , These represent the contribution levels of the spectral consistency loss, the structural similarity loss, and the injection regularization loss in the current iteration process, respectively.

2. The method according to claim 1, characterized in that, Before using the first image to be fused as input information for a predetermined resolution optimization model and outputting a first intermediate image through the resolution optimization model, the method further includes: Determine the data sources of the first image to be fused and the second image to be fused; If the data source is a first designated source, proceed to the step of using the first image to be fused as input information for a predetermined resolution optimization model, and outputting a first intermediate image through the resolution optimization model; If the data source is a second designated source, the first image to be fused and the second image to be fused are respectively subjected to radiometric calibration processing, geometric correction processing, atmospheric correction processing and geographic projection processing.

3. The method according to claim 2, characterized in that, Before using the first image to be fused as input information for a predetermined resolution optimization model and outputting a first intermediate image through the resolution optimization model, the method further includes: Pixel registration processing based on imaging geometry is performed on the first image to be fused and the second image to be fused, so that the image alignment difference between the first image to be fused and the second image to be fused is less than a predetermined pixel value.

4. The method according to claim 3, characterized in that, Before using the first image to be fused as input information for a predetermined resolution optimization model and outputting a first intermediate image through the resolution optimization model, the method further includes: According to a predetermined cutting size, the first image to be fused and the second image to be fused are cut to obtain a plurality of first sub-images after the first image to be fused is cut and a plurality of second sub-images after the second image to be fused is cut, wherein the area overlap of adjacent sub-images after cutting reaches a specified area threshold. The step of using the first image to be fused as input information to a predetermined resolution optimization model, and outputting a first intermediate image through the resolution optimization model, includes: For each of the multiple first sub-images, the first sub-image is used as the input information of a predetermined resolution optimization model, and the first intermediate image corresponding to the first sub-image is output through the resolution optimization model. After obtaining the target fused image obtained by fusing the fourth intermediate image and the first intermediate image, the method further includes: Based on the position information of each first sub-image in the first image to be fused, or based on the relative positional relationship of each first sub-image in the first image to be fused, the target fusion images corresponding to all the first sub-images are stitched together into a stitched fusion image.

5. The method according to any one of claims 1 to 4, characterized in that, Determining the second intermediate image based on the image features of the first intermediate image in each band and the weight of each band includes: Based on the weight of each band, the image features of the first intermediate image in each band are weighted and summed to obtain the second intermediate image, wherein... For each band, the weights of the bands are determined in the following ways: The weights of the multispectral bands are determined based on a predetermined spectral response function. Alternatively, the weights of the multispectral bands can be determined based on the correlation between the panchromatic band and the multispectral band.

6. The method according to claim 5, characterized in that, Adjusting the brightness and contrast of the second image to be fused to match the brightness and contrast of the second intermediate image includes: The second image to be fused is converted into a third intermediate image with the same brightness and contrast as the second intermediate image by linear fitting. ， This refers to the third intermediate image. Let a represent the second image to be fused, and b be the fitting coefficients estimated by the least squares method.

7. The method according to claim 4, characterized in that, The resolution optimization model is a deep learning network, and the methods for training the resolution optimization model include: Acquire a first historical image to be fused, a second historical image to be fused, and a historical target fused image, wherein the first historical image to be fused is a low-resolution multispectral image or a low-resolution hyperspectral image acquired by an optical remote sensing satellite in the historical fusion process, the second historical image to be fused is a high-resolution panchromatic image acquired by the optical remote sensing satellite in the historical fusion process, and the historical target fused image is the fusion result of the first historical image to be fused and the second historical image to be fused; Using the first historical image to be fused as input sample, an initial resolution optimization model is input, and the resolution optimization model is iteratively trained until the resolution difference between the intermediate multispectral image or intermediate hyperspectral image output by the resolution optimization model and the second historical image to be fused meets the constraint range of the first loss function, wherein the first loss function is: ， Let the first loss function be... , and These represent the pixel-level mean square error, spectral angle mapping loss, and total variation loss in the current iteration, respectively. , and These represent the contributions of the pixel-level mean square error, the spectral angle mapping loss, and the total variation loss to remote sensing image fusion, respectively.

8. A computer device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions being configured to cause the processor to perform the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The device stores computer-executable instructions configured to perform the method as described in any one of claims 1 to 7.

Images