A SFIM fusion method and apparatus based on scale adaptive transformation

By adaptively constructing convolution kernels and iteratively degrading the image, the scale-adaptive transformation method for panchromatic and multispectral images solves the image fusion problem of the SFIM algorithm under different scale differences, and achieves high-quality image fusion results.

CN117635450BActive Publication Date: 2026-11-17BEIJING INST OF REMOTE SENSING INFORMATION
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311271292.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-09-28
Publication Date
2026-11-17
Estimated Expiration
2043-09-28

AI Technical Summary

Technical Problem

The existing SFIM algorithm cannot effectively filter out high-frequency information when processing panchromatic multispectral images with different scales, resulting in color cast and blurring distortion in the image fusion results, and it cannot adapt to the multispectral scale differences of new satellites.

Method used

A scale-adaptive transformation-based method is adopted. By calculating the scale ratio of panchromatic and multispectral images, convolution kernels of different scales are adaptively constructed. The panchromatic image is iteratively degraded and then fused with the multispectral image upsampled to the panchromatic scale to ensure consistent sharpness.

Benefits of technology

The fusion effect of the SFIM algorithm has been improved, enabling it to adaptively process panchromatic multispectral images with different scales, maintain the clarity of multispectral images, and improve the quality of image fusion.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117635450B_ABST
    Figure CN117635450B_ABST
Patent Text Reader

Abstract

This invention discloses a scale-adaptive SFIM fusion method and apparatus. The method includes: acquiring panchromatic and multispectral images; processing the panchromatic and multispectral images to obtain their scale ratios; adaptively constructing convolution kernels of different scales according to the scale ratios; iteratively degrading the panchromatic image using the convolution kernels of different scales to obtain a degraded panchromatic image; upsampling the degraded panchromatic image and the multispectral image to the panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image; and fusing the upsampled panchromatic image and the upsampled multispectral image to obtain a fused image. This invention effectively improves the SFIM fusion effect and is unaffected by differences in panchromatic and multispectral scales, enabling adaptive fusion of panchromatic and multispectral images with varying scales.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a scale-adaptive transformation-based SFIM fusion method and apparatus. Background Technology

[0002] Due to hardware limitations, optical remote sensing cameras cannot acquire remote sensing images with both high spatial and high spectral resolution simultaneously. Current mainstream satellite designs employ spatial-spectral separation, using panchromatic (Pan) images to primarily store spatial information and multispectral (Mss) images to primarily store spectral information. This allows for the acquisition of as much information as possible within the maximum limits of the hardware design specifications. This limitation also contributes to the spatial-spectral separation characteristic of optical remote sensing images.

[0003] However, in ground applications, there is no limit to the pursuit of high resolution in remote sensing images, including both spatial and spectral resolution. How to acquire images with both high spatial and spectral resolution under hardware constraints has always been a research hotspot. To address this problem, a large number of excellent fusion algorithms have been developed, taking into account the spatial-spectral separation characteristics of remote sensing images. Currently, image fusion methods can be broadly classified into two categories: multi-resolution analysis and component transformation methods. Multi-resolution analysis methods utilize multi-scale decomposition to sharpen multispectral images using panchromatic images. Typical methods include Brovey, SFIM (smoothing filter-based intensity modulation), NBU_PanSharp, and Wavelets. Component transformation methods generally consist of three steps: 1) converting the multispectral image to another space, 2) replacing the most information-rich component with the panchromatic image, and 3) performing an inverse transformation to obtain the fused image. Typical component transformation methods include HIS, GS, and PCA.

[0004] Among these methods, the SFIM algorithm has always been favored due to its high computational efficiency and good fusion effect. However, the SFIM algorithm is relatively old and was mainly developed for images with a panchromatic multispectral scale difference of 4 times. Although it has good spectral information preservation capabilities when applied to the latest satellites, its spatial information integration is poor. The main reason is that the panchromatic multispectral scale differences of newly launched remote sensing satellites are more abundant. In addition to the common 4-fold difference, there are also 2.7-fold differences (Ziyuan-3), 8-fold differences (Gaofen-6), etc. When using the SFIM method for fusion, the high-frequency information of remote sensing images at different scales is not completely filtered out by a single convolution kernel. This leads to poor image fusion results, causing color casts, blurring, and distortion in the fused image. Summary of the Invention

[0005] The technical problem this invention aims to solve is to provide a scale-adaptive SFIM fusion method and apparatus that adapts to the fusion of satellite images with different panchromatic multispectral scale differences. This method can adaptively process panchromatic multispectral images with different scale differences, iteratively filtering out spatial information from the panchromatic images to ensure consistent sharpness with the multispectral images during ratio transformation. This method effectively improves the SFIM fusion effect and is unaffected by panchromatic multispectral scale differences, enabling adaptive fusion of panchromatic multispectral images with different scale differences.

[0006] To address the aforementioned technical problems, a first aspect of this invention discloses an SFIM fusion method based on scale-adaptive transformation, the method comprising:

[0007] S1, acquire panchromatic and multispectral images;

[0008] S2, Process the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image;

[0009] S3, construct convolution kernels of different scales adaptively according to the scale ratio;

[0010] S4, the panchromatic image is iteratively degraded according to the convolution kernels of different scales to obtain a degraded panchromatic image;

[0011] S5, Upsample the degraded panchromatic image and the multispectral image to the panchromatic scale to obtain the upsampled panchromatic image and the upsampled multispectral image;

[0012] S6, the upsampled panchromatic image and the upsampled multispectral image are fused to obtain a fused image.

[0013] As an optional implementation, in the first aspect of the present invention, processing the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image includes:

[0014] S21, Detect the geographic information of the panchromatic image and the multispectral image to obtain the geographic overlap range of the panchromatic image and the multispectral image;

[0015] S22, Based on the geographical overlap range, calculate the pixel coordinates of the overlapping area of ​​the panchromatic image and the multispectral image to obtain the pixel coordinates of the overlapping area of ​​the panchromatic image and the overlapping area of ​​the multispectral image;

[0016] S23, process the pixel coordinates of the overlapping area of ​​the panchromatic image and the pixel coordinates of the overlapping area of ​​the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image.

[0017] As an optional implementation, in the first aspect of the present invention, processing the pixel coordinates of the overlapping region of the panchromatic image and the pixel coordinates of the overlapping region of the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image includes:

[0018] Using the scale ratio calculation model of panchromatic image and multispectral image, the pixel coordinates of the overlapping area of ​​the panchromatic image and the pixel coordinates of the overlapping area of ​​the multispectral image are processed to obtain the scale ratio of the panchromatic image and the multispectral image.

[0019] The scale ratio calculation model for the panchromatic and multispectral images is as follows:

[0020]

[0021] Among them, Pan overlap (x0, y0, x1, y1) are the pixel coordinates of the overlapping region of the panchromatic image, Mss overlap (x0,y0,x1,y1) represents the pixel coordinates of the overlapping area of ​​the multispectral images, (x0,y0) represents the coordinates of the top left corner, (x1,y1) represents the coordinates of the bottom right corner, and ratio represents the scale ratio between the panchromatic image and the multispectral image.

[0022] As an optional implementation, in the first aspect of the present invention, the step of adaptively constructing convolutional kernels of different scales according to the scale ratio includes:

[0023] S31, Process the scale ratio to obtain the scale integer quantity factor;

[0024] S32, Process the scale ratio to obtain a floating-point scale factor;

[0025] S33, determine the integer scaling factor and the floating-point scaling factor. If the integer scaling factor and the floating-point scaling factor are equal, construct convolution kernels of different scales using the Gaussian pyramid method. If the integer scaling factor and the floating-point scaling factor are not equal, add 1 to the integer scaling factor to obtain the optimized integer scaling factor.

[0026] S34, construct convolution kernels of different scales adaptively according to the integer factor of the optimization scale.

[0027] As an optional implementation, in the first aspect of the present invention, the step of adaptively constructing convolutional kernels of different scales according to the integer factor of the optimization scale includes:

[0028] S341, determine the number of scale layers N to be constructed based on the integer quantity factor of the optimization scale;

[0029] S342, layers 1 to (N-1), with the standard deviation of the Gaussian convolution kernel set to σ = 1.6;

[0030] S343, the Nth layer, sets the standard deviation of the Gaussian convolution kernel to:

[0031] σ=(convNumsDouble-convNumsInt)*1.6

[0032] Where convNumsDouble is the floating-point scaling factor, and convNumsInt is the integer scaling factor.

[0033] S344, using the standard deviation of the Gaussian convolution kernel, construct convolution kernels of different scales.

[0034] As an optional implementation, in the first aspect of the present invention, the iterative degradation of the panchromatic image based on the convolution kernels of different scales to obtain a degraded panchromatic image includes:

[0035] S41, Set the downsampling ratio according to the different scales of the convolution kernels;

[0036] S42, the panchromatic image is iteratively downgraded according to the downsampling ratio to obtain a downgraded panchromatic image.

[0037] As an optional implementation, in a first aspect of the present invention, upsampling the degraded panchromatic image and the multispectral image to a panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image includes:

[0038] Using an upsampling calculation model, the degraded panchromatic image and the multispectral image are upsampled to the panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image.

[0039] The upsampling calculation model is as follows:

[0040]

[0041]

[0042] Among them, Q 11 = (x1, y1), Q 12 = (x1, y2), Q 21 = (x2, y1), Q 22 = (x2, y2), and these four points are the points around the target point p(x, y) of the upsampling.

[0043] A second aspect of this invention discloses an SFIM fusion device based on scale-adaptive transformation, the device comprising:

[0044] The data acquisition module is used to acquire panchromatic and multispectral images;

[0045] The scale ratio calculation module is used to process the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image.

[0046] A convolution kernel calculation module is used to adaptively construct convolution kernels of different scales according to the scale ratio;

[0047] The degradation module is used to iteratively degrade the panchromatic image according to the convolution kernels of different scales to obtain a degraded panchromatic image;

[0048] The upsampling module is used to upsample the degraded panchromatic image and the multispectral image to the panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image.

[0049] The fusion module is used to fuse the upsampled panchromatic image and the upsampled multispectral image to obtain a fused image.

[0050] As an optional implementation, in the second aspect of the present invention, the processing of the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image includes:

[0051] S21, Detect the geographic information of the panchromatic image and the multispectral image to obtain the geographic overlap range of the panchromatic image and the multispectral image;

[0052] S22, Based on the geographical overlap range, calculate the pixel coordinates of the overlapping area of ​​the panchromatic image and the multispectral image to obtain the pixel coordinates of the overlapping area of ​​the panchromatic image and the overlapping area of ​​the multispectral image;

[0053] S23, process the pixel coordinates of the overlapping area of ​​the panchromatic image and the pixel coordinates of the overlapping area of ​​the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image.

[0054] As an optional implementation, in the second aspect of the present invention, processing the pixel coordinates of the overlapping region of the panchromatic image and the pixel coordinates of the overlapping region of the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image includes:

[0055] Using the scale ratio calculation model of panchromatic image and multispectral image, the pixel coordinates of the overlapping area of ​​the panchromatic image and the pixel coordinates of the overlapping area of ​​the multispectral image are processed to obtain the scale ratio of the panchromatic image and the multispectral image.

[0056] The scale ratio calculation model for the panchromatic and multispectral images is as follows:

[0057]

[0058] Among them, Pan overlap (x0, y0, x1, y1) are the pixel coordinates of the overlapping region of the panchromatic image, Mss overlap (x0,y0,x1,y1) represents the pixel coordinates of the overlapping area of ​​the multispectral images, (x0,y0) represents the coordinates of the top left corner, (x1,y1) represents the coordinates of the bottom right corner, and ratio represents the scale ratio between the panchromatic image and the multispectral image.

[0059] As an optional implementation, in a second aspect of the present invention, the step of adaptively constructing convolutional kernels of different scales according to the scale ratio includes:

[0060] S31, Process the scale ratio to obtain the scale integer quantity factor;

[0061] S32, Process the scale ratio to obtain a floating-point scale factor;

[0062] S33, determine the integer scaling factor and the floating-point scaling factor. If the integer scaling factor and the floating-point scaling factor are equal, construct convolution kernels of different scales using the Gaussian pyramid method. If the integer scaling factor and the floating-point scaling factor are not equal, add 1 to the integer scaling factor to obtain the optimized integer scaling factor.

[0063] S34, construct convolution kernels of different scales adaptively according to the integer factor of the optimization scale.

[0064] As an optional implementation, in a second aspect of the present invention, the step of adaptively constructing convolutional kernels of different scales based on the integer factor of the optimization scale includes:

[0065] S341, determine the number of scale layers N to be constructed based on the integer quantity factor of the optimization scale;

[0066] S342, layers 1 to (N-1), with the standard deviation of the Gaussian convolution kernel set to σ = 1.6;

[0067] S343, the Nth layer, sets the standard deviation of the Gaussian convolution kernel to:

[0068] σ=(convNumsDouble-convNumsInt)*1.6

[0069] Where convNumsDouble is the floating-point scaling factor, and convNumsInt is the integer scaling factor.

[0070] S344, using the standard deviation of the Gaussian convolution kernel, construct convolution kernels of different scales.

[0071] As an optional implementation, in a second aspect of the present invention, the iterative degradation of the panchromatic image based on convolution kernels of different scales to obtain a degraded panchromatic image includes:

[0072] S41, Set the downsampling ratio according to the different scales of the convolution kernels;

[0073] S42, the panchromatic image is iteratively downgraded according to the downsampling ratio to obtain a downgraded panchromatic image.

[0074] As an optional implementation, in a second aspect of the present invention, upsampling the degraded panchromatic image and the multispectral image to a panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image includes:

[0075] Using an upsampling calculation model, the degraded panchromatic image and the multispectral image are upsampled to the panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image.

[0076] The upsampling calculation model is as follows:

[0077]

[0078]

[0079] Among them, Q 11 = (x1, y1), Q 12 = (x1, y2), Q 21 = (x2, y1), Q 22 = (x2, y2), and these four points are the points around the target point p(x, y) of the upsampling.

[0080] A third aspect of the present invention discloses another SFIM fusion device based on scale-adaptive transformation, the device comprising:

[0081] Memory containing executable program code;

[0082] A processor coupled to the memory;

[0083] The processor calls the executable program code stored in the memory to execute some or all of the steps in the SFIM fusion method based on scale adaptive transformation disclosed in the first aspect of the present invention.

[0084] The fourth aspect of the present invention discloses a computer-storable medium storing computer instructions, which, when invoked, are used to execute some or all of the steps in the SFIM fusion method based on scale adaptive transformation disclosed in the first aspect of the present invention.

[0085] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0086] To adapt to the fusion of satellite imagery with varying panchromatic multispectral scales, this invention proposes a scale-adaptive SFIM fusion improvement method. This method adaptively processes panchromatic multispectral images with different scales, iteratively filtering out spatial information from the panchromatic images to ensure consistent sharpness with the multispectral images during fusion. This method effectively improves the SFIM fusion effect and is unaffected by panchromatic multispectral scale differences, enabling adaptive fusion of panchromatic multispectral images with varying scales. Attached Figure Description

[0087] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0088] Figure 1 This is a flowchart illustrating an SFIM fusion method based on scale adaptive transformation disclosed in an embodiment of the present invention;

[0089] Figure 2 This is a schematic diagram of the structure of an SFIM fusion device based on scale adaptive transformation disclosed in an embodiment of the present invention;

[0090] Figure 3 This is a schematic diagram of another SFIM fusion device based on scale adaptive transformation disclosed in an embodiment of the present invention. Detailed Implementation

[0091] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0092] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0093] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0094] This invention discloses a scale-adaptive SFIM fusion method and apparatus. The method includes: acquiring panchromatic and multispectral images; processing the panchromatic and multispectral images to obtain their scale ratios; adaptively constructing convolution kernels of different scales according to the scale ratios; iteratively degrading the panchromatic image using the convolution kernels of different scales to obtain a degraded panchromatic image; upsampling the degraded panchromatic image and the multispectral image to the panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image; and fusing the upsampled panchromatic image and the upsampled multispectral image to obtain a fused image. This invention effectively improves the SFIM fusion effect and is unaffected by differences in panchromatic and multispectral scales, enabling adaptive fusion of panchromatic and multispectral images with different scales. Detailed descriptions follow.

[0095] Example 1

[0096] Please see Figure 1 , Figure 1 This is a flowchart illustrating a scale-adaptive transformation-based SFIM fusion method disclosed in an embodiment of the present invention. Wherein, Figure 1 The scale-adaptive transformation-based SFIM fusion method described herein is applied in the field of remote sensing image processing. It adaptively processes panchromatic multispectral images with different scale differences, iteratively filtering out spatial information from the panchromatic images to ensure consistent sharpness with the multispectral images during ratio transformation. This invention does not limit the scope of the embodiments. Figure 1 As shown, the SFIM fusion method based on scale adaptive transformation can include the following operations:

[0097] S1, acquire panchromatic and multispectral images;

[0098] Optionally, bilinear interpolation can be performed on the multispectral image to make it consistent with the scale of the panchromatic image;

[0099] For the interpolated multispectral image, a low-resolution panchromatic image is fitted using the least squares method;

[0100] pan LR =X·γ

[0101] Among them, pan LR X is a low-resolution panchromatic image, X is the interpolated multispectral image, and γ is the regression coefficient vector.

[0102] S2, Process the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image;

[0103] S3, construct convolution kernels of different scales adaptively according to the scale ratio;

[0104] S4, the panchromatic image is iteratively degraded according to the convolution kernels of different scales to obtain a degraded panchromatic image;

[0105] S5, Upsample the degraded panchromatic image and the multispectral image to the panchromatic scale to obtain the upsampled panchromatic image and the upsampled multispectral image;

[0106] S6, the upsampled panchromatic image and the upsampled multispectral image are fused to obtain a fused image.

[0107] Optionally, the processing of the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image includes:

[0108] S21, Detect the geographic information of the panchromatic image and the multispectral image to obtain the geographic overlap range of the panchromatic image and the multispectral image;

[0109] S22, Based on the geographical overlap range, calculate the pixel coordinates of the overlapping area of ​​the panchromatic image and the multispectral image to obtain the pixel coordinates of the overlapping area of ​​the panchromatic image and the overlapping area of ​​the multispectral image;

[0110] S23, process the pixel coordinates of the overlapping area of ​​the panchromatic image and the pixel coordinates of the overlapping area of ​​the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image.

[0111] Geographic information mainly includes RPB and geographic six coefficients. Geographic information detection is a prior art in this field, and this embodiment does not impose any limitations on it.

[0112] Optionally, processing the pixel coordinates of the overlapping regions of the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image includes:

[0113] Using the scale ratio calculation model of panchromatic image and multispectral image, the pixel coordinates of the overlapping area of ​​the panchromatic image and the pixel coordinates of the overlapping area of ​​the multispectral image are processed to obtain the scale ratio of the panchromatic image and the multispectral image.

[0114] The scale ratio calculation model for the panchromatic and multispectral images is as follows:

[0115]

[0116] Among them, Pan overlap (x0, y0, x1, y1) are the pixel coordinates of the overlapping region of the panchromatic image, Mss overlap (x0,y0,x1,y1) represents the pixel coordinates of the overlapping area of ​​the multispectral images, (x0,y0) represents the coordinates of the top left corner, (x1,y1) represents the coordinates of the bottom right corner, and ratio represents the scale ratio between the panchromatic image and the multispectral image.

[0117] Optionally, the step of adaptively constructing convolutional kernels of different scales according to the scale ratio includes:

[0118] S31, Process the scale ratio to obtain the scale integer quantity factor;

[0119] S32, Process the scale ratio to obtain a floating-point scale factor;

[0120] S33, determine the integer scaling factor and the floating-point scaling factor. If the integer scaling factor and the floating-point scaling factor are equal, construct convolution kernels of different scales using the Gaussian pyramid method. If the integer scaling factor and the floating-point scaling factor are not equal, add 1 to the integer scaling factor to obtain the optimized integer scaling factor.

[0121] S34, construct convolution kernels of different scales adaptively according to the integer factor of the optimization scale.

[0122] Optionally, the step of adaptively constructing convolutional kernels of different scales based on the integer factor of the optimization scale includes:

[0123] S341, determine the number of scale layers N to be constructed based on the integer quantity factor of the optimization scale;

[0124] S342, layers 1 to (N-1), with the standard deviation of the Gaussian convolution kernel set to σ = 1.6;

[0125] S343, the Nth layer, sets the standard deviation of the Gaussian convolution kernel to:

[0126] σ=(convNumsDouble-convNumsInt)*1.6

[0127] Where convNumsDouble is the floating-point scaling factor, and convNumsInt is the integer scaling factor.

[0128] S344, using the standard deviation of the Gaussian convolution kernel, construct convolution kernels of different scales.

[0129] Optionally, the step of iteratively degrading the panchromatic image based on convolution kernels of different scales to obtain a degraded panchromatic image includes:

[0130] S41, Set the downsampling ratio according to the different scales of the convolution kernels;

[0131] S42, the panchromatic image is iteratively downgraded according to the downsampling ratio to obtain a downgraded panchromatic image.

[0132] Optionally, upsampling the degraded panchromatic image and the multispectral image to a panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image includes:

[0133] Using an upsampling calculation model, the degraded panchromatic image and the multispectral image are upsampled to the panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image.

[0134] The upsampling calculation model is as follows:

[0135]

[0136]

[0137] Among them, Q 11 = (x1, y1), Q 12 = (x1, y2), Q 21 = (x2, y1), Q 22 = (x2, y2), and these four points are the points around the target point p(x, y) of the upsampling.

[0138] Example 2

[0139] An improved SFIM fusion method based on scale adaptive transformation provided by this invention includes the following steps:

[0140] Step S101: Calculate the scale ratio of the panchromatic image and the multispectral image:

[0141] Due to the spectral separation characteristics of optical remote sensing satellite imagery, panchromatic and multispectral images are generally not the same size, which is the main reason why panchromatic and multispectral image fusion is necessary. Furthermore, in actual imaging processes, the ratio of panchromatic to multispectral images is usually not a strictly designed value, and different satellites may have different designed ratios. Therefore, the first step is to calculate the scale ratio between the panchromatic and multispectral images.

[0142] The system determines whether geographic information exists in the input panchromatic and multispectral images. If geographic information exists, it calculates the geographic overlap between the panchromatic and multispectral images. Based on this overlap, the pixel coordinates of the overlapping areas in the panchromatic and multispectral images are then calculated to obtain the pixel coordinates of the overlapping region.

[0143] Pan overlap (x0,y0,x1,y1)

[0144] Mss overlap (x0,y0,x1,y1)

[0145] Where (x0, y0) are the coordinates of the top left corner, (x1, y1) are the coordinates of the bottom right corner, and ratio is the scale ratio between the panchromatic image and the multispectral image:

[0146]

[0147] Step S102: Adaptively construct convolutional kernels of different scales according to the scale ratio:

[0148] After calculating the scale ratio between the panchromatic image and the multispectral image in step S101, convolution kernels of different scales can be constructed. One of the most crucial steps in the SFIM method is the panchromatic image degradation. Classical SFIM methods use mean convolution kernels for degradation, while some improved algorithms use Gaussian convolution kernels. However, both mean and Gaussian convolution kernels require pre-defined kernels, necessitating the design of different kernels for different satellites. These kernels are typically based on pre-defined values ​​and do not account for scale changes during imaging. Furthermore, as the scale differences between panchromatic and multispectral images increase, a single convolution kernel becomes insufficient for large-scale panchromatic and multispectral fusion. Therefore, this invention adaptively constructs convolution kernels of different scales based on the Gaussian pyramid to achieve optimal results. Furthermore, only 2 Gaussian pyramids can be used to construct convolution kernels of different scales. n The convolution kernel has been improved by first calculating the number of integer scales (convNumsInt) needed to construct the kernel:

[0149] convNumsInt = log2(ratio)

[0150] Next, we need to construct a floating-point quantity convNumsDouble to calculate the kernel size:

[0151] convNumsDouble = log2(ratio)

[0152] If convNumsInt = convNumsDouble, it means the scale difference is exactly 2. n Multi-scale convolution kernels can be constructed using the traditional Gaussian pyramid method, but if they are not equal, it indicates that the scale difference is not equal to 2. n Then the number of scale layers that need to be constructed is: convNumsInt = convNumsInt + 1.

[0153] This adaptively determines the number of scales required for construction. Convolutional kernels for different layers are constructed based on the number of scales. Referring to the Gaussian pyramid, this invention uses Gaussian convolutional kernels. Specifically, within the first convNumsInt-1 layers, the Gaussian convolutional kernels all use a standard deviation σ = 1.6. The Gaussian convolutional kernel is represented as follows:

[0154]

[0155] In the above formula, (x,y) represents the coordinates of any point within the Gaussian filter kernel, (x′,y′) is the coordinates of the center point of the Gaussian filter kernel, σ is the Gaussian filter parameter, and f(x,y) is the filter output.

[0156] If convNumsInt ≠ convNumsDouble, at the convNumsInt level, different standard deviations need to be estimated. The estimation method is as follows:

[0157] σ=(convNumsDouble-convNumsInt)*1.6

[0158] If convNumsInt = convNumsDouble, the standard deviation at the convNumsInt level is σ = 1.6.

[0159] Apart from the difference in standard deviation σ, the construction method of the convolution kernel in the last layer is the same as that in the previous layers, all using Gaussian convolution kernels.

[0160] Step S103: Iteratively degrade the panchromatic image using the constructed convolution kernel.

[0161] Optionally, before performing iterative degradation, the multispectral image is first bilinearly interpolated to give it the same scale as the panchromatic image;

[0162] Then, PCA transformation is performed on the interpolated multispectral image:

[0163] Y = AX

[0164] Where: X is multispectral image data, Y is principal component data, and A is the multispectral covariance matrix C. X An orthogonal matrix formed by the eigenvectors of .

[0165] Based on the mean and variance of the first principal component obtained from the PCA transformation of the multispectral image, the gray values ​​of the panchromatic image are adjusted to obtain a new panchromatic image, which is equivalent to performing image histogram matching.

[0166]

[0167] In the formula, σ pca1 and σ pan The first principal component of the multispectral image and the standard deviation of the panchromatic image; u pca1 and u pan Pan represents the first principal component of the multispectral image and the mean of the panchromatic image. new The new panchromatic image is represented by 'pan', while 'pan' represents the original panchromatic image.

[0168] By adjusting the grayscale values ​​of the panchromatic image, a new panchromatic image is obtained. Subsequent processing and fusion of the new panchromatic image will yield a better fusion effect.

[0169] The next steps will be performed on the new panchromatic image.

[0170] In step S102, the required number of convolutional layers and the corresponding convolutional kernels for different layer numbers have been adaptively calculated. This step iteratively downgrades the new panchromatic image based on the calculation results from the previous step. Each layer uses the corresponding convolutional kernel for convolution, and downsampling is performed after convolution. The downsampling ratio for the first convNumsInt-1 layers is 2. Considering both computational efficiency and downsampling effect, this invention adopts a bilinear resampling method, as shown in the following formula:

[0171]

[0172]

[0173] Among them, Q 11 = (x1, y1), Q 12 = (x1, y2), Q 21 = (x2, y1), Q 22 = (x2, y2), and these four points are the points around the target point p(x, y) of the upsampling.

[0174] If convNumsInt ≠ convNumsDouble, the downsampling ratio of the convNumsInt layer is:

[0175] ratio last =(convNumsDouble-convNumsInt)*2

[0176] If convNumsInt = convNumsDouble, the downsampling ratio in the convNumsInt layer is 2.

[0177] After the final layer is convolved and downsampled, the degradation processing of the new panchromatic image is completed. At this point, the scale and sharpness of the panchromatic image are consistent with those of the multispectral image.

[0178] Step S104: Upsample the degraded panchromatic image and multispectral image to the panchromatic scale:

[0179] After the calculations in steps S101 to S103, a degraded panchromatic image with consistent resolution and scale with the multispectral image can be obtained. At this point, both the degraded panchromatic image and the multispectral image need to be upsampled to the panchromatic scale, i.e., the fusion scale. The upsampling resampling method, considering both computational efficiency and resampling effectiveness, also uses the bilinear resampling model mentioned in the previous steps.

[0180] After this step, we can obtain the original panchromatic image Pan, the degraded panchromatic image Pan′, and the upsampled multispectral image Mss′ at the panchromatic scale. At this point, we can perform the final fusion operation.

[0181] Step S105 uses the ratio method to fuse panchromatic and multispectral data to obtain the final fusion result.

[0182] The SFIM model states that:

[0183]

[0184] Fusion represents the fusion result. The ratio of Pan to Pan′ preserves only the edge details of the high-resolution image while essentially eliminating spectral and contrast information. The SFIM model can be viewed as adding high-resolution details to a low-resolution image; its fusion result is independent of the spectral differences in the high-resolution image, and the SFIM model exhibits good color preservation capabilities. However, the key to SFIM lies in the generation of superior degraded panchromatic images. The adaptive transformation degradation method proposed in this invention can adapt to different satellites and panchromatic and multispectral images with varying scales, adaptively acquiring superior degraded panchromatic images and effectively improving traditional SFIM methods.

[0185] Optionally, an improved SFIM method can be used to fuse panchromatic and multispectral image data to obtain the final fusion result. The improved SFIM method is as follows:

[0186] Using the formula:

[0187]

[0188] In the formula, I is the brightness component extracted from the multispectral image, and k1,k2,…,k k These are the adjustment factors for each band, where Band1, Band2, ..., Bandk are the bands of the multispectral image. SFIM Band2 SFIM ,…,Bandk SFIM For each band of the fused multispectral image, p represents the panchromatic image data, and p′ represents the multispectral bands. (k) This is panchromatic degraded image data for each band of multispectral image data.

[0189] Alternatively, the following methods can be used to process multispectral images:

[0190] First, bilinear interpolation is performed on the multispectral image to give it the same scale as the panchromatic image;

[0191] Then, PCA transformation is performed on the interpolated multispectral image:

[0192] Y = AX

[0193] Where: X is multispectral image data, Y is principal component data, and A is the multispectral covariance matrix C. X An orthogonal matrix formed by the eigenvectors of .

[0194] Based on the mean and variance of the first principal component obtained from the PCA transformation of the multispectral image, the gray values ​​of the panchromatic image are adjusted to obtain a new panchromatic image, which is equivalent to performing image histogram matching.

[0195]

[0196] In the formula, σ pca1 and σ pan The first principal component of the multispectral image and the standard deviation of the panchromatic image; u pca1 and u pan The first principal component of the multispectral image and the mean of the panchromatic image are given.

[0197] The new panchromatic image and the first principal component are respectively subjected to wavelet decomposition. In this embodiment, the redundancy-direction lifting wavelet transform is used, and the decomposition level is 3. The mean value of the coefficients at the corresponding positions of the low-frequency components is taken as the fusion coefficient, and its expression is:

[0198]

[0199] In the formula, and , respectively, are the coefficients of the first principal component of the multispectral image, the low-frequency component of the panchromatic image, and the fused image at (m,n).

[0200] For high-frequency components, the coefficient with the larger absolute value at corresponding positions of the two components is taken as the fusion coefficient, as shown below:

[0201]

[0202] In the formula, and , respectively, are the coefficients of the high-frequency components (m,n) in the j-th decomposition layer and k-direction of the first principal component of the multispectral image, the panchromatic image, and the fused image, where j = 1, 2, 3, and k represent the horizontal, vertical, and diagonal directions.

[0203] The fusion wavelet coefficients are readjusted according to the consistency principle. That is, if more than 6 coefficients in the 3×3 neighborhood space of the fusion coefficients at a certain position are from the panchromatic image, then the wavelet coefficients at that position are adjusted accordingly to be from the panchromatic image. Similarly, if most of the coefficients in the neighborhood space are from the first principal component coefficients of the multispectral image, a similar adjustment is made.

[0204] Based on the obtained fused wavelet coefficients, perform the corresponding inverse wavelet transform to obtain the fused principal component components;

[0205] Using the fused image components obtained above as the first principal components, and performing inverse PCA transform on them, the fused image can be obtained:

[0206] F = A T Y new

[0207] Among them, Y new This is the multispectral principal component data after the first principal component has been replaced, A T F is the transpose of the PCA transform matrix, and F is the fused multispectral image.

[0208] Among them, the redundant direction lifting wavelet transform is performed based on the lifting wavelet transform. In the prediction and update steps, the selection of prediction and update operators directly affects the effectiveness of the decomposition. According to the description of the adaptive direction lifting wavelet transform, The operator in the z domain can be improved to:

[0209]

[0210] Here θ vIt is the strongest directional correlation angle. The Sinc interpolation technique is used to obtain the sub-pixel values ​​between two adjacent integer pixels. This is used in calculating p. e (m,n) and u h When (m,n), the averaging algorithm is used:

[0211]

[0212]

[0213]

[0214] Where α i β j It represents the value of the i-th and j-th sub-pixels. This improves computational efficiency and saves memory, allowing for the acquisition of more directional information, rather than just the one from the strongest direction.

[0215] The "decomposition" step in traditional wavelet transform performs a downsampling step on the input image. To eliminate the Gibbs phenomenon caused by this step, a "copying" method is used instead of the downsampling step. That is, both the odd and even terms are consistent with the source image. This is called Redundant Lifting Wavelet Transform (RDL-WT).

[0216]

[0217] Here, 'I' represents the source image. The source image is decomposed into four coefficient matrices by RDL-WT, including one base layer matrix and three detail layer matrices.

[0218] Alternatively, the specific steps of another method for fusing multispectral and panchromatic images are as follows:

[0219] 1. First, perform bilinear interpolation on the multispectral image to make it consistent with the scale of the panchromatic image;

[0220] 2. Using the interpolated multispectral image, a low-resolution panchromatic image is fitted using the least squares method;

[0221] pan LR =X·γ

[0222] Among them, pan LR X is a low-resolution panchromatic image, X is the interpolated multispectral image, and γ is the regression coefficient vector.

[0223] γ=(X T ·X) -1 ·X·pan LR

[0224] 3. Using the fitted low-resolution panchromatic image as GS1, perform GS transform on the multispectral image (a classic image transformation method):

[0225]

[0226] In the formula, GS T X is the Tth component of GS; T-1 This represents the T-1 band component of the original multispectral array, with the band mean value as follows:

[0227]

[0228] In the formula, R and C are the number of rows and columns of the original image, respectively, and the parameter Φ(X) is... T-1 GS i )for:

[0229]

[0230] Where, σ(X) T-1 GS i ) is band X T-1 and band GS i The covariance.

[0231] 4. Perform histogram matching between the high-resolution panchromatic image and the low-resolution panchromatic image to ensure they have the same mean and variance:

[0232]

[0233] In the formula, and σ pan This represents the standard deviation of the low-resolution panchromatic image and the high-resolution panchromatic image. panLR and u pan This represents the average of the low-resolution panchromatic image and the high-resolution panchromatic image.

[0234] 5. Perform Level 1 NSCT (Non-Sampling Prototype Transform) decomposition on the fitted low-resolution panchromatic image in 8 directions. Perform Level 3 NSCT decomposition on the matched high-resolution image in 8, 2, and 2 directions respectively. Use the low-frequency sub-band image from the multispectral image as the fused low-frequency sub-band image, and use the Level 2 and 3 NSCT decomposition images from the high-resolution panchromatic image as the fused Level 2 and 3 NSCT decomposition images. Then, fuse the Level 1 8-direction sub-band images according to a fusion rule based on window region energy.

[0235] First, calculate the correlation coefficient based on the window region (5×5):

[0236]

[0237] When the correlation coefficient C XYWhen (m,n)≤a (a is 0.85), it indicates that the correlation between the two is relatively low. The fusion coefficient is selected based on the magnitude of the regional energy. The formula for calculating the regional energy is as follows:

[0238]

[0239]

[0240] Where w(i,j) is a low-pass filter template of size 5×5, and when the correlation coefficient C XY When (m,n)>a (a is 0.85), it indicates that the correlation coefficient between the two is relatively high. Based on the energy of the window, the fusion rule is selected as follows:

[0241] C F (m,n)=S(m,n)·C X (m,n)+(1-S(m,n))·C Y (m,n)

[0242]

[0243] 6. Perform NSCT inverse transform on the fused sub-band image to obtain a fused panchromatic image containing low-frequency information from the low-resolution panchromatic image and high-frequency information from the high-resolution panchromatic image;

[0244] 7. Replace GS1 with the fused panchromatic image and perform an inverse GS transform to obtain the fused multispectral image. The formula for the inverse GS transform is as follows:

[0245]

[0246] Optionally, after fusion, the fused multispectral images can be enhanced using the following methods to improve the quality of the fusion result:

[0247] The fused multispectral image to be enhanced is obtained. The fused multispectral image is decomposed into a base layer containing only structural information and a detail layer containing only details and noise using the TV norm model. The layers are then enhanced separately. The regularization coefficient of the TV model is adaptively given by the estimated noise level.

[0248] Base layer processing: The base layer is decomposed into illumination and reflection components using the Retinex model. The illumination component is processed using nonlinear Sigmoid transformation to stretch contrast and highlight details. The reflection component is equalized using gamma correction.

[0249] Detail layer processing: Use a Gaussian filter to remove excess noise from the detail layer and improve its quality.

[0250] The base layer and detail layer of the fused multispectral image to be enhanced are processed using the trained fused multispectral image enhancement model to obtain the enhanced fused multispectral image.

[0251] Optionally, the input fused multispectral image I(x,y) is a superposition of two layers:

[0252] I(x,y)=I base (x,y)+I detail (x,y)

[0253] The basic layer is I. base (x,y), the detail layer is I detail (x,y), the base layer is obtained by minimizing the following objective function.

[0254]

[0255] The first term is a difference term adapted to the texture components to preserve meaningful structure, the second term is a regularization term based on the total variation, ▽ represents the gradient operator, and λ is 3 times the global noise estimate σ, λ=3σ.

[0256] Global noise estimation σ is

[0257]

[0258] Where * represents the convolution operator, W and H are the width and height of image I(x,y) respectively, and N is the convolution template:

[0259]

[0260] The detail layer is:

[0261] I detail (x,y)=I(x,y)-I base (x,y)

[0262] Example 3

[0263] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an SFIM fusion device based on scale adaptive transformation disclosed in an embodiment of the present invention. Figure 2 The scale-adaptive transformation-based SFIM fusion device described herein is applied in the field of remote sensing image processing. It adaptively processes panchromatic multispectral images with different scale differences, iteratively filtering out spatial information from the panchromatic images to ensure consistent sharpness with the multispectral images during ratio transformation. This invention does not limit the scope of the embodiments described. Figure 2 As shown, the SFIM fusion device based on scale adaptive transformation can include the following operations:

[0264] S301, data acquisition module, used to acquire panchromatic and multispectral images;

[0265] S302, Scale ratio calculation module, used to process the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image;

[0266] S303, Convolution kernel calculation module, used to adaptively construct convolution kernels of different scales according to the scale ratio;

[0267] S304, Degradation module, used to iteratively degrade the panchromatic image according to the convolution kernels of different scales to obtain a degraded panchromatic image;

[0268] S305, Upsampling module, used to upsample the degraded panchromatic image and the multispectral image to the panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image;

[0269] S306, fusion module, used to fuse the upsampled panchromatic image and the upsampled multispectral image to obtain a fused image.

[0270] Example 4

[0271] Please see Figure 3 , Figure 3 This is a schematic diagram of another SFIM fusion device based on scale adaptive transformation disclosed in an embodiment of the present invention. Figure 3 The scale-adaptive transformation-based SFIM fusion device described herein is applied in the field of remote sensing image processing. It adaptively processes panchromatic multispectral images with different scale differences, iteratively filtering out spatial information from the panchromatic images to ensure consistent sharpness with the multispectral images during ratio transformation. This invention does not limit the scope of the embodiments described. Figure 3 As shown, the SFIM fusion device based on scale adaptive transformation can include the following operations:

[0272] Memory 401 storing executable program code;

[0273] Processor 402 coupled to memory 401;

[0274] The processor 402 calls the executable program code stored in the memory 401 to execute the steps in the SFIM fusion method based on scale adaptive transformation described in Embodiment 1 and Embodiment 2.

[0275] Example 5

[0276] This invention discloses a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to execute the steps in the scale-adaptive transformation-based SFIM fusion method described in Embodiments 1 and 2.

[0277] The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0278] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0279] Finally, it should be noted that the SFIM fusion method and apparatus based on scale adaptive transformation disclosed in the embodiments of the present invention are merely preferred embodiments of the present invention and are only used to illustrate the technical solutions of the present invention, not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A SFIM fusion method based on scale-adaptive transformation, characterized in that, The method includes: S1, acquire panchromatic and multispectral images; S2, Processing the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image, including: S21, Detect the geographic information of the panchromatic image and the multispectral image to obtain the geographic overlap range of the panchromatic image and the multispectral image; S22, Based on the geographical overlap range, calculate the pixel coordinates of the overlapping area of ​​the panchromatic image and the multispectral image to obtain the pixel coordinates of the overlapping area of ​​the panchromatic image and the overlapping area of ​​the multispectral image; S23, Process the pixel coordinates of the overlapping region of the panchromatic image and the overlapping region of the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image, including: Using the scale ratio calculation model of panchromatic image and multispectral image, the pixel coordinates of the overlapping area of ​​the panchromatic image and the pixel coordinates of the overlapping area of ​​the multispectral image are processed to obtain the scale ratio of the panchromatic image and the multispectral image. The scale ratio calculation model for the panchromatic and multispectral images is as follows: in, These are the pixel coordinates of the overlapping area of ​​the panchromatic images. These are the pixel coordinates of the overlapping region of the multispectral images. The coordinates of the top left corner point, The coordinates of the bottom right corner point, The scale ratio of panchromatic and multispectral images; S3, adaptively constructing convolutional kernels of different scales according to the scale ratio, including: S31, Process the scale ratio to obtain the scale integer quantity factor; S32, Process the scale ratio to obtain a floating-point scale factor; S33, determine the integer scaling factor and the floating-point scaling factor. If the integer scaling factor and the floating-point scaling factor are equal, construct convolution kernels of different scales using the Gaussian pyramid method. If the integer scaling factor and the floating-point scaling factor are not equal, add 1 to the integer scaling factor to obtain the optimized integer scaling factor. S34, adaptively constructing convolutional kernels of different scales based on the integer factor of the optimization scale, including: S341, determine the number of scale layers N to be constructed based on the integer quantity factor of the optimization scale; S342, layers 1 to (N-1), with the standard deviation of the Gaussian convolution kernel set to [value missing]. ; S343, the Nth layer, if convNumsInt convNumsDouble sets the standard deviation of the Gaussian convolution kernel to: If convNumsInt = convNumsDouble, the standard deviation is ,in, The scaling factor is a floating-point number. To optimize the integer scale factor; S344, using the standard deviation of the Gaussian convolution kernel, construct convolution kernels of different scales; S4, the panchromatic image is iteratively degraded according to the convolution kernels of different scales to obtain a degraded panchromatic image; S5, Upsample the degraded panchromatic image and the multispectral image to the panchromatic scale to obtain the upsampled panchromatic image and the upsampled multispectral image; S6, the upsampled panchromatic image and the upsampled multispectral image are fused to obtain a fused image.

2. The SFIM fusion method based on scale adaptive transformation according to claim 1, characterized in that, The iterative degradation of the panchromatic image based on convolution kernels of different scales to obtain a degraded panchromatic image includes: S41, Set the downsampling ratio according to the different scales of the convolution kernels; S42, the panchromatic image is iteratively downgraded according to the downsampling ratio to obtain a downgraded panchromatic image.

3. The SFIM fusion method based on scale adaptive transformation according to claim 1, characterized in that, The step of upsampling the degraded panchromatic image and the multispectral image to a panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image includes: Using an upsampling calculation model, the degraded panchromatic image and the multispectral image are upsampled to the panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image. The upsampling calculation model is as follows: in, , , , And these four points are the target points for upsampling. The surrounding points.

4. A scale-adaptive transformation-based SFIM fusion device, characterized in that, The device includes: The data acquisition module is used to acquire panchromatic and multispectral images; The scale ratio calculation module is used to process the panchromatic image and the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image, including: S21, Detect the geographic information of the panchromatic image and the multispectral image to obtain the geographic overlap range of the panchromatic image and the multispectral image; S22, Based on the geographical overlap range, calculate the pixel coordinates of the overlapping area of ​​the panchromatic image and the multispectral image to obtain the pixel coordinates of the overlapping area of ​​the panchromatic image and the overlapping area of ​​the multispectral image; S23, Process the pixel coordinates of the overlapping region of the panchromatic image and the overlapping region of the multispectral image to obtain the scale ratio of the panchromatic image and the multispectral image, including: Using the scale ratio calculation model of panchromatic image and multispectral image, the pixel coordinates of the overlapping area of ​​the panchromatic image and the pixel coordinates of the overlapping area of ​​the multispectral image are processed to obtain the scale ratio of the panchromatic image and the multispectral image. The scale ratio calculation model for the panchromatic and multispectral images is as follows: in, These are the pixel coordinates of the overlapping area of ​​the panchromatic images. These are the pixel coordinates of the overlapping region of the multispectral images. The coordinates of the top left corner point, The coordinates of the bottom right corner point, The scale ratio of panchromatic and multispectral images; A convolution kernel calculation module, used to adaptively construct convolution kernels of different scales according to the scale ratio, includes: S31, Process the scale ratio to obtain the scale integer quantity factor; S32, Process the scale ratio to obtain a floating-point scale factor; S33, determine the integer scaling factor and the floating-point scaling factor. If the integer scaling factor and the floating-point scaling factor are equal, construct convolution kernels of different scales using the Gaussian pyramid method. If the integer scaling factor and the floating-point scaling factor are not equal, add 1 to the integer scaling factor to obtain the optimized integer scaling factor. S34, adaptively constructing convolutional kernels of different scales based on the integer factor of the optimization scale, including: S341, determine the number of scale layers N to be constructed based on the integer quantity factor of the optimization scale; S342, layers 1 to (N-1), with the standard deviation of the Gaussian convolution kernel set to [value missing]. ; S343, the Nth layer, if convNumsInt convNumsDouble sets the standard deviation of the Gaussian convolution kernel to: If convNumsInt = convNumsDouble, the standard deviation is ,in, The scaling factor is a floating-point number. To optimize the integer scale factor; S344, using the standard deviation of the Gaussian convolution kernel, construct convolution kernels of different scales; The degradation module is used to iteratively degrade the panchromatic image according to the convolution kernels of different scales to obtain a degraded panchromatic image; The upsampling module is used to upsample the degraded panchromatic image and the multispectral image to the panchromatic scale to obtain an upsampled panchromatic image and an upsampled multispectral image. The fusion module is used to fuse the upsampled panchromatic image and the upsampled multispectral image to obtain a fused image.

5. A scale-adaptive transformation-based SFIM fusion device, characterized in that, The device includes: Memory containing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the SFIM fusion method based on scale adaptive transformation as described in any one of claims 1-3.

6. A computer-storable medium, characterized in that, The computer storage medium stores computer instructions, which, when invoked, are used to execute the SFIM fusion method based on scale adaptive transformation as described in any one of claims 1-3.