Remote sensing image fusion method and device, electronic equipment and storage medium

Abstract

The application discloses a remote sensing image fusion method and device, electronic equipment and storage medium, and particularly relates to the technical field of remote sensing image processing. The method comprises the following steps: performing normalization processing on multi-band remote sensing images to obtain normalized multi-band remote sensing images; generating a plurality of exposure state layers simulating different exposure degrees based on the normalized multi-band remote sensing images; performing multi-scale decomposition on each exposure state layer to obtain image subbands at different scale levels; at each scale level, determining the composite weight of the image subband according to the medium gray exposure degree, the image contrast, the image saturation, the level modulation feature and the cross-band consistency feature; and performing intra-layer weighted fusion on the image subbands of different exposure state layers at the same scale level based on the composite weight, and performing layer-by-layer reconstruction from top to bottom according to different scale levels to obtain a fused target remote sensing image.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a remote sensing image fusion method, apparatus, electronic device, and storage medium. Background Technology

[0002] In remote sensing imaging, the limited dynamic range of sensors leads to issues such as saturation in bright areas and loss of detail in dark areas in high-contrast scenes. Acquiring multiple images of the same area with different exposures in remote sensing presents significant hardware and spatiotemporal requirements. Spaceborne or aerial cameras often struggle to acquire multiple images with different exposures during the same passing time, unless specialized imaging hardware or multi-aircraft formation acquisition is used. Even when multi-exposure images are acquired, high-precision registration of the multi-temporal images is necessary; otherwise, ghosting and blur artifacts will occur during fusion. Especially in dynamic scenes where ground features and the imaging platform are moving, multiple exposure sequences cannot be perfectly aligned, and traditional multi-exposure fusion algorithms struggle to avoid ghosting and distortion caused by motion. For example, a common method in high dynamic range imaging is multi-exposure fusion, but because different exposures are acquired at different times, motion artifacts are easily generated, making it unsuitable for dynamic scenes.

[0003] How to construct multiple layers simulating different exposures and fuse them when only a single remote sensing image is available, in order to overcome the difficulty of acquiring multi-exposure images and avoid registration and ghosting, has become an urgent problem to be solved.

[0004] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to solve the problem of how to construct multiple layers simulating different exposures and fuse them when only a single remote sensing image is available, so as to overcome the difficulty of acquiring multi-exposure images and avoid registration and ghosting problems. This invention provides a remote sensing image fusion method, device, electronic device and storage medium.

[0006] The first aspect of this invention provides a remote sensing image fusion method, comprising: normalizing multi-band remote sensing images to obtain normalized multi-band remote sensing images; generating multiple exposure state layers simulating different exposure levels based on the normalized multi-band remote sensing images; performing multi-scale decomposition on each exposure state layer to obtain image sub-bands at different scale levels; determining the composite weight of the image sub-bands at each scale level based on mid-gray exposure, image contrast, image saturation, layer modulation features, and cross-band consistency features; and performing intra-layer weighted fusion of image sub-bands of different exposure state layers at the same scale level based on the composite weights, and performing layer-by-layer reconstruction from top to bottom according to the different scale levels to obtain a fused target remote sensing image.

[0007] In one embodiment of the present invention, generating multiple exposure state layers simulating different exposure levels based on the normalized multi-band remote sensing image includes: acquiring the intensity channel or brightness channel of the multi-band remote sensing image; and performing a nonlinear exponential transformation or a linear offset transformation on the pixel values ​​of the intensity channel or brightness channel to generate the multiple exposure state layers.

[0008] In one embodiment of the present invention, the step of performing multi-scale decomposition on each of the exposure state layers to obtain image subbands at different scale levels includes: performing multi-level decomposition on each of the exposure state layers using a Gaussian pyramid and a Laplacian pyramid, wherein the top layer of the pyramid is a low-frequency Gaussian image, and the remaining layers are Laplacian high-frequency detail images.

[0009] In one embodiment of the present invention, determining the composite weight of the image sub-band at each scale level based on mid-gray exposure, image contrast, image saturation, hierarchical modulation features, and cross-band consistency features includes: obtaining the image contrast, mid-gray exposure, and image saturation of the image sub-band at the current scale level, and applying preset power exponents for contrast weight, exposure weight, and saturation weight to adjust them exponentially; performing a joint product operation on the adjusted image contrast, mid-gray exposure, and image saturation with the hierarchical modulation features, the cross-band consistency features, and a pre-generated pixel mask to obtain an initial composite weight; and normalizing the initial composite weight to determine the composite weight.

[0010] In one embodiment of the present invention, obtaining the mid-gray exposure of the image sub-band at the current scale level includes: determining the difference between the pixel value of the low-frequency image sub-band at the current scale level and a preset normalized mid-gray brightness anchor point; inputting the difference into a preset Gaussian attenuation model for mapping calculation to obtain the mid-gray exposure.

[0011] In one embodiment of the present invention, before obtaining the image contrast, mid-gray exposure, and image saturation of the image sub-band at the current scale level, the method further includes: obtaining a preset intermediate scale level index, a dark area scale level index, and a bright area scale level index; determining a basic modulation value based on the level deviation between the index value of the current scale level and the intermediate scale level index using a preset attenuation function; constructing an adjustment function based on the dark area scale level index and the bright area scale level index, and correcting the basic modulation value based on the adjustment function to obtain the level modulation feature.

[0012] In one embodiment of the present invention, before obtaining the image contrast, mid-gray exposure, and image saturation of the image sub-band at the current scale level, the method further includes: summing the pixel values ​​of the normalized multi-band remote sensing image in each band to obtain a cross-band brightness reference value; introducing a preset bias constant into the normalized pixel value of the current band and the cross-band brightness reference value respectively; calculating the ratio of the normalized pixel value of the current band after introducing the bias constant to the cross-band brightness reference value after introducing the bias constant; and exponentially transforming the ratio using a preset consistency adjustment index to obtain the cross-band consistency feature.

[0013] A second aspect of the present invention provides a remote sensing image fusion apparatus, comprising: a preprocessing module for normalizing multi-band remote sensing images to obtain normalized multi-band remote sensing images; an exposure state construction module for generating multiple exposure state layers simulating different exposure levels based on the normalized multi-band remote sensing images; a decomposition module for performing multi-scale decomposition on each exposure state layer to obtain image sub-bands at different scale levels; a determination module for determining the composite weight of the image sub-bands at each scale level based on mid-gray exposure, image contrast, image saturation, layer modulation features, and cross-band consistency features; and a fusion module for performing intra-layer weighted fusion of image sub-bands of different exposure state layers at the same scale level based on the composite weight, and performing layer-by-layer reconstruction from top to bottom according to the different scale levels to obtain a fused target remote sensing image.

[0014] A third aspect of the present invention provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a remote sensing image fusion method as described in any of the first aspects.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the remote sensing image fusion method as described in the first aspect.

[0016] Compared with the prior art, the technical effects achieved by the present invention are as follows:

[0017] 1. This invention uses a single image to simulate multiple exposures for fusion, avoiding the cumbersome process and equipment requirements of traditional high dynamic range (HDR) imaging, which requires multiple shots.

[0018] 2. This invention only requires acquiring one multi-band remote sensing image to complete subsequent enhancement processing, making data acquisition simple and convenient, and more practical.

[0019] 3. Since all exposure layers originate from the same image at the same time, this invention fundamentally eliminates registration errors and dynamic scene ghosting problems in multi-image fusion. The fusion result retains key information from each exposure, enhancing details in both shadows and highlights, and significantly expanding the overall dynamic range. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a remote sensing image fusion method according to an embodiment of the present invention; Figure 2a This is one of the stretched display effect diagrams of the original image in an urban scene according to an embodiment of the present invention; Figure 2b This is a schematic diagram of the processing results of the Retinex algorithm in an urban scene according to an embodiment of the present invention; Figure 2c This is a schematic diagram of the algorithm processing results in an urban scene according to an embodiment of the present invention; Figure 3a This is a stretched display effect of the original image in a mountainous cloud and fog scene according to an embodiment of the present invention; Figure 3b This is a schematic diagram of the Retinex algorithm processing results in a mountainous cloud and fog scene according to an embodiment of the present invention; Figure 3c This is a schematic diagram of the algorithm processing results in a mountainous cloud and fog scene according to an embodiment of the present invention; Figure 4a This is a stretched display effect of the original image in a river scene according to an embodiment of the present invention; Figure 4b This is a schematic diagram of the processing results of the Retinex algorithm in a river scene according to an embodiment of the present invention; Figure 4c This is a schematic diagram of the algorithm processing results in a river scene according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the frame of a remote sensing image fusion device according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the frame of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0021] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0022] The technical solution of the present invention is illustrated below through specific embodiments. It should be understood that the one or more steps mentioned in the present invention do not preclude the existence of other methods and steps before or after the combined steps, or that other methods and steps may be inserted between these explicitly mentioned steps. It should also be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Unless otherwise stated, the numbering of each method step is only for the purpose of identifying each method step, and not for limiting the order of each method or limiting the scope of the present invention. Changes or adjustments to their relative relationships, without substantial changes to the technical content, can also be considered as within the scope of the present invention.

[0023] According to a first aspect of the present invention, a remote sensing image fusion method is provided, such as... Figure 1 As shown, the method may specifically include the following steps: S101, Normalize the multi-band remote sensing image to obtain the normalized multi-band remote sensing image. S102, based on normalized multi-band remote sensing images, generates multiple exposure state layers simulating different exposure levels; S103, perform multi-scale decomposition on each exposure state layer to obtain image subbands at different scale levels; S104. At each scale level, the composite weight of the image sub-band is determined based on the mid-gray exposure, image contrast, image saturation, hierarchical modulation characteristics, and cross-band consistency characteristics. S105, based on composite weights, performs intra-layer weighted fusion of image subbands of different exposure states at the same scale level, and performs layer-by-layer reconstruction from top to bottom according to different scale levels to obtain the fused target remote sensing image.

[0024] In this embodiment, multi-band remote sensing imagery refers to remote sensing image data containing two or more spectral bands, such as multispectral satellite imagery containing red, green, blue (RGB) and near-infrared bands. Due to the limited dynamic range of remote sensing sensors, the original input imagery often exhibits uneven brightness distribution or extreme value anomalies.

[0025] In this embodiment, normalization refers to mapping the original pixel values ​​to a unified numerical range to facilitate subsequent unified calculations. Specifically, in an optional embodiment, image data is first read using a full-width and line strip reading method, and invalid pixels (e.g., background pixels with all zeros) are filtered out using a pixel mask. Subsequently, the minimum and maximum values ​​of each band are calculated, and the maximum-minimum normalization method is used to map the effective pixel values ​​of each band to the [0, 1] interval, resulting in a normalized multi-band remote sensing image.

[0026] This embodiment uses a digital image enhancement algorithm to artificially construct a series of layers with different brightness distributions (e.g., simulating underexposure, normal exposure, and overexposure effects) to simulate multiple exposure state layers with different exposure levels. In practical applications, the intensity (or brightness) channel of multi-band remote sensing imagery can be extracted first. Subsequently, a preset nonlinear exponential transform (such as gamma correction) or linear offset model is used to generate multiple exposure state layers.

[0027] Next, each exposure state layer is decomposed into a hierarchical structure containing different frequency features, so that the extraction of fusion features and the allocation of weights can be performed independently at different spatial scales. In one example, a pyramid decomposition algorithm can be used to perform multi-level decomposition operations on each generated exposure state layer.

[0028] Then, for each image subband at each scale level, the mid-gray exposure, image contrast, image saturation, hierarchical modulation features, and cross-band consistency features are determined.

[0029] The steady-state brightness (mid-gray) around 0.5 (normalized brightness) is used as the anchor point, and a Gaussian decay function is used to calculate the mid-gray exposure. This is used to give pixels with moderate exposure (close to mid-gray) greater weight during fusion, thereby highlighting the normal exposure details of the scene.

[0030] In this embodiment, image contrast is used to measure the richness of image texture. For example, at fine-scale levels, the absolute value of the Laplacian image subband can be used, while at coarse-scale levels, the local variance of the Gaussian image subband can be used. Image saturation is used to prevent color dullness. For multi-band remote sensing images, image saturation can be obtained by calculating the local standard deviation of pixel values ​​in each band. The hierarchical modulation feature is a dynamic adjustment factor related to the scale level, and the cross-band consistency feature is used to overcome the problem of color / spectral distortion that easily occurs when a single HDR method is directly applied to remote sensing images.

[0031] In practical calculations, power exponents can be set for contrast, exposure, and saturation to perform nonlinear mapping. Finally, the above feature indicators are jointly multiplied to obtain the initial composite weights, and normalized across multiple exposure state dimensions to ultimately determine the composite weights of the image sub-band at the corresponding pixel positions.

[0032] After obtaining the composite weights, the final target image is output according to the logic of first intra-layer fusion and then cross-layer reconstruction. First, at the same scale level, the Laplacian high-frequency detail images (or the top Gaussian image) corresponding to different exposure states are weighted and summed using the normalized composite weights to complete the intra-layer weighted fusion. Then, starting from the top layer (low-frequency layer), the fusion result is added to the next layer's fusion result through interpolation and amplification.

[0033] Finally, through successive downward reconstruction operations, until the lowest level (i.e. the resolution of the original image) is restored, the final high-detail, high-dynamic-range, and spectrally reliable target remote sensing image is output.

[0034] This invention normalizes multi-band remote sensing images to obtain normalized multi-band remote sensing images. Based on the normalized multi-band remote sensing images, multiple exposure state layers simulating different exposure levels are generated. Each exposure state layer is decomposed into image sub-bands at different scale levels. At each scale level, the composite weight of the image sub-band is determined based on mid-gray exposure, image contrast, image saturation, layer modulation features, and cross-band consistency features. Based on the composite weight, intra-layer weighted fusion is performed on the image sub-bands of different exposure state layers at the same scale level, and layer-by-layer reconstruction is performed from top to bottom according to the different scale levels to obtain the fused target remote sensing image. By simulating multiple exposures with a single image for fusion, the cumbersome process and equipment requirements of traditional high dynamic range (HDR) imaging, which requires multiple shots, are avoided. Furthermore, since all exposure layers come from the same image at the same time, this invention fundamentally eliminates registration errors and dynamic scene ghosting problems in multi-image fusion. The fusion result preserves key information from each exposure, enhancing details in both shadows and highlights, and significantly expanding the overall dynamic range.

[0035] Optionally, in this embodiment, the above step S101 includes, but is not limited to: reading multi-band remote sensing images according to preset row strips, and setting overlapping areas between adjacent row strips.

[0036] In this embodiment, the multi-band remote sensing image is a single-scene multi-band remote sensing image.

[0037] Due to the massive data volume of high-resolution remote sensing images, it is impossible to load them entirely into computer memory at once for multi-scale decomposition and fusion calculations. Therefore, this embodiment adopts a parallel processing method based on stripes and rows. Specifically, the system will process high-resolution remote sensing images... The full-frame image of W is divided into multiple stripes along the height direction for block reading. In practical applications, the number of rows read each time can be controlled by setting a stripe size parameter. This parameter can dynamically adjust adaptively based on the available system memory, for example, set to 1000 rows. When reading the current stripe... At this time, all band data within a strip are read at once using a buffered layout such as BSQ (Band Sequential). To prevent boundary effects caused by subsequent block processing, an overlap area is set between adjacent strips during partitioning. Specifically, overlap and feathering half-width parameters (such as OVERLAP or HALO) can be set so that adjacent upper and lower strips share the image data of the OVERLAP row at the boundary.

[0038] Optionally, in this embodiment, step S102 includes, but is not limited to: S1021, Obtain the intensity channel or brightness channel of the multi-band remote sensing image; Specifically, in this embodiment, since the input remote sensing image typically contains multiple spectral bands, such as the visible red, green, and blue bands or the near-infrared band, in order to uniformly simulate exposure without disrupting the relative spectral proportions between the bands, it is first necessary to extract the global intensity channel (or brightness channel) of the multi-band remote sensing image. In practical implementation, the pixel values ​​of each band can be averaged, or a weighted sum can be performed based on the perception weights of different bands, thereby obtaining the intensity or brightness value representing the overall brightness of the pixel location.

[0039] S1022, perform nonlinear exponential transformation or linear offset transformation on the pixel values ​​of the intensity channel or luminance channel to generate the multiple exposure state layers.

[0040] In this step, multi-exposure layers are artificially constructed using data augmentation techniques through pixel-level enhancement transformations, thus eliminating the reliance of traditional high dynamic range (HDR) imaging on multiple original exposure images. In practical applications, let the total number of generated exposure state layers be... K default K The value ranges from 3 to 7. This applies to the intensity channel. I Specifically, one of the following two transformation methods can be used to generate it. K One exposure state layer I k .

[0041] (1) Using nonlinear exponential transformation In a preferred embodiment, a gamma correction algorithm is used for the intensity channel. I A non-linear mapping is performed to generate simulated underexposed, normally exposed, and overexposed image effects. The specific calculation formula is as follows: (1) in, Indicates the first k An exposure state layer at a pixel p The value at; I(p) This indicates the value of the original intensity channel; The gamma exponent factor is set; clip() The function is used to truncate and restrict the calculation result to a valid numerical range (e.g., [0,1]). In practice, to fully cover the brightness range of the image, a series of different values ​​can be selected. Values, for example {0.6,0.8,1.0,1.25,1.6}.

[0042] (2) Using linear offset transformation In another alternative embodiment, to reduce computational complexity, a linear transformation model can be used to generate different exposure states. The specific calculation formula is as follows: (2) in, This is the linear gain coefficient, used to control the degree of contrast stretching in the image; This is the brightness offset, and it usually satisfies... By assigning different exposure states to different exposure states Parameter combinations can also produce a series of layers with increasing or decreasing brightness distribution.

[0043] Optionally, in this embodiment, step S103 includes, but is not limited to: The exposure state layers are decomposed into multiple layers using Gaussian pyramids and Laplacian pyramids, where the top layer of the pyramid is a low-frequency Gaussian image and the remaining layers are high-frequency Laplacian detail images.

[0044] Specifically, this embodiment uses Gaussian Pyramid and Laplacian Pyramid to perform multi-scale decomposition of the image.

[0045] In practice, this applies to any exposure state layer. I k ,conduct L A pyramid decomposition with multiple layers. The number of layers is also considered. LIt can be customized according to the specific remote sensing band characteristics, with a typical value range of 5 to 7 layers.

[0046] The process of multi-scale decomposition is as follows: First, a Gaussian pyramid is constructed, and the Gaussian image of the s-th layer is obtained through continuous Gaussian low-pass filtering and downsampling. : (3) Then, a Laplacian pyramid is constructed, and high-frequency details are obtained by using the difference between two adjacent Gaussian images to obtain the Laplacian image of the s-th layer. : (4) in, up() This indicates an upsampling operation. After decomposition, the top layer of the pyramid (i.e., L -1 layer is a low-frequency Gaussian image. The first layer retains the basic tone and macroscopic brightness distribution of the image; the remaining layers are Laplacian high-frequency detail images, forming image sub-bands.

[0047] Optionally, in this embodiment, step S104 includes, but is not limited to: obtaining the image contrast, mid-gray exposure, and image saturation of the image sub-band at the current scale level, and applying preset power exponents for contrast weight, exposure weight, and saturation weight to perform exponential adjustments; performing joint multiplication operations on the exponentially adjusted image contrast, mid-gray exposure, and image saturation with the hierarchical modulation feature, the cross-band consistency feature, and the pre-generated pixel mask to obtain an initial composite weight; and normalizing the initial composite weight to determine the composite weight.

[0048] This embodiment integrates multi-dimensional image features, and its initial composite weight calculation model can be expressed as: (5) in, For image contrast, For mid-gray exposure, For image saturation, For hierarchical modulation, For cross-band consistency, M(p) is a pixel mask generated in the preprocessing stage, used to exclude invalid areas; The preset contrast weight power exponent. For the preset exposure weight power index, The preset saturation weight power exponent is used to control the degree of nonlinear influence of each feature index during the fusion process.

[0049] To ensure that the sum of the weights in the subsequent weighted fusion is 1, the above initial composite weights need to be applied to all exposure states. k Normalization is performed on the dimension of: (6) in, To prevent the minimum constant with a denominator of zero.

[0050] In some alternative embodiments, at fine-scale levels (e.g., s≤s) c The absolute value of the Laplacian image subband can be directly obtained. As an indicator of image contrast; at the coarse-scale level, the corresponding Gaussian image can be used. The local variance is used as the image contrast.

[0051] In some alternative embodiments, for multi-band remote sensing images, image saturation can be obtained by calculating the standard deviation of multi-band pixel values ​​within a local area.

[0052] Optionally, in this embodiment, obtaining the mid-gray exposure of the image sub-band at the current scale level includes: determining the difference between the pixel value of the low-frequency image sub-band at the current scale level and the preset normalized mid-gray brightness anchor point; inputting the difference into a preset Gaussian attenuation model for mapping calculation to obtain the mid-gray exposure.

[0053] In this embodiment, a Gaussian attenuation model is established using a normalized brightness of 0.5 as the mid-gray anchor point: (7) in, The standard deviation is typically between 0.18 and 0.22. This model is used to determine the pixel brightness value. The closer the value is to 0.5, the smaller the exponential decay, and the greater the calculated mid-gray exposure, thus highlighting the normal exposure details of the scene.

[0054] Optionally, in this embodiment, before obtaining the image contrast, mid-gray exposure, and image saturation of the image sub-band at the current scale level, the process includes, but is not limited to: obtaining preset intermediate scale level indexes, dark area scale level indexes, and bright area scale level indexes; determining a basic modulation value based on the level deviation between the index value of the current scale level and the intermediate scale level index using a preset attenuation function; constructing an adjustment function based on the dark area scale level index and the bright area scale level index, and correcting the basic modulation value based on the adjustment function to obtain the level modulation feature.

[0055] In this embodiment, a scale hierarchy is introduced. s mid Related hierarchical modulation features This is used to dynamically balance the weight allocation of dark, medium, and bright areas in layers. Specifically, it uses an intermediate scale layer index. s mid Centered on the distance decay function and combined with the adjustment function, Calculate this feature: (8) in, This is a level adjustment parameter, and its value range can be set to [1.0, 1.5]; Dark area scale level index. Highlight Scale Hierarchy Index .

[0056] Optionally, in this embodiment, before obtaining the image contrast, mid-gray exposure, and image saturation of the image sub-band at the current scale level, the method further includes, but is not limited to: summing the pixel values ​​of the normalized multi-band remote sensing image in each band to obtain a cross-band brightness reference value; introducing a preset bias constant into the normalized pixel value of the current band and the cross-band brightness reference value respectively; calculating the ratio of the normalized pixel value of the current band after introducing the bias constant to the cross-band brightness reference value after introducing the bias constant; and exponentially transforming the ratio using a preset consistency adjustment index to obtain the cross-band consistency feature.

[0057] In practical applications, remote sensing images often contain multiple spectral bands. If the weights of each band are calculated independently for fusion, inconsistent fusion ratios can easily occur, leading to severe color and spectral distortion. To overcome this problem, cross-band consistency features are used. Ensure consistency in multispectral image fusion decisions.

[0058] First, the normalized pixel values ​​of the current pixel across all bands are summed to obtain the cross-band brightness reference value. .

[0059] Subsequently, using the aforementioned brightness reference value as an anchor point, the normalized pixel value for the current band is calculated. The ratio of the brightness to the average brightness across the entire band, and the application of the uniformity adjustment index. (generally Perform an exponential transformation: (9) To improve numerical stability, bias constants are introduced into both the numerator and denominator. .

[0060] Finally, once the system obtains the normalized pixel-level composite weights, it can perform multi-scale image fusion and reconstruction (i.e., step S105), which may specifically include: S1051, Intra-layer weighted fusion; (1) For the topmost low-frequency Gaussian image (i.e. s=L-1 layer): The top layer contains the image's base tone, macroscopic brightness distribution, and contrast. The system is based on the composite weights of the top layer. Top Gaussian images under various exposure states We perform weighted fusion to obtain the fused top-level image. The specific formula is as follows: (10) (2) For the Laplacian high-frequency detail images of the remaining layers (i.e. s=0,...,L-2 layer): Specifically, the remaining layers contain high-frequency details such as image edges and textures. This is based on composite weights corresponding to the respective layers. Laplacian images under various exposure states Weighted fusion is performed to obtain the fused detailed images of each layer. The specific calculation formula is as follows: (11) In this embodiment, through the above-mentioned layered fusion method, the algorithm can smoothly mix the brightness information of different exposures at a coarse scale, while sharply preserving the clearest texture features under each exposure state at a fine scale, thereby effectively avoiding the color banding and abrupt seams that are easily generated by direct pixel-level fusion.

[0061] S1052, reconstructed layer by layer from top to bottom.

[0062] Specifically, starting from the top level (i.e., the coarsest scale level), the reconstruction results of the previous level are upsampled (interpolated and amplified). up (), and then merge it with the Laplacian detail image of the current level. F s Perform pixel-by-pixel addition. The iterative reconstruction formula is: (12) In the initial stage of the iteration F L-1 This refers to the top-level Gaussian image obtained after fusion. Through successive downward reconstruction operations, until the lowest level is restored (i.e., ... s=0 (Layers), to obtain the final full-resolution fused image. .

[0063] In a preferred embodiment, to further improve the visual clarity of the remote sensing image, edge-limited sharpening processing is applied to the fused target remote sensing image. Specifically, a sharpening coefficient can be introduced. sharp_cof[b] (The value ranges from 0.2 to 0.4), and sets a numerical upper limit for the gradient amplification operation during the sharpening process.

[0064] As a preferred embodiment, after performing step S1052, the following steps may also be performed, including but not limited to: when outputting the high dynamic range remote sensing image, using a smooth window function to blend the fusion results of adjacent strips in the overlapping area to eliminate abrupt boundary splicing.

[0065] This will yield independent local reconstruction results for each strip. To seamlessly stitch these local results into the final high dynamic range remote sensing image, a pre-defined smoothing window function is used to weighted mix the overlapping portions of adjacent (upper and lower) strips within the overlapping area.

[0066] As a preferred implementation of the smoothing window function, this embodiment uses a cosine window for transition mixing. Its weight allocation formula can be expressed as follows: (13) in, N The total number of rows in the overlapping area. n This is a local row index within the overlapping area.

[0067] Using the smoothing window function described above, the contribution weight of the upper strip at the bottom of the overlapping region gradually decreases smoothly, while the contribution weight of the lower strip at the top of the overlapping region gradually increases smoothly.

[0068] To further verify the effectiveness of the method in this embodiment, comparative experiments were conducted with existing methods in several typical remote sensing scenarios (such as...). Figures 2a to 4c Table 1 lists the evaluation metrics of the fusion results of this method and the cortical vision theory and image enhancement Retinex method for test areas such as urban built-up areas, mountainous forest areas, and river-covered areas. These metrics include color difference ΔE, peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and runtime. It can be seen that the ΔE metrics of this embodiment are generally lower than those of the Retinex method in various scenarios, indicating smaller color restoration errors and closer resemblance to natural colors. The SSIM metrics are generally higher, indicating better structural fidelity of the fused image. The PSNR metrics are close to those of traditional methods. Furthermore, due to the use of parallel acceleration, the runtime of this embodiment is close to or better than that of the Retinex algorithm under the same hardware environment. These quantitative results demonstrate that the remote sensing image fusion method in this embodiment has advantages in preserving image details and true colors, and also exhibits high fusion efficiency. In summary, the remote sensing image fusion method in this embodiment can robustly generate high-quality fused images in complex and varied remote sensing scenarios, providing more reliable high dynamic range data support for subsequent target recognition and information extraction.

[0069] Table 1. Comparison of performance indicators for different fusion methods in typical scenarios (Retinex algorithm vs. this embodiment)

[0070] According to a second aspect of the present invention, a remote sensing image fusion apparatus is provided, such as... Figure 5 As shown, the device may specifically include: The preprocessing module 50 is used to normalize the multi-band remote sensing image to obtain a normalized multi-band remote sensing image. Exposure state construction module 52 is used to generate multiple exposure state layers simulating different exposure levels based on the normalized multi-band remote sensing image; The decomposition module 54 is used to perform multi-scale decomposition on each of the exposure state layers to obtain image subbands at different scale levels; The determination module 56 is used to determine the composite weight of the image sub-band at each scale level based on mid-gray exposure, image contrast, image saturation, hierarchical modulation features, and cross-band consistency features. The fusion module 58 is used to perform intra-layer weighted fusion of image subbands of different exposure states at the same scale level based on the composite weight, and to perform layer-by-layer reconstruction from top to bottom according to the different scale levels to obtain the fused target remote sensing image.

[0071] Optionally, in this embodiment, the exposure state construction module 52 includes: The first acquisition submodule is used to acquire the intensity channel or brightness channel of the multi-band remote sensing image; The first processing submodule is used to perform nonlinear exponential transformation or linear offset transformation on the pixel values ​​of the intensity channel or luminance channel to generate the multiple exposure state layers.

[0072] Optionally, in this embodiment, the decomposition module 54 includes: The decomposition submodule is used to decompose each of the exposure state layers into multiple layers using a Gaussian pyramid and a Laplacian pyramid, wherein the top layer of the pyramid is a low-frequency Gaussian image, and the remaining layers are high-frequency Laplacian detail images.

[0073] Optionally, in this embodiment, the determining module 56 includes: The second acquisition submodule is used to acquire the image contrast, the mid-gray exposure and the image saturation of the image subband at the current scale level, and to apply preset contrast weight power index, exposure weight power index and saturation weight power index to perform exponential adjustment respectively. The second processing submodule is used to perform a joint product operation on the exponentially adjusted image contrast, mid-gray exposure and image saturation, the hierarchical modulation feature, the cross-band consistency feature and the pre-generated pixel mask to obtain the initial composite weight. The third processing submodule is used to normalize the initial composite weights in order to determine the composite weights.

[0074] Optionally, in this embodiment, the second acquisition submodule includes: The first determining unit is used to determine the difference between the pixel value of the low-frequency image sub-band at the current scale level and the preset normalized gray-brightness anchor point. The first calculation unit is used to input the difference into a preset Gaussian attenuation model for mapping calculation to obtain the mid-gray exposure.

[0075] Optionally, in this embodiment, it further includes: The third acquisition submodule is used to acquire preset intermediate scale level index, dark area scale level index and bright area scale level index before acquiring the image contrast, mid-gray exposure and image saturation of the image subband at the current scale level. The first determining submodule is used to determine the basic modulation value based on the hierarchical deviation between the index value of the current scale level and the index of the intermediate scale level, using a preset attenuation function. The fourth processing submodule is used to construct an adjustment function based on the dark area scale level index and the bright area scale level index, and to correct the basic modulation value based on the adjustment function to obtain the level modulation feature.

[0076] Optionally, in this embodiment, it further includes: The second determining submodule is used to sum the pixel values ​​of the normalized multi-band remote sensing image in each band before obtaining the image contrast, mid-gray exposure and image saturation of the image sub-band at the current scale level, so as to obtain cross-band brightness reference values. The fifth processing submodule is used to introduce preset bias constants into the normalized pixel value of the current band and the cross-band brightness reference value, respectively. The third determination submodule is used to calculate the ratio of the current band normalized pixel value after introducing the bias constant to the cross-band brightness reference value after introducing the bias constant; The sixth processing submodule is used to perform an exponential transformation on the ratio using a preset consistency adjustment index to obtain the cross-band consistency characteristics.

[0077] Another embodiment of the present invention relates to an electronic device, such as Figure 6As shown, it includes: at least one processor 601; and a memory 602 communicatively connected to the at least one processor 601; wherein the memory 602 stores instructions executable by the at least one processor 601, the instructions being executed by the at least one processor 601 to enable the at least one processor 601 to perform the remote sensing image fusion method in the above embodiments.

[0078] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0079] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0080] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0081] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A remote sensing image fusion method, characterized in that, include: The multi-band remote sensing images are normalized to obtain normalized multi-band remote sensing images. Based on the normalized multi-band remote sensing image, multiple exposure state layers simulating different exposure levels are generated. Each of the exposure state layers is decomposed into multiple scales to obtain image subbands at different scale levels; At each scale level, the composite weight of the image sub-band is determined based on mid-gray exposure, image contrast, image saturation, hierarchical modulation features, and cross-band consistency features. Based on the composite weights, image sub-bands of different exposure states at the same scale are fused intra-layer by weight, and reconstructed layer by layer from top to bottom according to the different scale levels to obtain the fused target remote sensing image.

2. The method according to claim 1, characterized in that, The process of generating multiple exposure state layers simulating different exposure levels based on the normalized multi-band remote sensing imagery includes: Obtain the intensity channel or brightness channel of the multi-band remote sensing image; The pixel values ​​of the intensity channel or luminance channel are subjected to a nonlinear exponential transformation or a linear offset transformation to generate the multiple exposure state layers.

3. The method according to claim 1, characterized in that, The step of performing multi-scale decomposition on each of the exposure state layers to obtain image subbands at different scale levels includes: The exposure state layers are decomposed into multiple layers using Gaussian pyramids and Laplacian pyramids, where the top layer of the pyramid is a low-frequency Gaussian image and the remaining layers are high-frequency Laplacian detail images.

4. The method according to claim 1, characterized in that, At each scale level, the composite weights of the image sub-bands are determined based on mid-gray exposure, image contrast, image saturation, hierarchical modulation features, and cross-band consistency features, including: Obtain the image contrast, mid-gray exposure, and image saturation of the image sub-band at the current scale level, and apply preset power exponents for contrast weight, exposure weight, and saturation weight to adjust the exponents respectively. The image contrast, mid-gray exposure, and image saturation after exponential adjustment are combined with the hierarchical modulation features, the cross-band consistency features, and the pre-generated pixel mask to obtain the initial composite weights. The initial composite weights are normalized to determine the composite weights.

5. The method according to claim 4, characterized in that, The step of obtaining the mid-gray exposure of the image sub-band at the current scale level includes: Determine the difference between the pixel value of the low-frequency image sub-band at the current scale level and the preset normalized gray-brightness anchor point; The difference is input into a preset Gaussian attenuation model for mapping calculation to obtain the mid-gray exposure.

6. The method according to claim 4, characterized in that, Before obtaining the image contrast, mid-gray exposure, and image saturation of the image subband at the current scale level, the method further includes: Obtain the preset intermediate scale level index, dark area scale level index, and bright area scale level index; Based on the hierarchical deviation between the index value of the current scale level and the index of the intermediate scale level, the basic modulation value is determined by a preset attenuation function. An adjustment function is constructed based on the dark area scale level index and the bright area scale level index, and the basic modulation value is corrected based on the adjustment function to obtain the level modulation feature.

7. The method according to claim 4, characterized in that, Before obtaining the image contrast, mid-gray exposure, and image saturation of the image subband at the current scale level, the method further includes: The pixel values ​​in each band of the normalized multi-band remote sensing image are summed to obtain cross-band brightness reference values. A preset bias constant is introduced into the normalized pixel value of the current band and the cross-band brightness reference value, respectively; Calculate the ratio of the current band normalized pixel value after introducing the bias constant to the cross-band brightness reference value after introducing the bias constant; The cross-band consistency characteristics are obtained by exponentially transforming the ratio using a preset consistency adjustment index.

8. A remote sensing image fusion device, characterized in that, include: The preprocessing module is used to normalize multi-band remote sensing images to obtain normalized multi-band remote sensing images. The exposure state construction module is used to generate multiple exposure state layers simulating different exposure levels based on the normalized multi-band remote sensing image. The decomposition module is used to perform multi-scale decomposition on each of the exposure state layers to obtain image subbands at different scale levels; The determination module is used to determine the composite weight of the image sub-band at each scale level based on mid-gray exposure, image contrast, image saturation, hierarchical modulation features, and cross-band consistency features. The fusion module is used to perform intra-layer weighted fusion of image subbands of different exposure states at the same scale level based on the composite weight, and to perform layer-by-layer reconstruction from top to bottom according to the different scale levels to obtain the fused target remote sensing image.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the remote sensing image fusion method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the remote sensing image fusion method as described in any one of claims 1 to 7.

Images