Remote sensing image fusion method and system based on dynamic region guidance

Abstract

This invention belongs to the field of image fusion technology and discloses a remote sensing image fusion method and system based on dynamic region guidance. The method utilizes structural saliency priors and content-adaptive gating mechanisms to dynamically filter key regions. It combines local region attention, sparse graph relationship enhancement, multi-scale convolutional branching, and a two-dimensional selective fusion strategy to achieve collaborative processing of image patch detail interaction, long-range relationship modeling between key regions, and multi-scale local compensation. Experimental results show that the method achieves good performance on the WV3 dataset. Ablation experiments verify the effectiveness of each component module and key parameter settings. Comparative experiments demonstrate that the method outperforms most existing methods in quantitative indicators, better recovering structural details of areas such as buildings and roads in complex scenes while maintaining good color consistency.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of image fusion technology, and particularly relates to a remote sensing image fusion method and system based on dynamic region guidance. Background Technology

[0002] Remote sensing image fusion is one of the key technologies in the field of remote sensing information processing. Its core objective is to fuse low-spatial-resolution multispectral images with high-spatial-resolution panchromatic images to generate high-resolution multispectral images that simultaneously possess high spatial resolution and multispectral band information. This technology has significant application value in fields such as land resource surveys, ecological environment monitoring, precision agriculture, urban planning, and disaster assessment.

[0003] Currently, mainstream remote sensing image fusion methods can be broadly categorized into traditional methods and deep learning methods. Traditional methods mainly include component substitution, multi-resolution analysis, and variational optimization. While these methods possess clear physical meaning and interpretability, they generally suffer from insufficient spatial detail extraction or severe spectral distortion, and their fusion performance is highly dependent on manually designed prior knowledge. In contrast, deep learning-based methods, with their powerful feature learning and representation capabilities, significantly improve the quality of fused images and have become a current research hotspot.

[0004] However, existing deep learning-based remote sensing image fusion methods still generally suffer from the following technical bottlenecks: On the one hand, most methods adopt a globally uniform feature extraction strategy, failing to fully consider the spatial heterogeneity and structural differences of different regions in the image. The spatial details such as edges and textures contained in panchromatic images exhibit significant spatial imbalances—regions with dramatic changes in ground features contain rich structural information, while flat regions are dominated by spectral information. Existing methods often use the same network parameters and computational paths when processing these two types of regions, resulting in insufficient detail enhancement in structurally rich regions and the potential introduction of unnecessary texture noise in flat regions. On the other hand, existing methods lack effective collaborative mechanisms for information flow between multiple feature processing modules. The regional saliency priors extracted by shallow modules are difficult to effectively transfer and guide the feature enhancement process of deep modules, resulting in redundant allocation of computational resources and limiting the improvement of fusion accuracy.

[0005] Therefore, how to achieve differentiated processing of different regions of an image and effectively share and guide regional saliency priors in the deep structure of the network to further improve the spatial detail fidelity and spectral preservation accuracy of the fused image is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a remote sensing image fusion method and system based on dynamic region guidance.

[0007] This invention is implemented as follows: A remote sensing image fusion method based on dynamic region guidance includes:

[0008] Step 1: The shallow feature extraction module is used to extract shallow features from the combined multispectral and panchromatic images.

[0009] Step 2, the dynamic region information construction module includes a structural saliency estimation unit and a gating prediction unit; wherein, the structural saliency estimation unit generates a structural saliency map based on the gradient information of the panchromatic image, and the gating prediction unit generates a content-adaptive gating map based on the initial fusion features jointly extracted from the multispectral image and the panchromatic image; the structural saliency map is used for key region ranking and Top-K selection, and the content-adaptive gating map is used for key region quantity estimation and enhancement intensity control;

[0010] Step 3: The structure-guided enhancement module executes two branches in parallel around the key region: one branch performs local detail modeling and sparse relation enhancement in sequence, and the other branch performs multi-scale information supplementation. Then, the two branches are selectively fused. Multiple structure-guided enhancement modules share content adaptive gating graphs, Top-K key region index sets, and structure saliency graphs.

[0011] Step 4: Output high-resolution multispectral images through the reconstruction module.

[0012] Furthermore, the extraction and reconstruction of the joint shallow features includes: concatenating the upsampled low-resolution multispectral image with the high-resolution panchromatic image in the channel dimension to obtain joint input features; obtaining initial fusion features through shallow convolution mapping; inputting the initial fusion features into multiple cascaded structure-guided enhancement modules to obtain deep fusion features; generating residual terms through a reconstruction module, and adding the residual terms to the upsampled low-resolution multispectral image to obtain a high-resolution multispectral image.

[0013] Furthermore, the construction of the dynamic region information includes: generating a gradient magnitude map based on the gradient information of the high-resolution panchromatic image in the horizontal and vertical directions, and normalizing the gradient magnitude map to obtain a structural saliency map; generating a content-adaptive gating map based on the initial fusion features through a gating prediction module; wherein, the structural saliency map is used to characterize the intensity of structural changes in the boundary areas of edges, textures and ground features, and the content-adaptive gating map is used to characterize the enhancement requirements of different spatial locations.

[0014] Furthermore, the generation of the Top-K key region index set includes: mapping the structural saliency map and the content adaptive gating map to the image patch scale to obtain the image patch-level structural saliency representation and the image patch-level gating representation; adaptively estimating the number of key regions based on the image patch-level gating representation; and selecting the corresponding number of image patches from high to low based on the image patch-level structural saliency representation to obtain the Top-K key region index set.

[0015] Furthermore, the structure-guided enhancement module includes a local region detail modeling branch, a multi-scale information supplementation branch, a sparse global relation enhancement unit, and a selective fusion unit; wherein, the local region detail modeling branch is used to model the spatial dependence within the key region, the sparse global relation enhancement unit is used to perform cross-regional relation reasoning on the Top-K key regions, the multi-scale information supplementation branch is used to extract local contextual information under different receptive fields, and the selective fusion unit is used to fuse sparse relation enhancement features and multi-scale supplementary features.

[0016] Furthermore, the sparse global relationship enhancement unit selects node features corresponding to key image blocks based on the Top-K key region index set, and constructs similarity relationships between key nodes and anchors through learnable anchors, so that the key node features are aggregated and backpropagated in the anchor space to generate global relationship enhancement features; the selective fusion unit adaptively fuses the global relationship enhancement features with multi-scale supplementary features, and controls the fusion intensity in combination with the content adaptive gating graph.

[0017] Another object of the present invention is to provide a remote sensing image fusion system based on dynamic region guidance, comprising:

[0018] The shallow feature extraction module is used to extract shallow features from the combined multispectral and panchromatic images.

[0019] The dynamic region information construction module includes a structural saliency estimation unit and a gating prediction unit. The structural saliency estimation unit generates a structural saliency map based on gradient information from a high-resolution panchromatic image, while the gating prediction unit generates a content-adaptive gating map based on initial fusion features jointly extracted from the multispectral and panchromatic images. The structural saliency map is used for key region ranking and Top-K selection, while the content-adaptive gating map is used for key region quantity estimation and enhancement intensity control.

[0020] The structure-guided enhancement module is used to execute two branches in parallel around the key region: one branch performs local detail modeling and sparse relation enhancement in sequence; the other branch performs multi-scale information supplementation; then the two branches are selectively fused, and the structure-guided enhancement modules share content adaptive gating, the Top-K key region index set and the structure saliency map;

[0021] The reconstruction module is used to output high-resolution multispectral images.

[0022] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the remote sensing image fusion method based on dynamic region guidance.

[0023] Another object of the present invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the remote sensing image fusion method based on dynamic region guidance.

[0024] Another objective of the present invention is to provide an information data processing terminal for implementing the remote sensing image fusion system based on dynamic region guidance.

[0025] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0026] This invention addresses the challenge of varying structural complexity across different regions in remote sensing images, where uniform processing methods struggle to balance detail restoration and spectral preservation. It proposes a dynamic region-guided remote sensing image fusion method. This method utilizes structural saliency priors and content-adaptive gating mechanisms to dynamically select key regions. It combines local region attention, sparse graph relationship enhancement, multi-scale convolutional branching, and a two-dimensional selective fusion strategy to achieve collaborative processing of detail interaction within image patches, long-range relationship modeling between key regions, and multi-scale local compensation. Experimental results demonstrate that the proposed method achieves superior performance on the WV3 dataset. Ablation experiments validate the effectiveness of the component modules and key parameter settings. Comparative experiments show that the proposed method outperforms most existing methods in both quantitative metrics and visualization, better restoring structural details of areas such as buildings and roads in complex scenes while maintaining good color consistency. Overall, this invention provides an effective approach for remote sensing image fusion addressing complex regional differences. Attached Figure Description

[0027] Figure 1 This is a flowchart of a remote sensing image fusion method based on dynamic region guidance provided in an embodiment of the present invention.

[0028] Figure 2 This is a block diagram of a remote sensing image fusion system based on dynamic region guidance provided in an embodiment of the present invention.

[0029] Figure 3 This is a diagram of the SGSRN network structure provided in an embodiment of the present invention.

[0030] Figure 4 This is a structural guidance enhancement module architecture diagram provided in an embodiment of the present invention.

[0031] Figure 5 This is a schematic diagram of the attention in the gating region provided in an embodiment of the present invention.

[0032] Figure 6 This is a schematic diagram of the multi-scale sensing module provided in an embodiment of the present invention.

[0033] Figure 7 This is a schematic diagram of anchor point graph convolution provided in an embodiment of the present invention.

[0034] Figure 8 This is a diagram of the two-dimensional selection fusion module provided in an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0036] like Figure 1 As shown, an embodiment of the present invention provides a remote sensing image fusion method based on dynamic region guidance, comprising the following steps:

[0037] S101, the shallow feature extraction module is used to complete the joint shallow feature extraction of multispectral and panchromatic images;

[0038] S102, the dynamic region information construction module includes a structural saliency estimation unit and a gating prediction unit; wherein, the structural saliency estimation unit generates a structural saliency map based on the gradient information of the panchromatic image, and the gating prediction unit generates a content-adaptive gating map based on the initial fusion features jointly extracted from the multispectral image and the panchromatic image; the structural saliency map is used for key region ranking and Top-K selection, and the content-adaptive gating map is used for key region quantity estimation and enhancement intensity control;

[0039] S103, the structure-guided enhancement module executes two branches in parallel around the key region: one branch performs local detail modeling and sparse relation enhancement in sequence, and the other branch performs multi-scale information supplementation. Then, the two branches are selectively fused. Multiple structure-guided enhancement modules share content adaptive gating graphs, Top-K key region index sets, and structure saliency graphs.

[0040] S104 outputs high-resolution multispectral images through the reconstruction module.

[0041] like Figure 2As shown, an embodiment of the present invention provides a remote sensing image fusion system based on dynamic region guidance, comprising:

[0042] The shallow feature extraction module is used to extract shallow features from the combined multispectral and panchromatic images.

[0043] The dynamic region information construction module includes a structural saliency estimation unit and a gating prediction unit. The structural saliency estimation unit generates a structural saliency map based on gradient information from the panchromatic image, while the gating prediction unit generates a content-adaptive gating map based on initial fusion features jointly extracted from the multispectral and panchromatic images. The structural saliency map is used for key region ranking and Top-K selection, while the content-adaptive gating map is used for key region quantity estimation and enhancement intensity control.

[0044] The structure-guided enhancement module is used to execute two branches in parallel around the key region: one branch performs local detail modeling and sparse relation enhancement in sequence; the other branch performs multi-scale information supplementation; then the two branches are selectively fused, and the structure-guided enhancement modules share content adaptive gating, the Top-K key region index set and the structure saliency map;

[0045] The reconstruction module is used to output high-resolution multispectral images.

[0046] Another objective of this invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the remote sensing image fusion method based on dynamic region guidance.

[0047] Another objective of this invention is to provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the remote sensing image fusion method based on dynamic region guidance.

[0048] Another objective of this invention is to provide an information data processing terminal for implementing the remote sensing image fusion system based on dynamic region guidance.

[0049] Specific implementation of the present invention:

[0050] Overall framework

[0051] To address the aforementioned issues, this invention proposes a Structure-Guided Sparse Reasoning Network (SGSRN) based on dynamic regions. The core idea of ​​this method is to utilize the rich spatial information in panchromatic images to dynamically identify regions worthy of enhancement, and then perform targeted feature modeling and relation enhancement around these key regions. The SGSRN network first extracts structural saliency priors from the panchromatic image to describe complex regions. Simultaneously, it adaptively adjusts the enhancement range and intensity by combining gating information learned by the network, enabling the model to perform differentiated processing based on image content differences. Building upon this, this invention further combines local region attention, multi-scale convolution, and sparse global relation extraction, allowing the network to both enhance detail recovery within key regions and establish long-range structural connections between key regions. Furthermore, it optimizes spatial detail injection and spectral preservation through two-dimensional selective fusion.

[0052] In terms of the overall framework, Figure 3 The diagram shows the proposed SGSRN network structure. The proposed network consists of four parts: shallow feature extraction, dynamic region information construction, stacked structure-guided enhancement modules, and reconstruction. The shallow feature extraction part is used to obtain joint shallow features from multispectral and panchromatic images. The dynamic region information construction part uses panchromatic image gradient information and fused features to generate a structure saliency map and a content-adaptive gating map, respectively, providing a basis for key region selection and enhancement intensity control. The core structure-guided enhancement module executes two branches in parallel around the key regions: one branch sequentially completes local detail modeling and sparse relation enhancement, and the other branch completes multi-scale information supplementation, followed by selective fusion to obtain enhanced features. The structure-guided enhancement modules share the content-adaptive gating map, the Top-K key region index set, and the structure saliency map.

[0053] Let the upsampled low-resolution multispectral image be .in, Indicates the number of spectral channels. and These represent the height and width of the image, respectively, and their corresponding high-resolution panchromatic images are represented as follows: The details of the network are as follows:

[0054] First, the upsampled multispectral image and the panchromatic image are concatenated along the channel dimension to form joint input features:

[0055] (1)

[0056] in, This indicates a channel concatenation operation. Subsequently, a convolutional layer is used to extract initial shallow features:

[0057] (2)

[0058] in, This represents the shallow convolution mapping function.

[0059] After shallow feature extraction, the network further constructs prior information required for dynamic region modeling from the input image and fused features, and uses this information to drive the subsequent fusion process. The backbone network consists of... It consists of cascaded Structure-Guided Enhancement Blocks (SGEBs), assuming the first one is... The input and output features of each SGEB are respectively and Then its recursive form can be expressed as:

[0060] , (3)

[0061] in, Indicates the first Each SGEB module has a content-adaptive gating graph. The structural saliency diagram is as follows The index of the key image patch obtained by dynamic selection is Within the block, local detail modeling, sparse global relationship enhancement, and feature fusion will be completed by combining regional priors.

[0062] After After completing the SGEB modules, the final deep fusion features are obtained. Subsequently, the residual terms are generated using the tail reconstruction module:

[0063] (4)

[0064] in, This represents the tail reconstruction module. The final fusion result is obtained through residual connections:

[0065] (5)

[0066] in, This is the final output high-resolution multispectral fusion image.

[0067] SGSRN first extracts shallow fusion features, then sequentially completes key region selection, local detail modeling, sparse relation enhancement and adaptive fusion in multiple SGEBs, and finally obtains the output through residual reconstruction.

[0068] Dynamic region selection and gating mechanism

[0069] Since the structural complexity and computational requirements of different regions are usually inconsistent, image regions need to be screened during the fusion process so that the model can prioritize structurally complex locations and apply differentiated processing to different regions. A dynamic region selection and gating mechanism is designed in SGSRN, which achieves key region selection and enhancement intensity control by constructing structural saliency maps and content-adaptive gating maps respectively. Considering that panchromatic images contain rich spatial structural information, the network uses the magnitude of their gradient values ​​to describe the structural complexity of different locations. Let the input panchromatic image be... Then its gradient response in the horizontal and vertical directions can be expressed as follows:

[0070] (6)

[0071] (7)

[0072] in, This represents the convolution operation. and These represent the convolution kernels used to extract gradient information in the horizontal and vertical directions, respectively. In this embodiment of the invention, the Sobel operator is used to calculate the gradient of the panchromatic image. Further, a gradient magnitude map can be obtained. To unify the numerical ranges across different samples, normalization is performed:

[0073] (8)

[0074] in, This represents a structural saliency map. This map measures the importance of a spatial location; a higher value indicates that the vicinity of that location typically contains stronger edge or texture variations.

[0075] In addition to assessing the importance of regions, the network also needs to estimate the intensity of augmentation demand at different locations. To this end, this method learns to generate a content-adaptive gating graph from the initial fusion features. Let the initial fusion features extracted from the shallow layer be... The gating prediction process can then be expressed as:

[0076] (9)

[0077] (10)

[0078] in, The term "gated prediction module" refers to a module that takes initial fused features as input, extracts discriminative spatial responses step by step through convolutional mapping and spatial attention, and then maps the features into a single-channel gated representation. express function, This represents a temperature coefficient used to adjust the smoothness of the gating graph. From this, an initial content-adaptive gating graph can be obtained. The content-adaptive gating graph directly participates in the control of subsequent dynamic region quantity estimation and feature enhancement; its value can be understood as the intensity of enhancement at different locations in the network.

[0079] This invention maps structural saliency maps and content-adaptive gating maps to image patch scales for subsequent region selection. Assume the size of the image patch is... The corresponding image patch grid size is , The total number of image patches is Average pooling can be used to map content-adaptive gating maps and structural saliency maps to a patch-level image representation.

[0080] (11)

[0081] (12)

[0082] in, Indicates step size is Average pooling. Used to evaluate the structural importance of each image patch. Used to estimate the number of regions in the current sample that require focused modeling.

[0083] Since different input images differ in spatial structure, the number of regions that need to be modeled in the network should not be fixed. This method considers image patches with significant structure as key regions, which will be further modeled subsequently. Considering that the number of these regions will vary with the input content, this embodiment of the invention uses an adaptive approach to estimate their number. First, the image patch-level gating map is... Flattened into a vector For each sample, the dynamic threshold is defined as:

[0084] (13)

[0085] in, and These represent the mean and standard deviation, respectively. This is the control coefficient. Next, the number of image patches exceeding the threshold is counted:

[0086] (14)

[0087] To ensure training stability and prevent excessive noise in the data, the algorithm imposes maximum and minimum value constraints. The final dynamically selected number can be written as:

[0088] (15)

[0089] in, and These represent the minimum and maximum allowed number of regions, respectively. In this way, the network can adaptively adjust the degree of subsequent sparse global modeling based on the current input image.

[0090] After obtaining the number of dynamic regions Subsequently, embodiments of the present invention are based on structural saliency diagrams. The size of each element value, selected from largest to smallest. From image patches, obtain the key region index set.

[0091] (16)

[0092] Therefore, this embodiment of the invention achieves dynamic screening of key regions. It should be noted that in this method, the structural saliency map and the content-adaptive gating map perform different functions. The structural saliency map is calculated using gradient priors and is used to characterize the edge and texture variation intensity of each image patch, serving as the basis for key region ranking and Top-K selection. The gating map, on the other hand, is learned from the initial fused features and is mainly used to estimate the number of regions in the current sample that require focused modeling, and to control the enhancement magnitude in subsequent feature enhancement stages. Therefore, the dynamic region selection in this embodiment of the invention considers both structural priors and content responses, where the structural saliency map determines the location of the key modeling regions, and the content-adaptive gating map determines the region selection scale and enhancement intensity.

[0093] Structure-guided enhancement module

[0094] Dynamic region selection provides the network with location priors for key modeling, but how to further organize local detail restoration and global structure compensation around these regions remains key to improving fusion quality. To address this, this invention proposes a structure-guided enhancement module that models local spatial dependencies within key regions and introduces sparse relational reasoning between regions, thereby achieving synergistic optimization of local detail enhancement and global structural consistency modeling. This module comprises four steps: local region modeling, sparse global relational reasoning, multi-scale feature extraction, and bi-branch fusion.

[0095] Let the first The input features of each structure-guided enhancement module are Content-adaptive gating graph is The structural saliency diagram is as follows The index of the key image patch obtained by dynamic selection is Therefore, the overall process of this module can be summarized as follows:

[0096] (17)

[0097] in, This indicates a structure-guided enhancement module.

[0098] like Figure 4 As shown, this module mainly consists of four stages. First, it uses gated region attention to model local regions of the input features to enhance the spatial dependency representation within key regions. Then, it maps the locally enhanced 2D features to image patch-level representations and performs sparse graph relation inference only on selected key regions to obtain cross-regional structural compensation information. Next, it applies the relation enhancement results back to the spatial features to further update the local enhanced representations. Finally, it combines the parallel multi-scale perception module branches and the dual-branch adaptive fusion module to generate the output features of the current structure-guided enhancement module. Its internal flow is shown in the figure and can be written as follows:

[0099] (18)

[0100] (19)

[0101] (20)

[0102] (twenty one)

[0103] in This indicates detailed modeling of a local area. This represents structure-guided sparse relations. This indicates a multi-scale sensing module. This indicates a two-dimensional selection module.

[0104] Local region detail modeling and multi-scale representation

[0105] like Figure 5 As shown, relying solely on conventional convolution operations is insufficient to fully characterize the dependencies between different locations within an image patch. Therefore, gated region attention is introduced into the structure-guided enhancement module. By establishing correlations between pixels within a local image patch, explicit modeling of detailed interactions within key regions is performed. Furthermore, the enhancement result is adaptively modulated using a gated map, thereby improving the expressive power of local structures.

[0106] Let the input features be First, according to size The features are divided into non-overlapping image patches, and then self-attention calculation is performed on the pixels within the image patch by querying Q, key K, and value V. For the first... For a given local region, the regional attention can be represented as:

[0107] (twenty two)

[0108] in and They represent the first Query and key features within an image patch This describes the correlation between pixels within the local region. Based on the attention matrix, the results of local feature relationships can be obtained:

[0109] (twenty three)

[0110] in, This represents the value features of the corresponding image patch.

[0111] After restoring the enhancement results of all regions to their original spatial size, a content-adaptive gating map is further used to modulate them position-by-position, so that local detail enhancement is more focused on regions with higher enhancement needs. The corresponding process can be written as:

[0112] (twenty four)

[0113] in, For the content-adaptive gating graph generated in the previous section, This represents element-wise multiplication. Finally, the output features after local region detail modeling are obtained through linear mapping and residual connections:

[0114] (25)

[0115] Furthermore, in order to further supplement the multi-scale contextual information in the local continuous space, this embodiment of the invention introduces a multi-scale convolution branch in parallel with the region attention branch. Figure 6 This demonstrates the specific implementation process of the multi-scale perception module. Given an input feature map, the module first extracts local responses under different receptive fields through four parallel convolutional branches, with the convolutional kernel sizes being [sizes to be filled in]. , , and The smaller-scale branches primarily capture fine-grained textures and local high-frequency variations, while the larger-scale branches focus more on wide edges, structural contours, and neighborhood context. Subsequently, the convolutional results at each scale are rearranged into groups after passing through an activation function, and then progressively concatenated within each group to construct a joint feature representation containing information from multiple scales. Next, the recombined multi-scale features are mapped back to a unified tensor and grouped... Convolution performs intra-scale aggregation and channel compression, and then normalization and nonlinear activation are combined to further coordinate responses at different scales. Finally, tails are utilized. Convolution performs a unified mapping on the aggregation results, outputting the final multi-scale detailed feature map.

[0116] Sparse global relation enhancement and feature fusion

[0117] While gating region attention can enhance the interaction of details within image patches, relying solely on local modeling is insufficient to guarantee structural consistency between different key regions. For remote sensing images, features such as contours and textures often exhibit cross-regional correlations, thus necessitating the establishment of long-range dependencies between key regions. To address this, this invention further designs an AnchorGraph Convolutional Network (AnchorGCN) and a Dual-Dimension Selective Fusion (DDSF) module. This module performs global relationship modeling only on dynamically selected key regions and combines conditional modulation and multi-scale feature fusion to further enhance spatial details.

[0118] like Figure 7 As shown, firstly, the locally enhanced features are mapped to image block-level node representations, and then the key region index set obtained in the previous section is used. Before extraction Several key nodes form a sparse node set:

[0119] (26)

[0120] in, This represents the node features of the entire image at the image patch scale. This represents the key node features involved in global relational reasoning. The purpose of this is to abstract regional features into point features, thereby improving the relevance and efficiency of global modeling.

[0121] Building upon this, to establish long-range dependencies between key regions while controlling computational overhead, a set of learnable anchor points is introduced as shared intermediate representations. This way, different key nodes do not need to interact directly pairwise; instead, information is first aggregated into the anchor point space, and then the anchor point representations are fed back to the key nodes, thus achieving cross-regional information propagation and structural relationship modeling with lower complexity. The features of the obtained key nodes... Subsequently, embodiments of the present invention further introduce There are learnable anchor points used to construct sparse relationship propagation channels between key regions. Let the learnable anchor point matrix be... ,in Indicates the number of anchor points. This represents the node feature dimension. Based on the feature similarity between key nodes and anchor points, a similarity matrix can be calculated between them:

[0122] (27)

[0123] in, and These represent the linear mappings between anchor points and key nodes, respectively. Based on the similarity matrix... We can further construct representations from key nodes to anchor points:

[0124] (28)

[0125] in, This represents the anchor points obtained by aggregating key regions. This process corresponds to feature aggregation from key regions to anchor points. After obtaining the anchor point representation, this embodiment of the invention further learns the anchor point correlation. First, the anchor points obtained by aggregating key regions are added to the learnable anchor point priors, and then mapped through a lightweight linear transformation module to obtain the enhanced anchor point features. Specifically, [the following is omitted as it is not directly related to the previous sentence]. With learnable anchor prior By adding a stable global prior based on the current sample adaptive aggregation result, the enhanced anchor features can simultaneously retain content adaptability and cross-sample consistency.

[0126] (29)

[0127] in, This represents a feature transformation module consisting of linear layers and nonlinear activations.

[0128] Contextual information, originally scattered across different key nodes, can first interact within the anchor space to form a more globally consistent relational representation. Subsequently, the transpose of the similarity matrix is ​​used to construct the backpropagation matrix from the anchor points to the key nodes. The enhanced anchor features are then remapped back to the key node space to obtain the relationship enhancement result:

[0129] (30)

[0130] To highlight the contribution of structurally significant regions, this embodiment of the invention further fuses the relationship enhancement features after graph inference with the key node features before graph inference, and weights them in conjunction with structural scores:

[0131] (31)

[0132] in, This represents the weights of key regions generated from the structural saliency map. This design allows regions with stronger structural responses to have higher compensation strength in global relation enhancement.

[0133] like Figure 8 As shown, sparse graph inference yields image block-level compensation information. This requires restoring the key node indexes to the complete node set and restoring the original spatial resolution to form a global compensation feature ΔF. Unlike direct addition updates, this embodiment of the invention uses modulation to inject global compensation information into local features. Specifically, this is achieved through linear mapping from... The scaling and bias terms are generated and applied to local features:

[0134] (32)

[0135] in, and These represent the global compensation feature ΔF passing through a... The convolutional mapping is generated. This mapping first projects ΔF onto two sets of responses with the same number of channels as the local features, then partitions them into scaling and bias terms along the channel dimension, and finally... The function constrains its range of values. The former is used to adjust the response intensity of local features, while the latter is used to provide additional translation compensation. The learnable coefficient.

[0136] Furthermore, by utilizing a two-dimensional selection fusion module, relationship enhancement features are applied. With multi-scale convolutional features Perform adaptive fusion. The fusion result can be represented as follows:

[0137] (33)

[0138] in and These represent the combined weights for the two branches.

[0139] Finally, to make the enhancement results more in line with the detail restoration needs of different regions, the intensity of the fused features is controlled again using a content-adaptive gating graph to obtain the output of the current structure-guided enhancement block:

[0140] (34)

[0141] In summary, the sparse global relation enhancement and feature fusion module achieves coordinated optimization of local detail modeling and global structural consistency modeling through steps such as key region extraction, sparse graph relation inference, conditional modulation, multi-scale compensation, and gated fusion. Specifically, sparse graph inference establishes long-distance dependencies between key regions, multi-scale convolutional branches supplement continuous local statistical information, and the gating mechanism adaptively constrains the enhancement results. This effectively suppresses spectral distortion caused by over-enhancement while improving spatial detail recovery capabilities.

[0142] ablation experiment

[0143] To verify the effectiveness of each component module, this embodiment of the invention conducted ablation verification on the gated region attention module, multi-scale perception module, anchor map convolution module, and two-dimensional selection fusion module. Experimental results show that removing any of the above modules leads to varying degrees of decline in spatial detail recovery, spectral preservation, and overall evaluation metrics of the fused image, indicating that each module can synergistically improve the quality of remote sensing image fusion. Specifically, the multi-scale perception module helps supplement local details under different receptive fields, the anchor map convolution module helps establish long-range relationships between key regions, the gated region attention enhances the internal structural expression of key regions, and the two-dimensional selection fusion module coordinates the information contribution of relationship enhancement branches and multi-scale branches. Further experiments show that the number of structure-guided enhancement modules, the image patch division scale, the Top-K selection range, and the number of anchor points affect the fusion effect; this embodiment optimally uses three structure-guided enhancement modules, an image patch size of 8, a Top-K dynamic range of 0.10–0.50, and 64 anchor points.

[0144] Quantitative index analysis

[0145] In the comparison method, this embodiment of the invention selected nine methods to compare with the proposed SGSRN method, which were divided into quantitative index comparisons. The methods compared include traditional methods AWLP, BDSD-PC, and MTF-GLP, as well as deep learning-based methods PanNet, FusionNet, ADKNet, LACNet, DLSKNet, and PanFormer.

[0146] Table 1 Quantitative Indicators on the WV3 Dataset

[0147]

[0148] As shown in Table 1, the method of this invention outperforms the comparative methods in most metrics on the WV3 dataset, especially in Q, SAM, ERGAS, and QNR, indicating that the method achieves a relatively stable balance between detail enhancement and spectral preservation. The embodiments of this invention achieve scores of 0.9174, 2.9186, 2.1697, 0.9734, 0.9148, 0.0131, and 0.9589 on the seven metrics of Q, SAM, ERGAS, SCC, Qn, Qλ, and QNR, respectively, all of which are the best results on this dataset. Specifically, the method of this invention achieves the highest values ​​for Q, Qn, and SCC, and the lowest values ​​for SAM and ERGAS, indicating that the method effectively controls spectral bias and overall error while maintaining structural relevance. In the absence of reference metrics, the method of this invention also outperforms the other methods in Qλ and QNR, demonstrating that it maintains good spatial spectral consistency in real-world scenarios. Although DLSKNet slightly outperforms the method of this embodiment in terms of Qs (0.0256 vs. 0.0285), the difference is small. The method of this embodiment maintains a comprehensive lead in all other key metrics, indicating that the method of this embodiment has a more outstanding overall performance on the WV3 dataset.

[0149] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a USB flash drive, a solid-state drive (SSD), a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0150] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A remote sensing image fusion method based on dynamic region guidance, characterized in that, The method includes the following steps: Step 1: The shallow feature extraction module is used to extract shallow features from the combined multispectral and panchromatic images. Step 2, the dynamic region information construction module includes a structural saliency estimation unit and a gating prediction unit; wherein, the structural saliency estimation unit generates a structural saliency map based on the gradient information of the panchromatic image, and the gating prediction unit generates a content-adaptive gating map based on the initial fusion features jointly extracted from the multispectral image and the panchromatic image; the structural saliency map is used for key region ranking and Top-K selection, and the content-adaptive gating map is used for key region quantity estimation and enhancement intensity control; Step 3: The structure-guided enhancement module executes two branches in parallel around the key region: one branch performs local detail modeling and sparse relation enhancement in sequence, and the other branch performs multi-scale information supplementation. Then, the two branches are selectively fused. Multiple structure-guided enhancement modules share content adaptive gating graphs, Top-K key region index sets, and structure saliency graphs. Step 4: Output high-resolution multispectral images through the reconstruction module.

2. The remote sensing image fusion method based on dynamic region guidance as described in claim 1, characterized in that, The extraction and reconstruction of the joint shallow features include: concatenating the upsampled low-resolution multispectral image with the high-resolution panchromatic image in the channel dimension to obtain joint input features; obtaining initial fusion features through shallow convolution mapping; inputting the initial fusion features into multiple cascaded structure-guided enhancement modules to obtain deep fusion features; generating residual terms through a reconstruction module, and adding the residual terms to the upsampled low-resolution multispectral image to obtain a high-resolution multispectral image.

3. The remote sensing image fusion method based on dynamic region guidance as described in claim 1, characterized in that, The construction of the dynamic region information includes: generating a gradient magnitude map based on the gradient response of the high-resolution panchromatic image in the horizontal and vertical directions, and normalizing the gradient magnitude map to obtain a structural saliency map; generating a content-adaptive gating map based on the initial fusion features through a gating prediction module; wherein, the structural saliency map is used to characterize the intensity of structural changes in the boundary areas of edges, textures and ground features, and the content-adaptive gating map is used to characterize the enhancement requirements of different spatial locations.

4. The remote sensing image fusion method based on dynamic region guidance as described in claim 3, characterized in that, The generation of the Top-K key region index set includes: mapping the structural saliency map and the content adaptive gating map to the image patch scale to obtain the image patch-level structural saliency representation and the image patch-level gating representation; adaptively estimating the number of key regions based on the image patch-level gating representation; and selecting the corresponding number of image patches from high to low based on the image patch-level structural saliency representation to obtain the Top-K key region index set.

5. The remote sensing image fusion method based on dynamic region guidance as described in claim 1, characterized in that, The structure-guided enhancement module includes a local region detail modeling branch, a multi-scale information supplementation branch, a sparse global relation enhancement unit, and a selective fusion unit. The local region detail modeling branch is used to model the spatial dependencies within key regions, the sparse global relation enhancement unit is used to perform cross-regional relation reasoning on Top-K key regions, the multi-scale information supplementation branch is used to extract local contextual information under different receptive fields, and the selective fusion unit is used to fuse sparse relation enhancement features and multi-scale supplementary features.

6. The remote sensing image fusion method based on dynamic region guidance as described in claim 5, characterized in that, The sparse global relation enhancement unit selects node features corresponding to key image blocks based on the Top-K key region index set, and constructs similarity relationships between key nodes and anchors through learnable anchors, so that the key node features are generated into global relation enhancement features after being aggregated and backpropagated in the anchor space. The selective fusion unit adaptively fuses global relation enhancement features with multi-scale supplementary features, and controls the fusion intensity by combining content adaptive gating graphs.

7. A remote sensing image fusion system based on dynamic region guidance, implementing the remote sensing image fusion method based on any one of claims 1-6, characterized in that, The remote sensing image fusion system based on dynamic region guidance includes: The shallow feature extraction module is used to extract shallow features from the combined multispectral and panchromatic images. The dynamic region information construction module includes a structural saliency estimation unit and a gating prediction unit. The structural saliency estimation unit generates a structural saliency map based on gradient information from the panchromatic image, while the gating prediction unit generates a content-adaptive gating map based on initial fusion features jointly extracted from the multispectral and panchromatic images. The structural saliency map is used for key region ranking and Top-K selection, while the content-adaptive gating map is used for key region quantity estimation and enhancement intensity control. The structure-guided enhancement module is used to execute two branches in parallel around the key region: one branch performs local detail modeling and sparse relation enhancement in sequence, and the other branch performs multi-scale information supplementation. Then, the two branches are selectively fused. Multiple structure-guided enhancement modules share content adaptive gating graphs, Top-K key region index sets, and structure saliency graphs. The reconstruction module is used to output high-resolution multispectral images.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the remote sensing image fusion method based on dynamic region guidance as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the remote sensing image fusion method based on dynamic region guidance as described in any one of claims 1-6.

10. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the remote sensing image fusion system based on dynamic region guidance as described in claim 7.

Images