A remote sensing ground feature refinement extraction method based on a double-flow hybrid architecture and gradient guided gating

Abstract

The application discloses a remote sensing ground object fine extraction method based on a double-flow mixed architecture and gradient guide gating, specifically comprising the following steps: acquiring a high-resolution remote sensing image of a region to be monitored, performing standardization processing on the high-resolution remote sensing image, and constructing an input image tensor; inputting the input image tensor into a high-frequency response enhancement branch and a global context branch at the same time, and performing parallel feature extraction to obtain local high-frequency features and global semantic features; inputting the local high-frequency features and the global semantic features into a dynamic perception gating unit to generate a fusion feature map, collecting fusion feature maps of different feature extraction levels to obtain a multi-level fusion feature map; inputting the multi-level fusion feature map into a decoder to generate a ground object category probability map; and performing post-processing on the ground object category probability map to output vectorized ground object distribution data. The problems of large-scale regional texture fragmentation and small-scale target edge blur in the existing remote sensing ground object extraction technology are solved.

Description

Technical Field

[0001] This application relates to the fields of computer vision, pattern recognition and remote sensing image processing, and in particular to a method for refined extraction of remote sensing ground features based on a dual-stream hybrid architecture and gradient-guided gating. Background Technology

[0002] With the development of low-altitude UAV remote sensing technology, centimeter-level ultra-high resolution imagery is widely used. While such imagery is extremely rich in information, it faces a severe "scale-texture duality" challenge when automatically extracting data using deep learning: on the one hand, the high-frequency texture details within large-scale features (such as large tracts of farmland) cause traditional convolutional networks to overemphasize local details, resulting in numerous broken holes; on the other hand, small-scale features (such as field ridges or boundary lines less than 0.2 meters wide) easily lose structural features during network downsampling, leading to blurred extraction boundaries.

[0003] In existing technologies, convolutional networks, represented by ConvNeXt V2, have effectively alleviated the feature collapse problem in natural image classification and representation learning by introducing Global Response Normalization (GRN) and Fully Convolutional Masked Autoencoder (FCMAE). However, when directly transferred to ultra-high resolution remote sensing ground feature extraction tasks, the early large-step downsampling may not be suitable for fine-grained structures, resulting in the weakening of some slender targets and weak boundary information in the shallow encoding stage.

[0004] On the other hand, to balance global and local features, existing remote sensing extraction models have widely adopted dual-stream hybrid architectures, boundary enhancement branches, or CNN / Transformer fusion schemes. For example, DSHNet enhances the synergy between local and global features through dual-stream hybrid encoding, BEDSN strengthens edge representation through a boundary enhancement dual-stream structure, and models such as ILGS-Net, which are geared towards building extraction, employ parallel modeling of local detail streams and global semantic streams. These works collectively demonstrate that the combination of local detail modeling and global context modeling has become an important technical direction in high-resolution remote sensing scenarios.

[0005] Existing technologies mainly focus on the following approaches: first, improving boundary representation through boundary-assisted branches or edge detection branches; second, alleviating scale discrepancies through local-global dual-stream encoding; and third, enhancing the stability of backbone representation through pre-training and normalization strategies. However, the problems of texture fragmentation in large-scale regions and blurred edges of small-scale targets remain unresolved. Summary of the Invention

[0006] Based on this, this application addresses the problems of fragmented textures in large-scale regions and blurred edges of small-scale targets in existing remote sensing ground feature extraction technologies by providing a refined extraction method for remote sensing ground features based on a dual-stream hybrid architecture and gradient-guided gating.

[0007] The first aspect of this application provides a method for refined extraction of remotely sensed ground features based on a dual-stream hybrid architecture and gradient-guided gating, including:

[0008] Acquire high-resolution remote sensing images of the area to be monitored, perform standardization processing on the high-resolution remote sensing images, and construct an input image tensor;

[0009] The input image tensor is simultaneously input to the high-frequency response enhancement branch and the global context branch, and parallel feature extraction is performed to obtain local high-frequency features and global semantic features.

[0010] The local high-frequency features and global semantic features are input into the dynamic perception gating unit to generate a fused feature map. The fused feature maps of different feature extraction levels are summarized to obtain a multi-level fused feature map.

[0011] The multi-level fused feature map is input into the decoder to generate a land cover category probability map;

[0012] The probability map of the land cover categories is post-processed to output vectorized land cover distribution data.

[0013] The high-frequency response enhancement branch first performs shallow feature encoding on the input image tensor through a progressively overlapping downsampling module to output a shallow feature map, and then performs multi-stage encoding on the shallow feature map through a convolutional feature extraction network to obtain local high-frequency features.

[0014] The global context branch encodes the input image tensor or the feature representation of the input image tensor through a feature extraction network based on a self-attention mechanism, models the contextual association between different spatial regions, and obtains global semantic features.

[0015] The generation of the fused feature map includes,

[0016] Feature interaction is performed between the corresponding levels of the high-frequency response enhancement branch and the global context branch through a dynamic perception gating unit;

[0017] The dynamic perception gating unit calculates the spatial gradient features of local texture features in the input image tensor and obtains the channel modulation statistics generated by the global response normalization layer in the high-frequency response enhancement branch. The spatial gradient features and the channel modulation statistics are used together as the joint gating basis to generate a dynamic gating weight map.

[0018] Using the aforementioned dynamic gating weight map, a fused feature map is generated by fusing local high-frequency features and global semantic features through a local dominant dynamic reweighting mechanism.

[0019] The calculation of the fused feature obtained by dynamically weighting local and global features is as follows:

[0020] ;

[0021] in, Indicates fusion characteristics, Represents local texture features. Represents global semantic features. This indicates element-wise multiplication, and α represents the preset gain adjustment coefficient. This represents a dynamic gating weight graph.

[0022] include,

[0023] The progressive overlap downsampling module includes a first convolutional layer and a second convolutional layer connected in sequence; both the first and second convolutional layers use 3x3 convolutional kernels, with stride set to 2 and padding set to 1.

[0024] The specific operations that use the spatial gradient features and channel modulation statistics as the basis for joint gating include:

[0025] In any feature extraction level, the global response intensity of each channel is calculated in the spatial dimension for the local high-frequency features output by the high-frequency response enhancement branch, and the global response intensity is normalized in the channel dimension to obtain the channel modulation statistics.

[0026] The channel modulation statistics are broadcast along the spatial dimension to obtain a channel modulation statistics map that is consistent with the local high-frequency feature size.

[0027] Spatial gradient features are calculated based on the local high-frequency features, and the spatial gradient features are used to characterize the boundaries of ground features and the location of local high-frequency changes.

[0028] The local high-frequency features, global semantic features, spatial gradient features, and channel modulation statistics are concatenated along the channel dimension to obtain the joint gated input features.

[0029] The generation of the dynamic gating weight graph includes,

[0030] Local high-frequency features are processed using an edge detection operator to generate a spatial gradient magnitude map.

[0031] The channel response statistics are broadcast in the spatial dimension to obtain a channel saliency map;

[0032] The spatial gradient magnitude map, channel saliency, local high-frequency features, and global semantic features are concatenated to obtain joint gated input features;

[0033] The calculation of the joint gated input feature Z includes,

[0034] = ;

[0035] Where Z represents the joint gated input feature, This indicates concatenation along the channel dimension. Represents local texture features. Represents global semantic features. Represents the spatial gradient magnitude map. Represents the saliency of the channels;

[0036] The joint gated input features are input into a gated mapping network, and a dynamic gated weight map is generated through convolutional mapping and Sigmoid activation.

[0037] ;

[0038] in, Represents a dynamic gating weight graph. This represents the Sigmoid activation function. This represents a gated mapping network consisting of convolutional layers.

[0039] It also includes,

[0040] The high-frequency response enhancement branch is initialized and trained using a fully convolutional masked autoencoder, specifically as follows:

[0041] In the pre-training stage, given the characteristics of high-resolution remote sensing images, such as dense high-frequency texture distribution, rich fine boundary structures, and high requirements for spatial continuity of patches, a random masking strategy is adopted to perform pixel masking processing on the high-resolution remote sensing images, with the masking ratio set to 50%-75%. Only the visible areas of the high-resolution remote sensing images that are not masked are selected as input to the encoder, and the feature encoding of the visible areas is completed through sparse convolution operation. The masked areas do not participate in the convolution calculation during the encoding process. The decoder combines the feature information extracted from the visible areas and the mask position information to perform pixel reconstruction of the masked areas of the high-resolution remote sensing images. Finally, a reconstruction loss function is constructed based on the pixel differences between the reconstructed image and the original high-resolution remote sensing image.

[0042] In a second aspect, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned method.

[0043] In a third aspect, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method.

[0044] Beneficial Effects: In the scenario of high-resolution remote sensing image refinement, a combination of progressive overlap downsampling and jointly gated dynamic reweighted fusion is used to improve the representation quality of slender features and complex edges. The jointly gated dynamic reweighted fusion does not simply use existing boundary enhancement branches or conventional dual-stream weighting methods, but rather dynamically adjusts the fusion bias of local and global branches at different spatial locations and channels through joint control of spatial gradient information and channel modulation statistics. Boundary consistency loss is used as an auxiliary optimization technique in the decoding stage of some implementations to further improve the stability of boundary output.

[0045] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0046] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein:

[0047] Figure 1 This is based on the overall process diagram provided in this application;

[0048] Figure 2 This is the overall flowchart of the dual-stream hybrid architecture and dynamic perception gating provided in this application;

[0049] Figure 3 This is an internal structure diagram of the dynamic perception gating unit combined with dual priors to generate gating weights, as provided in this application.

[0050] Figure 4 The effect curve and fusion diagram are based on the locally dominant dynamic reweighted fusion mechanism provided in this application;

[0051] Figure 5 This is an image recognition effect of a large area of ​​purple-covered farmland along a river based on the traditional spliced ​​dual-stream network model provided in this application;

[0052] Figure 6 This is an image recognition effect of water villages and river-surrounded covered farmland based on the traditional spliced ​​dual-flow network model provided in this application;

[0053] Figure 7 This is an image extracted from a model based on spatial gradient and channel statistics combined gating optimization of images of large areas of purple-covered farmland along the river provided in this application.

[0054] Figure 8 The image is an extraction result of the model based on the images of water villages and river-surrounded covered farmland provided in this application, optimized by joint gating of spatial gradient and channel statistics. Detailed Implementation

[0055] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0056] like Figure 1 , Figure 2 As shown, the first aspect of this application provides a method for refined extraction of remotely sensed ground features based on a dual-stream hybrid architecture and gradient-guided gating, comprising:

[0057] S1: Acquire high-resolution remote sensing images of the area to be monitored, perform standardization processing on the high-resolution remote sensing images, and construct the input image tensor.

[0058] S2: The input image tensor is simultaneously input to the high-frequency response enhancement branch and the global context branch, and parallel feature extraction is performed to obtain local high-frequency features and global semantic features. It should be noted that:

[0059] The high-frequency response enhancement branch first performs shallow feature encoding on the input image tensor through a progressively overlapping downsampling module to output a shallow feature map, and then performs multi-stage encoding on the shallow feature map through a convolutional feature extraction network to obtain local high-frequency features.

[0060] To address the issue of easily losing small features, this application constructs a progressively overlapping downsampling module at the input of the high-frequency response enhancement branch. The progressively overlapping downsampling module consists of at least two cascaded convolutional layers, each requiring a kernel stride smaller than the kernel size, resulting in denser local sampling coverage through overlapping convolutions. Compared to traditional large-stride non-overlapping blocks, this structure is more advantageous for preserving local structural information related to small features and slender boundaries during the shallow encoding stage.

[0061] The global context branch encodes the input image tensor or the feature representation of the input image tensor through a feature extraction network based on a self-attention mechanism, models the contextual association between different spatial regions, and obtains global semantic features.

[0062] The high-frequency response enhancement branch is used to extract local texture, high-frequency edge, and fine structural features from the input image tensor. Specifically, the high-frequency response enhancement branch first performs shallow feature encoding on the input image tensor through a progressively overlapping downsampling module to preserve detailed information such as ground boundaries, field ridges, road edges, and building outlines while reducing the feature map resolution; then, it performs multi-stage encoding on the shallow features through a convolutional feature extraction network to obtain local high-frequency features.

[0063] The high-frequency response enhancement branch includes multiple feature extraction modules, each of which has a global response normalization layer. The global response normalization layer is used to enhance the response competition between channels and alleviate the channel representation homogenization problem caused by similar ground object spectra or similar texture patterns.

[0064] The global context branch is used to extract large-scale semantic context features from the input image tensor. Specifically, the global context branch encodes the input image tensor or its feature representation using a feature extraction network based on a self-attention mechanism, models the contextual relationships between different spatial regions, and obtains global semantic features.

[0065] The local high-frequency features and global semantic features serve as inputs to the subsequent dynamic perception gating unit. Specifically, the local high-frequency features are used to extract spatial gradient information and channel statistics to generate boundary location cues and channel discrimination cues required for gating estimation; the global semantic features provide broad semantic constraints during subsequent feature fusion.

[0066] S3: The local high-frequency features and global semantic features are input into the dynamic perception gating unit to generate a fused feature map. The fused feature maps from different feature extraction levels are then summarized to obtain a multi-level fused feature map. It should be noted that:

[0067] The generation of the fused feature map includes,

[0068] Feature interaction is performed between the corresponding levels of the high-frequency response enhancement branch and the global context branch through a dynamic perception gating unit;

[0069] The dynamic perception gating unit calculates the spatial gradient features of local texture features in the input image tensor and obtains the channel modulation statistics generated by the global response normalization layer in the high-frequency response enhancement branch. The spatial gradient features and channel modulation statistics are used together as the joint gating criteria to generate a dynamic gating weight map, such as... Figure 3 As shown.

[0070] Using the aforementioned dynamic gating weight map, a fused feature map is generated by fusing local high-frequency features and global semantic features through a local dominant dynamic reweighting mechanism.

[0071] The calculation of the fused feature obtained by dynamically weighting local and global features is as follows:

[0072] ;

[0073] in, Indicates fusion characteristics, Represents local texture features. Represents global semantic features. This indicates element-wise multiplication, and α represents the preset gain adjustment coefficient. This represents a dynamic gating weight graph.

[0074] The preset gain adjustment coefficient α is used to adjust the contribution bias between local and global branches, and the value of α ranges from 0.2 to 2.0; when When the value approaches 1, local texture features The contribution to the fusion results is enhanced; when When the value approaches 0, the fusion result retains more... It provides semantic supplementation to smooth regions.

[0075] include,

[0076] The progressive overlapping downsampling module includes a first convolutional layer and a second convolutional layer connected in sequence. Both the first and second convolutional layers use 3x3 convolutional kernels, with a stride of 2 and a padding of 1. Compared to a single-layer non-overlapping block embedding operation with a stride of 4, the progressive overlapping downsampling module improves the local sampling coverage density of the input image in the early coding stage through multiple overlapping convolution operations, thereby reducing the risk of information loss in small target structures during downsampling.

[0077] The specific operations that use the spatial gradient features and channel modulation statistics as the basis for joint gating include:

[0078] In any feature extraction level, the global response intensity of each channel is calculated in the spatial dimension for the local high-frequency features output by the high-frequency response enhancement branch, and the global response intensity is normalized in the channel dimension to obtain the channel modulation statistics.

[0079] The channel modulation statistics are broadcast along the spatial dimension to obtain a channel modulation statistics map that is consistent with the local high-frequency feature size.

[0080] Spatial gradient features are calculated based on the local high-frequency features, and the spatial gradient features are used to characterize the boundaries of ground features and the location of local high-frequency changes.

[0081] The local high-frequency features, global semantic features, spatial gradient features, and channel modulation statistics are concatenated along the channel dimension to obtain the joint gated input features.

[0082] The generation of the dynamic gating weight graph includes,

[0083] Local high-frequency features are processed using an edge detection operator to generate a spatial gradient magnitude map.

[0084] The channel response statistics are broadcast in the spatial dimension to obtain a channel saliency map;

[0085] The spatial gradient magnitude map, channel saliency, local high-frequency features, and global semantic features are concatenated to obtain joint gated input features;

[0086] The calculation of the joint gated input feature Z includes,

[0087] = ;

[0088] Where Z represents the joint gated input feature, This indicates concatenation along the channel dimension. Represents local texture features. Represents global semantic features. Represents the spatial gradient magnitude map. Represents the saliency of the channels;

[0089] The joint gated input features are input into a gated mapping network, and a dynamic gated weight map is generated through convolutional mapping and Sigmoid activation.

[0090] ;

[0091] in, Represents a dynamic gating weight graph. This represents the Sigmoid activation function. This represents a gated mapping network consisting of convolutional layers.

[0092] To address the issue of high-frequency edge features gradually weakening with increasing network depth, this application proposes a dynamically perceptual gating unit and a locally dominant dynamic reweighting fusion mechanism. First, the dynamically perceptual gating unit jointly extracts the spatial gradient magnitude and channel modulation statistics represented by the output of the global response normalization layer, forming spatial variation cues and channel modulation criteria, thereby generating a dynamic gating weight map. Subsequently, this weight map is used to dynamically reweight the local and global branches, ensuring that local texture branches within edge-sensitive regions contribute a higher proportion to the fusion result, thus improving the representation quality of slender features and high-contrast boundaries.

[0093] Figure 4 The diagram shows the effect curve and fusion illustration of the locally dominant dynamic reweighted fusion mechanism. While the traditional weighted averaging method (blue) can increase signal intensity at region edges, it lacks feature amplification capabilities. In contrast, the locally dominant dynamic reweighted fusion mechanism of this invention (green) uses a Gmap dynamic gating map for dominant weighting. When the region has edge texture, the contribution coefficient of local texture features is adaptively amplified (greater than 1.0), while global semantic features are suppressed. This significantly enhances the dominance of local branches in the fusion result, greatly improving the detail preservation ability and boundary classification sharpness within the edge neighborhood.

[0094] S4: Input the multi-level fused feature map into the decoder to generate a land cover category probability map. It should be noted that:

[0095] The different feature extraction levels include shallow, medium, and deep feature levels, or multiple feature extraction level segments in the network encoding stage. Shallow fused feature maps mainly preserve land cover boundaries, texture, and fine structural information, while deep fused feature maps mainly contain land cover category semantics and global context information.

[0096] The decoder upsamples, adjusts channels, and fuses cross-layer features in the multi-level fused feature map to restore it to the spatial resolution corresponding to the input image, and outputs the probability distribution of each pixel belonging to different land cover categories to form a land cover category probability map.

[0097] S5: Post-process the aforementioned land cover category probability map to output vectorized land cover distribution data. It should be noted that:

[0098] It also includes,

[0099] The high-frequency response enhancement branch is initialized and trained using a fully convolutional masked autoencoder, specifically as follows:

[0100] In the pre-training phase, given the characteristics of high-resolution remote sensing images—dense high-frequency texture distribution, rich fine boundary structures, and high requirements for spatial continuity of patches—a random masking strategy is employed to perform pixel masking on the high-resolution remote sensing images, with the masking ratio set to 50%-75%. Only the visible regions of the high-resolution remote sensing images that are not masked are selected as input to the encoder, and the features of the visible regions are encoded through sparse convolution operations. The masked regions do not participate in the convolution calculations during the encoding process. The decoder combines the feature information extracted from the visible regions with the masking position information to perform pixel reconstruction of the masked regions of the high-resolution remote sensing images. Finally, a reconstruction loss function is constructed based on the pixel differences between the reconstructed image and the original high-resolution remote sensing image. The random masking ratio, set to 50% to 75%, is used to create varying degrees of spatial missingness and contextual fragmentation in the input image, rather than acting as a single fixed parameter.

[0101] During the pre-training phase, the model learns to recover ground feature structures under conditions of high proportion of missing and fragmented contexts, thereby enhancing its ability to represent the continuity of remotely sensed ground feature structures such as field ridges, road edges, building outlines, water-land boundaries, and farmland patch boundaries. A masking ratio of 50% to 75%, together with sparse convolutional coding and occlusion region reconstruction tasks, constitutes the pre-training strategy for high-resolution remotely sensed ground feature extraction.

[0102] This application uses fully convolutional mask autoencoders, global response normalization, and boundary consistency loss as auxiliary means to implement the core architecture. Among them, the fully convolutional mask autoencoder enhances the pre-training of remote sensing image features through 50% to 75% random masks, global response normalization is used to enhance inter-channel response competition and alleviate channel representation homogenization caused by the similarity of ground object spectra, and boundary consistency loss provides auxiliary constraints on boundary geometric consistency at the decoding end in some implementations.

[0103] This embodiment constructs a parallel dual-stream network (HybriSeg-Remote) for 0.05-meter-level ultra-high resolution remote sensing images acquired by UAVs. The network's hardware operating environment is a deep learning workstation configured with an NVIDIA RTX 4090 GPU, and the software framework is built based on PyTorch 2.0.

[0104] Radiometric correction and geometric registration are performed on the acquired UAV ultra-high resolution remote sensing imagery. Subsequently, a sliding window overlap cropping strategy is used to crop the wide-format imagery into image patches of fixed resolution. This embodiment sets the input image tensor. , where B is the batch size, and the pixel values ​​are normalized before input (mean and standard deviation are calculated along the RGB channels).

[0105] The input image tensor X is fed into the high-frequency response enhancement branch. To address the signal aliasing defect that occurs in small-scale features (such as field ridges 0.1m to 0.2m wide) in remote sensing images under this sampling method, this embodiment constructs a microstep downsampling module, the network layer configuration of which is shown in Table 1.

[0106] Table 1: Network Hierarchy Configuration of the Microstepping Downsampling Module

[0107]

[0108] Through the aforementioned concatenated structure, although the input image tensor is ultimately downsampled to 128×128 (the overall downsampling rate remains at 4 times, meeting the input requirements of the subsequent backbone network), because the stride is strictly smaller than the convolution kernel size, any input pixel is overlapped and covered by at least 4 different sliding convolution kernels. This overlapping sampling is equivalent to introducing a local smoothing low-pass effect at the physical signal processing level, effectively mitigating aliasing distortion in high-frequency structures.

[0109] The features after microstep downsampling are input into the improved ConvNeXt V2 backbone network, which is divided into four feature extraction levels (feature dimensions of 96, 192, 384, and 768, respectively). Within each basic block, a global response normalization layer is introduced to address the "feature collapse" caused by the spectral similarity of remotely sensed ground objects.

[0110] To capture the internal structural connectivity of large-scale features such as extensive farmland and woodland, a global context branch is set up in parallel. This embodiment uses the Swin Transformer-Tiny structure, and at Stage 3 and Stage 4, a self-attention mechanism based on local moving windows is used to extract global semantic features, denoted as... .

[0111] Local texture features of high-frequency branch output With global branch output Interact with it.

[0112] Assuming that at Stage 3, the output feature dimensions of both branches are... (e.g., 32×32×384).

[0113] Spatial gradient information: First, a 1×1 convolution is used to convert the high-dimensional semantic tensor into a spatial gradient information. Dimensionality reduction projection to a single channel, followed by application of fixed-weight Sobel operators to the pairs. Convolutions are performed in both the horizontal and vertical directions, and the root mean square is taken to obtain the gradient magnitude map. The larger this value, the stronger the edge or the higher the frequency of texture. This avoids the mathematical errors of directly performing edge operators on high-dimensional features.

[0114] Channel modulation statistics: This refers to the channel normalization results output from the GRN layer in Example 1. Extend it to space broadcasting , is used to characterize the relative modulation intensity of each channel in the gating estimation.

[0115] Dynamic gating weight graph generation:

[0116] Will , , and The concatenation is performed along the channel dimension, and a cross-modal joint attention layer is used to reconcile the differences between channel and spatial attributes. The result is then fed into the perceptual coding layer. This perceptual coding layer consists of two 1×1 convolutional layers. The first layer maps the concatenated features to the intermediate channel dimension C', which is set to 192 in this embodiment. The second layer outputs... A consistent channel dimension is used to generate the gated weight graph. After activation by the Sigmoid function, the gated weight graph is obtained. .

[0117] in, The channel normalization result output from the GRN layer is spatially broadcast to provide modulation information along the channel dimension for gating estimation, rather than directly indicating whether a spatial location is an edge. This is combined with spatial gradient information. Subsequently, DPGU utilized Provide spatial location clues, utilize It provides a basis for channel dimension modulation, and the two together serve the gating weight estimation to form a controlled fusion between local texture information and global semantic information.

[0118] Locally dominant dynamic reweighted fusion:

[0119] Execute the fusion formula: .

[0120] Set the gain adjustment coefficient α = 1.0.

[0121] Calculation Example: Suppose a feature point is located at the "edge of a building," has a high Sobel gradient response, and the corresponding channel exhibits strong response competition characteristics after GRN normalization, resulting in a Gmap prediction value of 0.9 for that location. In this case:

[0122] The contribution coefficient of local texture features = 1 + 1.0 × 0.9 = 1.9;

[0123] Global semantic feature contribution coefficient = 1 - 1.0 × 0.9 = 0.1.

[0124] In this scenario, the fusion result will rely more on local branches, thus improving the ability to preserve details within the edge neighborhood. Conversely, if located in the smooth region of the "water body center," If the predicted value is 0.1, the fusion result retains more of the semantic supplementation effect of global features on smooth regions.

[0125] To adapt to the high-frequency, detailed characteristics of remote sensing images, the high-frequency response enhancement branch is pre-trained in unsupervised mode using a fully convolutional mask autoencoder (FCMAE) before supervised training.

[0126] 1. Constructing masked data: 100,000 unlabeled 0.05-meter-level remote sensing image patches are used. Considering that the information redundancy of remote sensing data is relatively low compared to natural images, this embodiment strictly limits the masking ratio to between 50% and 75% (preferably 60%).

[0127] 2. Sparse Convolution Reconstruction: The standard convolution is replaced by a 3D sparse convolution algorithm such as Minkowski Engine. The model only performs forward propagation encoding on the 40% of visible pixels that are not occluded. This not only significantly reduces the computational cost of pre-training, but also forces the network to learn the ability to infer hidden building corners and road continuity in a highly fragmented context.

[0128] It should be noted that the FCMAE pre-training strategy adopted in this embodiment is mainly aimed at the local texture preservation and structural completion requirements in ultra-high resolution remote sensing images. The focus is on providing adaptive initialization for the subsequent segmentation backbone through fully convolutional reconstruction. This is different from the FG-MAE route which is aimed at building multispectral or SAR remote sensing basic models. The latter emphasizes improving the cross-modal or multi-source remote sensing semantic representation ability through feature-guided reconstruction targets. However, this embodiment does not introduce feature guidance mechanisms for multispectral index, HOG guidance, or cross-modal reconstruction targets.

[0129] 3. Parameter transfer: After pre-training is completed, the decoder is discarded, and the encoder weights are directly used as the initialization parameters of the microstep downsampling module and ConvNeXt V2 Block, and the process begins to enter steps S4 and S5 with labeled data.

[0130] In the decoding and supervised fine-tuning stage (S4), a joint loss function containing multiple terms is constructed, and in some implementations, boundary consistency loss is introduced as an auxiliary supervision term. This loss function constrains the matching degree between the continuous smooth probability field of the network output and the discrete label boundary, making the convergence process of boundary sharpening more stable.

[0131] To verify the effectiveness of each technical aspect of this application in the target business scenario, this embodiment conducted comparative and ablation experiments based on an internal UAV mapping dataset and a publicly available remote sensing segmentation benchmark dataset. The internal dataset experiments primarily illustrate the feasibility of implementing this invention in ultra-high resolution remote sensing ground feature refinement tasks, its parameter sensitivity, and the relative roles of the structural combinations in the target business scenario. The publicly available dataset experiments primarily examine the generalization performance and horizontal comparability of this invention in standard remote sensing semantic segmentation evaluation scenarios.

[0132] Experimental setup: The AdamW optimizer was used with an initial learning rate of 0.0001 and a cosine annealing decay strategy; the batch size was set to 16, and training lasted for 100 epochs. For the public dataset experiments, the remote sensing dataset (LoveDA) was selected as the standard evaluation dataset, and data partitioning and common evaluation criteria were adopted consistent with existing public methods.

[0133] To demonstrate the gain effect of the micro-stride module designed in Table 1 on small targets, the Stem structure at the input end was replaced while keeping other modules of the network unchanged. The results are shown in Table 2.

[0134] Table 2: Impact of input-side downsampling strategy on the extraction effect of slender features

[0135]

[0136] At a resolution of 0.05m, progressive overlay downsampling increased the IoU of extremely narrow features by 7.7% by improving the early sampling coverage density, indicating that this structure has a positive effect on alleviating the "loss of small-scale feature structures".

[0137] The effect of the preset gain coefficient α on the final boundary sharpness of the DPGU gating module is shown in Table 3.

[0138] Table 3: Influence of gain coefficient on boundary sharpening and fusion effects in locally dominant dynamic reweighting mechanisms

[0139]

[0140] The locally dominant dynamic reweighting strategy plays a crucial role in the representation of boundary regions. As α gradually increases from 0.2 to 1.0, local branches gain a higher contribution ratio within the edge neighborhood, and the boundary quality continues to improve. When α continues to increase to 1.5 and 2.0, although the boundary sharpening effect still exists, the risk of artifacts due to excessively strong local responses also begins to increase. Therefore, α=1.0 achieves a better balance between enhancing boundary details and suppressing artifact risk, and effective fusion regulation can be achieved when α ranges from 0.2 to 2.0.

[0141] This application selected representative remote sensing segmentation networks for overall performance comparison. The evaluation metrics used were global average intersection-over-union ratio (mIoU), overall accuracy (OA), number of network parameters (Params), and model computational cost (FLOPs), as shown in Table 4.

[0142] Table 4: Global Model Performance Comparison Analysis

[0143]

[0144] In high-resolution remote sensing test scenarios, the proposed method (HybriSeg-Remote) achieved an mIoU of 84.8% and an OA of 91.3% on a publicly available farmland dataset, while maintaining controllable parameter count (68.5 MParams) and computational cost (1050 G FLOPs). Combined with internal scenario experiments, the performance improvement can be attributed to the synergistic effect of progressive overlapping downsampling, GRN-assisted channel representation stabilization, FCMAE pre-training, and the dynamic reweighting fusion mechanism.

[0145] Figure 5 This is a rendering of the image recognition results of a large area of ​​purple-covered farmland along a river using a traditional stitched dual-stream network model. Figure 6 This is an image recognition rendering of a water village and river-surrounded, mulched farmland, based on a traditional stitched dual-flow network model. Figure 7 This image shows the extraction results of a model based on joint gating optimization using spatial gradient and channel statistics, representing a large area of ​​purple-covered farmland along the river. Figure 8 This image shows the extraction results of a model based on joint gating optimization of spatial gradient and channel statistics for images of water villages and river-surrounded covered farmland. Figure 5 and Figure 6 As can be seen in the image, there are holes inside, and the boundaries are not clear. Figure 7 and Figure 8 The holes in the image have clearly disappeared, and boundary recognition has also been significantly improved.

[0146] In a second aspect, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned method.

[0147] In a third aspect, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method.

[0148] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed.

[0149] The units may or may not be physically separate. The components shown as units can be one or more physical units, meaning they can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0151] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, essentially, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for refined extraction of remotely sensed ground features based on a dual-stream hybrid architecture and gradient-guided gating, characterized in that, include: Acquire high-resolution remote sensing images of the area to be monitored, perform standardization processing on the high-resolution remote sensing images, and construct an input image tensor; The input image tensor is simultaneously input to the high-frequency response enhancement branch and the global context branch, and parallel feature extraction is performed to obtain local high-frequency features and global semantic features. The local high-frequency features and global semantic features are input into the dynamic perception gating unit to generate a fused feature map. The fused feature maps of different feature extraction levels are summarized to obtain a multi-level fused feature map. The multi-level fused feature map is input into the decoder to generate a land cover category probability map. The probability map of the land cover categories is post-processed to output vectorized land cover distribution data.

2. The method for refined extraction of remote sensing ground features based on a dual-stream hybrid architecture and gradient-guided gating as described in claim 1, characterized in that: The high-frequency response enhancement branch first performs shallow feature encoding on the input image tensor through a progressively overlapping downsampling module to output a shallow feature map, and then performs multi-stage encoding on the shallow feature map through a convolutional feature extraction network to obtain local high-frequency features.

3. The method for refined extraction of remote sensing ground features based on a dual-stream hybrid architecture and gradient-guided gating as described in claim 2, characterized in that: The global context branch encodes the input image tensor or the feature representation of the input image tensor through a feature extraction network based on a self-attention mechanism, models the contextual association between different spatial regions, and obtains global semantic features.

4. A method for refined extraction of remote sensing ground features based on a dual-stream hybrid architecture and gradient-guided gating according to any one of claims 1-3, characterized in that: The generation of the fused feature map includes, Feature interaction is performed between the corresponding levels of the high-frequency response enhancement branch and the global context branch through a dynamic perception gating unit; The dynamic perception gating unit calculates the spatial gradient features of local texture features in the input image tensor and obtains the channel modulation statistics generated by the global response normalization layer in the high-frequency response enhancement branch. The spatial gradient features and the channel modulation statistics are used together as the joint gating basis to generate a dynamic gating weight map. Using the aforementioned dynamic gating weight map, a fused feature map is generated by fusing local high-frequency features and global semantic features through a local dominant dynamic reweighting mechanism. The calculation of the fused feature obtained by dynamically weighting local and global features is as follows: ； in, Indicates fusion characteristics, Represents local texture features. Represents global semantic features. This indicates element-wise multiplication, and α represents the preset gain adjustment coefficient. This represents a dynamic gating weight graph.

5. The method for refined extraction of remote sensing ground features based on a dual-stream hybrid architecture and gradient-guided gating as described in claim 2, characterized in that: include, The progressive overlap downsampling module includes a first convolutional layer and a second convolutional layer connected in sequence; both the first and second convolutional layers use 3x3 convolutional kernels, with stride set to 2 and padding set to 1.

6. The method for refined extraction of remote sensing ground features based on a dual-stream hybrid architecture and gradient-guided gating as described in claim 4, characterized in that: The specific operations that use the spatial gradient features and channel modulation statistics as the basis for joint gating include: In any feature extraction level, the global response intensity of each channel is calculated in the spatial dimension for the local high-frequency features output by the high-frequency response enhancement branch, and the global response intensity is normalized in the channel dimension to obtain the channel modulation statistics. The channel modulation statistics are broadcast along the spatial dimension to obtain a channel modulation statistics map that is consistent with the local high-frequency feature size. Spatial gradient features are calculated based on the local high-frequency features, and the spatial gradient features are used to characterize the boundaries of ground features and the location of local high-frequency changes. The local high-frequency features, global semantic features, spatial gradient features, and channel modulation statistics are concatenated along the channel dimension to obtain the joint gated input features.

7. The method for refined extraction of remote sensing ground features based on a dual-stream hybrid architecture and gradient-guided gating as described in claim 6, characterized in that: The generation of the dynamic gating weight graph includes, Local high-frequency features are processed using an edge detection operator to generate a spatial gradient magnitude map. The channel response statistics are broadcast in the spatial dimension to obtain a channel saliency map; The spatial gradient magnitude map, channel saliency, local high-frequency features, and global semantic features are concatenated to obtain joint gated input features; The calculation of the joint gated input feature Z includes, = ； Where Z represents the joint gated input feature, This indicates concatenation along the channel dimension. Represents local texture features. Represents global semantic features. Represents the spatial gradient magnitude map. Represents the saliency of the channels; The joint gated input features are input into a gated mapping network, and a dynamic gated weight map is generated through convolutional mapping and Sigmoid activation. ； in, Represents a dynamic gating weight graph. This represents the Sigmoid activation function. This represents a gated mapping network consisting of convolutional layers.

8. The method for refined extraction of remote sensing ground features based on a dual-stream hybrid architecture and gradient-guided gating as described in claim 7, characterized in that: It also includes, The high-frequency response enhancement branch is initialized and trained using a fully convolutional masked autoencoder, specifically as follows: In the pre-training stage, given the characteristics of high-resolution remote sensing images, such as dense high-frequency texture distribution, rich fine boundary structures, and high requirements for spatial continuity of patches, a random masking strategy is adopted to perform pixel masking processing on the high-resolution remote sensing images, with the masking ratio set to 50%-75%. Only the visible areas of the high-resolution remote sensing images that are not masked are selected as input to the encoder, and the feature encoding of the visible areas is completed through sparse convolution operation. The masked areas do not participate in the convolution calculation during the encoding process. The decoder combines the feature information extracted from the visible areas and the mask position information to perform pixel reconstruction of the masked areas of the high-resolution remote sensing images. Finally, a reconstruction loss function is constructed based on the pixel differences between the reconstructed image and the original high-resolution remote sensing image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images