High-resolution remote sensing image road extraction method, system and device based on HRNetformer network and medium

Abstract

The invention discloses a high-resolution remote sensing image road extraction method, system and device based on an HRNetformer network, and a medium. The method comprises the following steps: constructing a high-resolution remote sensing image road extraction HRNetformer network model; training a high-resolution remote sensing image road extraction HRNetformer network model by using a training set in a high-resolution remote sensing image road extraction public data set, and outputting a training weight file HRNetformer.pt; performing road extraction on the high-resolution remote sensing image road extraction HRNetformer network model by using a test set and a weight file HRNetformer.pt in the high-resolution remote sensing image road extraction public data set to obtain a road extraction result; the system, the equipment and the medium are used for implementing the method. According to the method, the precision, connectivity and detail keeping capability of road extraction are effectively improved, the robustness and generalization capability of the model under a complex background and a high-resolution remote sensing image are enhanced, and an efficient and reliable technical scheme with application value is provided for a high-precision road extraction task.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, further relates to deep learning and digital image processing technology, and specifically relates to a high-resolution remote sensing image road extraction method, system, device and medium based on an HRNetformer network. BACKGROUND

[0002] With the development of remote sensing technology and aerospace sensors, high-resolution remote sensing images have been significantly improved in terms of spatial resolution, spectral resolution and temporal resolution, providing more detailed data support for geographic information acquisition and surface feature extraction. Roads, as an important part of urban infrastructure, have wide application value in traffic management, urban planning, disaster emergency and autonomous driving. Therefore, efficient and accurate extraction of road information from high-resolution remote sensing images has become a research hotspot in remote sensing image interpretation and intelligent analysis.

[0003] In recent years, researchers have proposed various methods for road extraction from remote sensing images, from traditional image processing methods based on edge detection, texture analysis and mathematical morphology, to model methods based on probability graph models and energy function optimization, to the rapidly developing deep learning and convolutional neural network (CNN) methods in recent years. These methods have improved the automation and accuracy of road extraction to some extent, laying a foundation for the construction of intelligent road information systems.

[0004] However, due to the complex morphology, uneven lighting, vegetation or building shielding of roads in remote sensing images, there are still many challenges. Especially in complex urban or rural road environments, the extraction algorithm is prone to problems such as cracking, discontinuity, false extraction or missing extraction. Therefore, how to combine the characteristics of high-resolution remote sensing data to design a robust and generalizable road extraction method is still a research topic that needs to be solved.

[0005] The patent application file with publication number CN119648992A discloses a remote sensing image road extraction method. The implementation steps of the method are to obtain a remote sensing road image and crop the region of interest; use the improved baseline model RNGDet backbone network to extract the deep features of the region of interest to obtain a first feature tensor; based on the first feature tensor, use the improved feature pyramid network structure to perform road segmentation prediction to obtain two distribution maps; crop and rasterize the obtained historical trajectory map to obtain a rasterized map; input the two distribution maps and the rasterized map into the backbone network to obtain a second feature tensor; perform a splicing operation on the first feature tensor and the second feature tensor to obtain a target feature tensor; input the target feature tensor as the feature extraction result of the backbone network into the Transformer structure for prediction; and update the road network map according to the prediction result. However, the invention still has the following deficiencies: the road structure continuity and semantic consistency of the method are still poor.

[0006] The patent application file with publication number CN119313557A discloses a remote sensing image road extraction method based on CATransFuse model. The implementation steps of the method are to obtain a remote sensing image, slice it according to a fixed size to obtain a remote sensing image slice. Input the remote sensing image slice into the trained network CATransFuse for forward propagation operation to obtain the road features therein and output the road features. However, the invention still has the following deficiencies: when facing complex environments, the road extraction accuracy of the method is still low.

[0007] In summary, the existing technology has the following defects and deficiencies: 1. It is difficult to guarantee the road structure continuity and semantic consistency. Existing methods often focus on pixel-level classification accuracy, but ignore the overall topological properties of road networks as network structures, resulting in frequent problems such as broken, misconnected, and incomplete. This defect can greatly affect subsequent road network construction, path planning, and spatial analysis, limiting the value of road extraction results in practical applications.

[0008] 2. The existing method still has low accuracy when facing complex environments, such as rural roads, and is prone to "broken roads" or "pseudo-connected" situations. SUMMARY

[0009] To solve the problems in the prior art, the present application aims to provide a high-resolution remote sensing image road extraction method, system, device and medium based on an HRNetformer network, which first proposes a bidirectional feature extraction module (BiFE) to realize bidirectional information propagation and enhance the richness and diversity of feature representation; secondly, a context and direction perception feature perception enhancement module (CDFPE) is proposed to refine the features, which fuses shallow and deep information and adopts local and global attention mechanisms to improve the network's ability to perceive direction patterns and context information, which is crucial for depicting road structures; the architecture adopts a parallel multi-branch strategy to realize synchronous processing of different scale features, promotes cross-scale communication through stacked BiFE modules, and significantly improves resolution consistency and semantic coherence; then, a dynamic multi-scale feature fusion module (DyMFF) is proposed to synthesize consistent representations of learned features, which adaptively fuses semantic information of all branches to ensure robust and context-aware road segmentation; finally, a hybrid loss function is designed to increase the integrity and spatial continuity of road structures, and actual tests show that the present application can realize high-precision road extraction.

[0010] To achieve the above-mentioned purposes, the technical solutions adopted by the present application are as follows: A high-resolution remote sensing image road extraction method based on an HRNetformer network, comprising the following steps: Step one: constructing a high-resolution remote sensing image road extraction HRNetformer network model; Step two: training the high-resolution remote sensing image road extraction HRNetformer network model constructed in step one using the training set in the high-resolution remote sensing image road extraction public dataset, and outputting the training weight file HRNetformer.pt; Step three: performing road extraction on the high-resolution remote sensing image road extraction HRNetformer network model constructed in step one using the test set in the high-resolution remote sensing image road extraction public dataset and the weight file HRNetformer.pt obtained in step two, and obtaining the road extraction result.

[0011] The high-resolution remote sensing image road extraction HRNetformer network model in the step one is based on an HRNet architecture, and includes a bidirectional feature extraction module BiFE, a context and direction perception feature enhancement module CDFPE, a dynamic multi-scale feature fusion module DyMFF, and a hybrid loss function module, wherein: the bidirectional feature extraction module BiFE is used to realize bidirectional information propagation at each level; the context and direction perception feature enhancement module CDFPE is used to fuse shallow and deep features, and to improve the modeling ability of the network for direction patterns and context information through local and global attention mechanisms; the dynamic multi-scale feature fusion module DyMFF is used to adaptively integrate features from each branch to generate consistent semantic representations; and the hybrid loss function module is used to constrain the model training process and enhance the integrity and spatial continuity of the road structure. The input high-resolution remote sensing image is first extracted by an initial convolution module to obtain low-level features, and the obtained results are sent in parallel to a multi-scale branch, in which a bidirectional feature extraction module BiFE is introduced to realize bidirectional information propagation at each level, and the output of the bidirectional feature extraction module BiFE is input to a context and direction perception feature enhancement module CDFPE to fuse shallow and deep features and improve the modeling ability of the network for direction patterns and context information through local and global attention mechanisms; finally, adaptive fusion of multi-branch features is completed in a dynamic multi-scale feature fusion module DyMFF, and optimization is performed in combination with a hybrid loss function to realize accurate extraction of roads in high-resolution remote sensing images.

[0012] The bidirectional feature extraction module BiFE includes a global attention module GAM, a depth separation convolution module, a self-attention module, two multi-head self-attention modules MHSA, a multi-layer perception module MLP, an attention gate module, and a layer normalization module LayerNorm; wherein the input feature map is first extracted by the global attention module GAM to obtain global context information, and the feature map is obtained by passing through the depth separation convolution module and the self-attention module in parallel to obtain local spatial features and global dependency relationships, and the feature map is obtained; then, the feature map and the feature map are sent to the two multi-head self-attention modules MHSA for cross-channel and cross-space interaction modeling to extract multi-scale context features, and the fused feature is obtained through the attention gate module; on this basis, the fused feature is nonlinearly mapped and reorganized by the multi-layer perception module MLP; and the feature map together into a layer normalization module (LayerNorm) to ensure numerical stability and consistency of feature distribution, so as to realize efficient interaction and enhanced expression of bidirectional information. The bidirectional feature extraction module BiFE is expressed as follows:

[0013]

[0014]

[0015]

[0016] wherein, denotes a self-attention module, denotes a layer normalization module, denotes a multi-layer perception, denotes an attention gate, denotes a multi-head self-attention module, denotes a depthwise separable convolution module, denotes a global attention module, denotes a module output feature map, denotes an input feature map of the bidirectional feature extraction module BiFE.

[0017] The context and direction perception feature enhancement module CDFPE comprises a Window Attention module, a global attention module GAM, two feature pyramid networks FPN, two multi-head self-attention modules MHSA, and an attention gate. The input of the module comes from the output feature map of the bidirectional feature extraction module BiFE , specifically, the input feature map is first subjected to window-level local attention modeling via the Window Attention module to extract spatial neighborhood information; meanwhile, another branch input shallow feature map at the same resolution is subjected to global context perception feature acquisition via the global attention module GAM; the two features and enter the feature pyramid network FPN respectively to realize integration of semantic features at different scales, to obtain feature map and feature map ; then, the two multi-head self-attention modules MHSA are used to model deep dependencies of feature map and feature map respectively; finally, the feature maps and The screening and weighting are performed in an attention gate module (Attention Gate), and final context and direction perception features are output , so as to enhance the direction sensitivity and context expression ability of the road structure while maintaining the multi-scale consistency The context and direction perception feature perception enhancement module CDFPE is expressed as follows:

[0018]

[0019] .

[0020] The dynamic multi-scale feature fusion module DyMFF includes four convolutional layers with a kernel of 3 , one convolutional layer with a kernel of 1 , one weighting module Weight, one feature splicing module , five Batch Normalization (BN) layers and five ReLU activation functions; wherein the input of the module comes from the output feature map of the four bidirectional feature extraction modules BiFE ; feature map First, input the first convolutional layer with a kernel of 3 , then input the output of the first convolutional layer with a kernel of 3 to the first Batch Normalization (BN) layer, and finally input the output of the first Batch Normalization (BN) layer to the first ReLU activation function for preliminary feature extraction and normalization processing to obtain a feature map ; feature map First, input the second convolutional layer with a kernel of 3 , then input the output of the second convolutional layer with a kernel of 3 to the second Batch Normalization (BN) layer, and finally input the output of the second Batch Normalization (BN) layer to the second ReLU activation function for preliminary feature extraction and normalization processing to obtain a feature map ; feature map First, input the third convolutional layer with a kernel of 3 , then input the output of the third convolutional layer with a kernel of 3 The output of the third Batch Normalization (BN) layer is input into a third ReLU activation function for preliminary feature extraction and normalization processing to obtain a feature map ; feature map First, a fourth convolutional layer with a kernel of 3 is input Then, the output of the fourth convolutional layer with a kernel of 3 is input into a fourth Batch Normalization (BN) layer, and finally the output of the fourth Batch Normalization (BN) layer is input into a fourth ReLU activation function for preliminary feature extraction and normalization processing to obtain a feature map ; The feature map is input into a weighting Weight module to obtain a weighted feature map The feature map is input into a weighting Weight module to obtain a weighted feature map The feature map is input into a weighting Weight module to obtain a weighted feature map The feature map is input into a weighting Weight module to obtain a weighted feature map ; wherein the weighting Weight module comprises a global average pooling module GAP, a multi-layer perceptron MLP, and a Sigmoid activation function, the feature map , the feature map , the feature map , and the feature map are sequentially input into the global average pooling module GAP, the multi-layer perceptron MLP, and the Sigmoid activation function to calculate weighting coefficients, thereby obtaining four weighting coefficients , , and Then, the input feature map is multiplied by the weighting coefficient to obtain a weighted feature map ; the input feature map is multiplied by the weighting coefficient to obtain a weighted feature map ; the input feature map is multiplied by the weighting coefficient to obtain a weighted feature map ; and the input feature map is multiplied by the weighting coefficient The weighted feature map is obtained by multiplication ; then the weighted feature map , the feature map , the feature map and the feature map are fused in the feature splicing module , and the fused features are sequentially subjected to nonlinear enhancement and numerical stabilization by a fifth ReLU activation function and a fifth Batch Normalization (BN) layer, and finally subjected to channel compression and feature reorganization by a convolution layer with a kernel of 1 to obtain the final dynamic fusion feature output , realizing adaptive integration and consistent expression of multi-scale semantic information. The dynamic multi-scale feature fusion module DyMFF is represented as follows:

[0021]

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

[0030] wherein, is global average pooling, , , and represent weighting coefficients, MLP represents a multi-layer perceptron, and BN represents a Batch Normalization layer.

[0031] The cross-entropy loss and the adversarial loss function are combined as the mixed loss function, which is used to evaluate the convergence of the network, and the mixed loss function is represented as follows:

[0032]

[0033] wherein, is a cross-entropy loss function, is an adversarial loss function, is a pixel point of the neighborhood.

[0034] In the step two, the training rounds are set to be ≥ 300, and the batch size batch_size ≥ 32.

[0035] In the step three, the batch size batch_size ≥ 16.

[0036] The application further provides a high-resolution remote sensing image road extraction system based on an HRNetformer network, comprising: a model construction module, which is used for constructing a high-resolution remote sensing image road extraction HRNetformer network model; a model training module, which is used for training the high-resolution remote sensing image road extraction HRNetformer network model by using a training set in a high-resolution remote sensing image road extraction public data set, and outputting a training weight file HRNetformer.pt; a road extraction module, which is used for performing road extraction on the high-resolution remote sensing image road extraction HRNetformer network model by using a test set in the high-resolution remote sensing image road extraction public data set and the weight file HRNetformer.pt, and obtaining a road extraction result.

[0037] The application further provides a high-resolution remote sensing image road extraction device based on an HRNetformer network, comprising: a memory, which stores a computer program of the high-resolution remote sensing image road extraction method based on the HRNetformer network and is a computer-readable device; a processor, which is used for executing the computer program to realize the high-resolution remote sensing image road extraction method based on the HRNetformer network.

[0038] The application further provides a computer-readable storage medium, which stores a computer program, and the computer program can realize the high-resolution remote sensing image road extraction method based on the HRNetformer network when executed by a processor.

[0039] Compared with the prior art, the application has the following beneficial effects: 1. The application proposes a bidirectional feature extraction module BiFE, which constructs a bidirectional feature flow mechanism from shallow to deep, realizes sufficient interaction and fusion of context information. The bidirectional feature extraction module BiFE can effectively capture high-level semantic features while retaining fine-grained spatial details, thus showing significant advantages in road edge recognition and slender target continuity. This design not only improves the problem of information loss in the process of one-way feature transmission, but also improves the accuracy and structural integrity of the road extraction result.

[0040] 2. The application further designs a context and direction perception feature enhancement module CDFPE, which comprehensively uses global attention and multi-head self-attention mechanism to enhance the network's modeling ability of multi-scale directional features. This module can explicitly model the directional features (such as bending, branching, and crossing) of roads in high-resolution remote sensing images, enabling the model to have stronger geometric structure perception ability in complex scenes. The CDFPE module enhances direction perception and context association modeling, which not only improves the extraction accuracy of road targets, but also significantly improves the connectivity and detail retention ability of road structures.

[0041] 3. To solve the challenges of scale difference and information redundancy in multi-scale feature fusion process, the application proposes a dynamic multi-scale feature fusion module DyMFF. This module uses a weighted Weight module to adaptively assign weights to different levels of features, highlighting key information and suppressing redundant information. At the same time, the dynamic multi-scale feature fusion module DyMFF improves semantic consistency through cross-scale feature interaction, so that it can still maintain clear boundaries and structural integrity of road targets in complex backgrounds. Compared with traditional static fusion methods, the dynamic multi-scale feature fusion module DyMFF significantly enhances the complementarity and context relevance between features, effectively improving the robustness and generalization ability of road extraction.

[0042] 4. To solve the problems of easy breaking, edge blur and local missing in road extraction tasks, the application designs a hybrid loss function, which can not only improve the accuracy of pixel-level prediction, but also enhance the overall integrity and spatial connectivity of road targets through structure constraints. In addition, the spatial continuity constraint can effectively suppress noise points and isolated fragments, making the final extraction result more consistent with the real topological structure of the road, thus realizing more reliable and accurate high-resolution road extraction.

[0043] In summary, the application realizes the collaborative modeling of details and semantics, the explicit modeling of complex direction structure, the adaptive fusion of multi-scale features, and the structure topology constraint optimization of the results by introducing four innovative designs of bidirectional feature extraction module BiFE, context and direction perception feature enhancement module CDFPE, dynamic multi-scale feature fusion module DyMFF and hybrid loss function. The series of innovations not only effectively improve the precision, connectivity and detail preservation ability of road extraction, but also enhance the robustness and generalization ability of the model under complex background and high-resolution remote sensing images. It provides an efficient, reliable and application valuable technical solution for high-precision road extraction task. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a principle flow schematic diagram of an embodiment of the application.

[0045] Figure 2 is a network structure diagram of an embodiment of the application.

[0046] Figure 3 is a bidirectional feature extraction module BiFE structure diagram of an embodiment of the application.

[0047] Figure 4 is a context and direction perception feature enhancement module CDFPE structure diagram of an embodiment of the application.

[0048] Figure 5 is a dynamic multi-scale feature fusion module DyMFF structure diagram of an embodiment of the application.

[0049] Figure 6 is a road extraction precision comparison diagram of the application and the prior art. DETAILED DESCRIPTION

[0050] The technical solutions of the application will be further described in detail below with reference to the drawings.

[0051] As shown in Figure 1 , a high-resolution remote sensing image road extraction method, system, device and medium based on HRNetformer network, wherein the high-resolution remote sensing image road extraction network HRNetformer (HRNetTransformer) involved is improved on the basis of the existing high-resolution remote sensing image road extraction network HRNet. The application can be used for high-resolution remote sensing image road extraction.

[0052] The steps include: Step one: as shown in Figure 2 , a high-resolution remote sensing image road extraction HRNetformer network model is constructed; The high-resolution remote sensing image road extraction HRNetformer network model in step one is based on the HRNet architecture and includes a bidirectional feature extraction module (BiFE), a context and orientation-aware feature enhancement module (CDFPE), a dynamic multi-scale feature fusion module (DyMFF), and a hybrid loss function module. Specifically: the Bidirectional feature extraction module (BiFE) enables bidirectional information propagation at each level, enhancing the richness and diversity of feature representations; the CDFPE module fuses shallow and deep features and improves the network's ability to model directional patterns and contextual information through local and global attention mechanisms; the DyMFF module adaptively integrates features from various branches to generate consistent semantic representations; and the hybrid loss function module constrains the model training process, enhancing the integrity and spatial continuity of the road structure. The input high-resolution remote sensing image first undergoes an initial convolutional module to extract low-level features. The results are then fed into a multi-scale branch in parallel. In this multi-scale branch, a bidirectional feature extraction module (BiFE) is introduced to achieve bidirectional information propagation at each level, enhancing the richness and diversity of feature representations. Simultaneously, the output of the BiFE module is input into the context and orientation-aware feature enhancement module (CDFPE) to fuse shallow and deep features. Local and global attention mechanisms are used to improve the network's ability to model orientation patterns and contextual information. Finally, the adaptive fusion of multi-branch features is completed in the dynamic multi-scale feature fusion module (DyMFF), and optimized using a hybrid loss function to achieve accurate road extraction from the high-resolution remote sensing image. This significantly improves the accuracy and robustness of road extraction, especially in complex high-resolution remote sensing scenarios.

[0053] Step 101: As Figure 3 As shown, the bidirectional feature extraction module BiFE includes a global attention module (GAM), a depthwise separable convolution module, a self-attention module, two multi-head self-attention modules (MHSA), a multilayer perceptron (MLP), an attention gate module, and a layer normalization module (LayerNorm). The input feature map is first processed by the global attention module (GAM) to extract global context information, thus obtaining the feature map. The feature map is obtained by passing the deep separation convolution module and the self-attention module in parallel to obtain local spatial features and global dependencies. Subsequently, the feature map and feature map The data are fed into two multi-head self-attention (MHSA) modules to perform cross-channel and cross-space interaction modeling, extracting multi-scale contextual features. Simultaneously, the fused features are obtained through an attention gating module. Based on this, the features after fusion Nonlinear mapping and feature reorganization via multi-layer perceptron (MLP); and input into a layer normalization module (LayerNorm) together to ensure numerical stability and consistency of feature distribution, thereby realizing efficient interaction and enhanced expression of bidirectional information. The bidirectional feature extraction module BiFE is represented as follows:

[0054]

[0055]

[0056]

[0057] wherein, represents a self-attention module, represents a layer normalization module, represents a multi-layer perceptron, represents an attention gate module, represents a multi-head self-attention module, represents a depthwise separable convolution module, represents a global attention module, represents a module output feature map, represents an input feature map of the bidirectional feature extraction module BiFE; Step 102: As shown in Figure 4 , the context and direction perception feature enhancement module CDFPE includes a Window Attention module, a global attention module GAM, two feature pyramid networks FPN, two multi-head self-attention modules MHSA, and an attention gate module; wherein the input of the module comes from the output feature map of the bidirectional feature extraction module BiFE, specifically, the input feature map is first subjected to window-level local attention modeling via the Window Attention module to extract spatial neighborhood information; at the same time, the other branch input shallow feature map under the same resolution, obtains global context perception feature via the global attention module GAM; the two features and enter the feature pyramid network FPN respectively to realize integration of semantic features of different scales, and obtain feature maps and ; then, the two multi-head self-attention modules MHSA are used to respectively process the feature maps and ​Perform deep dependency modeling; finally, analyze the feature maps after deep dependency modeling. and feature map The attention gate module filters and weights the data to output the final context and orientation-aware features. This enhances the directional sensitivity and contextual expressiveness of the road structure while maintaining multi-scale consistency. The Context and Direction Aware Feature Awareness Enhancement Module (CDFPE) is represented by the following formula:

[0058]

[0059] ; Step 103: As Figure 5 As shown, the dynamic multi-scale feature fusion module DyMFF includes four convolutional layers with a kernel size of 3. A convolutional layer with a kernel of 1 A weighted weight module and a feature concatenation module. The module consists of five Batch Normalization (BN) layers and five ReLU activation functions; the input to this module comes from the output integral graphs of four bidirectional feature extraction modules (BiFE). ; Feature map First, input the first convolutional layer with a kernel of 3. Then, the first convolutional layer with a kernel of 3... The output of the first Batch Normalization (BN) layer is fed into the first Batch Normalization (BN) layer. Finally, the output of the first Batch Normalization (BN) layer is fed into the first ReLU activation function for preliminary feature extraction and normalization, resulting in a feature map. ; Feature map First, input the second convolutional layer with a kernel of 3. Then, a second convolutional layer with a kernel of 3 is added. The output of the first layer is fed into the second Batch Normalization (BN) layer. Finally, the output of the second Batch Normalization (BN) layer is fed into the second ReLU activation function for preliminary feature extraction and normalization, resulting in a feature map. ; Feature map First, input the third convolutional layer with a kernel of 3. Then, a third convolutional layer with a kernel of 3 is added. The output of the fourth Batch Normalization (BN) layer is input into a fourth ReLU activation function, to perform preliminary feature extraction and normalization processing, to obtain a feature map ; feature map First, a fourth convolutional layer with a kernel of 3 is input Then, the output of the fourth convolutional layer with a kernel of 3 is input into a fourth Batch Normalization (BN) layer, and finally the output of the fourth Batch Normalization (BN) layer is input into a fourth ReLU activation function, to perform preliminary feature extraction and normalization processing, to obtain a feature map ; The feature map is input into a weighting Weight module to obtain a weighted feature map The feature map is input into a weighting Weight module to obtain a weighted feature map The feature map is input into a weighting Weight module to obtain a weighted feature map The feature map is input into a weighting Weight module to obtain a weighted feature map ; wherein the weighting Weight module comprises a global average pooling module GAP, a multi-layer perceptron MLP and a Sigmoid activation function, the feature map , the feature map , the feature map and the feature map are sequentially input into the global average pooling module GAP, the multi-layer perceptron MLP and the Sigmoid activation function, to calculate the weighting coefficients, to obtain four weighting coefficients , , and Then, the input feature map is multiplied by the weighting coefficient to obtain a weighted feature map ; the input feature map is multiplied by the weighting coefficient to obtain a weighted feature map ; the input feature map is multiplied by the weighting coefficient to obtain a weighted feature map ; the input feature map is multiplied by the weighting coefficient The weighted feature map is obtained by multiplication The weighted feature map is obtained by multiplication The weighted feature map is obtained by multiplication The weighted feature map is obtained by multiplication The weighted feature map is obtained by multiplication In the feature splicing module The fused features are sequentially subjected to a fifth ReLU activation function and a fifth Batch Normalization (BN) layer for non-linear enhancement and numerical stabilization, and finally subjected to a convolution layer with a kernel of 1 Channel compression and feature reorganization are performed to obtain the final dynamic fusion feature output , realizing adaptive integration and consistent expression of multi-scale semantic information. The dynamic multi-scale feature fusion module DyMFF is represented as follows:

[0060]

[0061]

[0062]

[0063]

[0064]

[0065]

[0066]

[0067]

[0068]

[0069] wherein, is a global average pooling, , , and represent a weighting coefficient, MLP represents a multi-layer perceptron, and BN represents a Batch Normalization layer.

[0070] Step 104: The mixed loss function obtained by combining the cross-entropy loss and the adversarial loss is used as the loss function of the network, and the convergence of the network is evaluated, and the loss function is represented as follows:

[0071]

[0072] wherein, is a cross-entropy loss function, is a pixel point of a neighborhood.

[0073] Step two: set the training round to be >= 300, the batch size batch_size >= 32, use the training set in the high-resolution remote sensing image road extraction public data set to train the high-resolution remote sensing image road extraction HRNetformer network model, and output the training weight file HRNetformer.pt; Step three: the batch size batch_size >= 16, use the test set in the high-resolution remote sensing image road extraction public data set and the weight file HRNetformer.pt obtained in step two to perform road extraction on the high-resolution remote sensing image road extraction HRNetformer network model, and obtain the road extraction result.

[0074] The application also provides a high-resolution remote sensing image road extraction system based on the HRNetformer network, comprising: A model construction module is used to realize the construction of the high-resolution remote sensing image road extraction HRNetformer network model in step one. A model training module is used to realize the training of the high-resolution remote sensing image road extraction HRNetformer network model constructed in step one using the training set in the high-resolution remote sensing image road extraction public data set, and output the training weight file HRNetformer.pt. A road extraction module is used to realize the road extraction of the high-resolution remote sensing image road extraction HRNetformer network model constructed in step one using the test set in the high-resolution remote sensing image road extraction public data set and the weight file HRNetformer.pt obtained in step two, and obtain the road extraction result.

[0075] The application also provides a high-resolution remote sensing image road extraction device based on the HRNetformer network, comprising: A memory is used to store the computer program of the high-resolution remote sensing image road extraction method based on the HRNetformer network, and the computer program is readable by the device. A processor is used to execute the computer program and realize the high-resolution remote sensing image road extraction method based on the HRNetformer network.

[0076] The application further provides a computer-readable storage medium, which stores a computer program, and the computer program can realize the high-resolution remote sensing image road extraction method based on the HRNetformer network when executed by a processor.

[0077] The effect of the application is further described below in combination with a simulation experiment.

[0078] 1. Simulation experiment conditions The hardware platform of the simulation experiment of the application is as follows: the processor is Intel i9-13900K, the main frequency is 3.0 GHz, and the running memory is 64 G.

[0079] The software platform of the simulation experiment platform of the application is as follows: Windows 11 operating system and PyCharm, PyTorch2.1.0, and CUDA 12.1.

[0080] 2. Simulation steps The training set in the high-resolution remote sensing image road extraction public data set is input into the high-resolution remote sensing image road extraction HRNetformer network model for optimization training, and the training process is as follows: the input image is first subjected to synchronous processing of the multi-branch high and low resolution network, the shallow branch retains the fine-grained edge details of the road, the high-level branch captures the global semantic and complex background information, and information complementation is realized through cross-resolution feature exchange in the training process. A hybrid loss function is used to improve the pixel-level classification accuracy and maintain the road connectivity, so as to realize accurate and continuous extraction of the road in the high-resolution remote sensing image. The iteration number epoch is set to 300.

[0081] 3. Simulation content and result analysis The simulation experiment of the application is to perform road extraction processing on 50 high-resolution remote sensing images by using the application, and the result is as shown in Figure 6 , and the result is an average value.

[0082] The simulation effect of the application is further described below. Figure 6

[0083] Figure 6 ​is a road extraction precision comparison chart of the present application and the prior method, the comparison algorithm is Seg-Road (Tao, J.; Chen, Z.; Sun, Z.; Guo, H.; Leng, B.; Yu, Z.; Wang, Y.; He, Z.; Lei, X.; Yang, J. Seg-Road: A Segmentation Network for Road Extraction Based on Transformer and CNN with Connectivity Structures. Remote Sens[J]. 2023, 15, 1602.), and the precision evaluation index is a Dice coefficient, which is widely used in an image segmentation task and is used for measuring the similarity of a predicted area and an actual area.

[0084] As shown in Figure 6 , it can be seen that the Dice of the present application is higher than that of Seg-Road. The experimental results show that the high-resolution remote sensing image road extraction method based on the HRNetformer network proposed in the present application can meet the requirement of high-precision road extraction.

Claims

1. A method for road extraction from high-resolution remote sensing images based on HRNetformer network, characterized in that, Includes the following steps: Step 1: Construct the HRNetformer network model for road extraction from high-resolution remote sensing images; Step 2: Train the HRNetformer network model for road extraction from high-resolution remote sensing images constructed in Step 1 using the training set in the publicly available high-resolution remote sensing image road extraction dataset, and output the training weight file HRNetformer.pt; Step 3: Use the test set from the public dataset for high-resolution remote sensing image road extraction and the weight file HRNetformer.pt obtained in Step 2 to perform road extraction on the HRNetformer network model for high-resolution remote sensing image road extraction constructed in Step 1, and obtain the road extraction results.

2. The method for road extraction from high-resolution remote sensing images based on HRNetformer network according to claim 1, characterized in that, The high-resolution remote sensing image road extraction HRNetformer network model in step one is based on the HRNet architecture and includes a bidirectional feature extraction module (BiFE), a context and orientation-aware feature enhancement module (CDFPE), a dynamic multi-scale feature fusion module (DyMFF), and a hybrid loss function. Specifically: the Bidirectional feature extraction module (BiFE) is used to achieve bidirectional information propagation at each level; the CDFPE module is used to fuse shallow and deep features and improve the network's ability to model directional patterns and contextual information through local and global attention mechanisms; the DyMFF module is used to adaptively integrate features from each branch to generate consistent semantic representations; and the hybrid loss function module is used to constrain the model training process and enhance the integrity and spatial continuity of the road structure. The input high-resolution remote sensing image first extracts low-level features through an initial convolutional module. The results are then fed into a multi-scale branch in parallel. In the multi-scale branch, a bidirectional feature extraction module (BiFE) is introduced to achieve bidirectional information propagation at each level. Simultaneously, the output of the BiFE module is input to the context and orientation-aware feature enhancement module (CDFPE) to fuse shallow and deep features. Furthermore, local and global attention mechanisms are used to enhance the network's ability to model orientation patterns and contextual information. Finally, the adaptive fusion of multi-branch features is completed in the dynamic multi-scale feature fusion module (DyMFF), and optimization is performed using a hybrid loss function to achieve accurate extraction of roads from the high-resolution remote sensing image.

3. The method for road extraction from high-resolution remote sensing images based on HRNetformer network according to claim 2, characterized in that, The bidirectional feature extraction module BiFE includes a global attention module (GAM), a depthwise separable convolution module, a self-attention module, two multi-head self-attention modules (MHSA), a multilayer perceptron (MLP), an attention gate module, and a layer normalization module (LayerNorm). The input feature map is first processed by the global attention module (GAM) to extract global context information, thus obtaining the feature map. The feature map is obtained by passing the deep separation convolution module and the self-attention module in parallel to obtain local spatial features and global dependencies. Subsequently, the feature map and feature map The data are fed into two multi-head self-attention (MHSA) modules to perform cross-channel and cross-space interaction modeling, extracting multi-scale contextual features. Simultaneously, the fused features are obtained through an attention gating module. Based on this, the features after fusion Nonlinear mapping and feature reconstruction are performed via a multilayer perceptron (MLP); and combined with the feature map Together they are input into the LayerNorm module to ensure numerical stability and consistency of feature distribution, thereby achieving efficient interaction of bidirectional information and enhanced expression; The bidirectional feature extraction module BiFE is represented by the following formula: in, This represents a self-attention module. Presentation layer normalization module, This represents a multilayer perceptron. This refers to the Attention Gate module. This indicates a multi-head self-attention module. This indicates a depthwise separable convolutional module. This represents the global attention module. This indicates that the module outputs a feature map. This represents the input feature map of the Bi-directional feature extraction module BiFE.

4. The method for road extraction from high-resolution remote sensing images based on HRNetformer network according to claim 2, characterized in that, The Context and Direction Aware Feature Enhancement Module (CDFPE) includes a Window Attention module, a Global Attention module (GAM), two Feature Pyramid Networks (FPN), two Multi-Head Self-Attention Modules (MHSA), and an Attention Gate module. The input to this module comes from the output feature map of the Bidirectional Feature Extraction Module (BiFE). Specifically, the input feature map First, the Window Attention module performs window-level local attention modeling to extract spatial neighborhood information; simultaneously, another branch inputs a shallow feature map at the same resolution, which is then processed by the Global Attention (GAM) module to obtain global context-aware features; these two feature paths... and The features are fed into the Feature Pyramid Network (FPN) to integrate semantic features at different scales, resulting in feature maps. and feature map Subsequently, the feature maps are processed by two multi-head self-attention modules (MHSA). and feature map Perform deep dependency modeling; finally, analyze the feature maps after deep dependency modeling. and feature map The attention gate module filters and weights the data to output the final context and orientation-aware features. This enhances the directional sensitivity and contextual expressiveness of the road structure while maintaining multi-scale consistency. The Context and Direction Aware Feature Awareness Enhancement Module (CDFPE) is represented by the following formula: 。 5. The method for road extraction from high-resolution remote sensing images based on HRNetformer network according to claim 2, characterized in that, The dynamic multi-scale feature fusion module DyMFF includes four convolutional layers with a kernel size of 3. A convolutional layer with a kernel of 1 A weighted weight module and a feature concatenation module. It consists of five Batch Normalization (BN) layers and five ReLU activation functions; the input to this module comes from the output integral graphs of four bidirectional feature extraction modules (BiFE). ; Feature map First, input the first convolutional layer with a kernel of 3. Then, the first convolutional layer with a kernel of 3... The output of the first Batch Normalization (BN) layer is fed into the first Batch Normalization (BN) layer. Finally, the output of the first Batch Normalization (BN) layer is fed into the first ReLU activation function for preliminary feature extraction and normalization, resulting in a feature map. ; Feature map First, input the second convolutional layer with a kernel of 3. Then, a second convolutional layer with a kernel of 3 is added. The output of the first layer is fed into the second Batch Normalization (BN) layer, and finally the output of the second Batch Normalization (BN) layer is fed into the second ReLU activation function for preliminary feature extraction and normalization, resulting in a feature map. ; Feature map First, input the third convolutional layer with a kernel of 3. Then, a third convolutional layer with a kernel of 3 is added. The output of the first layer is fed into the third Batch Normalization (BN) layer, and finally the output of the third Batch Normalization (BN) layer is fed into the third ReLU activation function for preliminary feature extraction and normalization, resulting in a feature map. ; Feature map First, input the fourth convolutional layer with a kernel of 3. Then, the fourth convolutional layer with a kernel of 3... The output of the first layer is fed into the fourth Batch Normalization (BN) layer, and finally the output of the fourth Batch Normalization (BN) layer is fed into the fourth ReLU activation function for preliminary feature extraction and normalization, resulting in a feature map. ; feature map The input is fed into the weighted weight module to obtain the weighted feature map. ; feature map The input is fed into the weighted weight module to obtain the weighted feature map. ; feature map The input is fed into the weighted weight module to obtain the weighted feature map. ; feature map The input is fed into the weighted weight module to obtain the weighted feature map. The weighted weight module includes a global average pooling module (GAP), a multilayer perceptron (MLP), and a sigmoid activation function. The feature map... Feature map Feature map and feature map The inputs are sequentially fed into the Global Average Pooling (GAP) module, the Multilayer Perceptron (MLP), and the Sigmoid activation function to calculate the weighting coefficients, resulting in four weighting coefficients. , , and Then, the input feature map with weighting coefficients Multiply to obtain the weighted feature map. Input feature map with weighting coefficients Multiply to obtain the weighted feature map. Input feature map with weighting coefficients Multiply to obtain the weighted feature map. Input feature map with weighting coefficients Multiply to obtain the weighted feature map. Then the weighted feature map Feature map Feature map and feature map In the feature splicing module Multi-scale feature fusion is performed, and the fused features are sequentially enhanced and numerically stabilized through a fifth ReLU activation function and a fifth Batch Normalization (BN) layer, and finally passed through a convolutional layer with a kernel of 1. Channel compression and feature recombination are performed to obtain the final dynamic fusion feature output. This enables adaptive integration and consistent expression of multi-scale semantic information. The dynamic multi-scale feature fusion module DyMFF is represented by the following formula: in, For global tie pooling, , , and Indicates the weighting coefficient. This represents a multilayer perceptron, and BN represents a batch normalization layer.

6. The method for road extraction from high-resolution remote sensing images based on HRNetformer network according to claim 2, characterized in that, The cross-entropy loss and adversarial loss function are combined to form the hybrid loss function, which is used to evaluate the convergence of the network. The hybrid loss function is expressed as follows: in, Let cross-entropy be the loss function. To counteract the loss function, It is a pixel. of Neighborhood.

7. The method for road extraction from high-resolution remote sensing images based on HRNetformer network according to claim 1, characterized in that, In step two, the number of training rounds is set to ≥300, and the batch size is ≥32; In step three, the batch size is greater than or equal to 16.

8. A high-resolution remote sensing image road extraction system based on the HRNetformer network according to the method of claim 1, characterized in that, include: Model building module: Used to build HRNetformer network models for road extraction from high-resolution remote sensing images; Model training module: Used to train the HRNetformer network model for road extraction from high-resolution remote sensing images using the training set in the publicly available high-resolution remote sensing image road extraction dataset, and outputs the training weight file HRNetformer.pt; Road Extraction Module: This module is used to extract roads from the HRNetformer network model of the high-resolution remote sensing image road extraction public dataset using the test set and weight file HRNetformer.pt, and obtain the road extraction results.

9. A high-resolution remote sensing image road extraction device based on HRNetformer network, characterized in that, include: Memory: A computer program for a high-resolution remote sensing image road extraction method based on HRNetformer network as described in any one of claims 1-7, and is a computer-readable device; Processor: Used to implement the high-resolution remote sensing image road extraction method based on HRNetformer network as described in any one of claims 1-7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the high-resolution remote sensing image road extraction method based on the HRNetformer network as described in any one of claims 1-7.

Images