Road extraction method based on mixed attention mechanism and direction prior

Abstract

The invention relates to a road extraction method based on a mixed attention mechanism and direction prior, and the method comprises the following steps: obtaining a remote sensing image, inputting a trained road extraction model based on the mixed attention mechanism and the direction prior, and obtaining a road extraction result; the model comprises an encoder and a decoder, the encoder comprises a plurality of mixed attention modules connected in sequence, and in each mixed attention module, input data firstly extracts local features through a plurality of sparse attention layers connected in sequence, and finally captures global semantic association through an improved dense attention layer; the decoder comprises a direction enhancement fusion module and a plurality of lightweight mixed attention modules, and the direction enhancement fusion module performs convolution operation in a plurality of directions and obtains a fusion feature map through dynamic adjustment. Compared with the prior art, the method can ensure the continuity and accuracy of the road extraction result while improving the calculation efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a road extraction method based on a hybrid attention mechanism and direction prior. Background Art

[0002] As an important infrastructure, the real-time and accurate update of road-related information plays a key role in many practical applications, such as urban planning, traffic management, and emergency response. With the rapid development of high-resolution remote sensing technology, the application value of remote sensing images in road extraction has become increasingly prominent. The widespread popularity of high-resolution remote sensing images, on the one hand, has laid a solid data foundation for the real-time positioning and identification of road targets, and on the other hand, it has also made road extraction face more complex challenges, especially in scenes with diverse morphologies and complex semantics. For example, from country roads to urban highways, road types show significant differences and are often obscured by vegetation or interfered with by other targets. What's more tricky is that existing algorithms find it difficult to achieve high-precision extraction while ensuring computational efficiency.

[0003] In response to the above problems, researchers have conducted a lot of exploration. The existing road extraction methods can be mainly divided into three categories: rule-based methods, traditional machine learning methods, and deep learning methods. Among them, rule-based methods mainly rely on the color and texture information of the image, and detect road edges through morphological and texture analysis. Such methods have poor adaptability in complex scenes, especially when the road is occluded or merged with the background. Traditional machine learning methods usually use feature engineering and classifiers for target detection, but they rely heavily on artificially designed features and are difficult to fully capture the overall and local features of the road. In recent years, deep learning methods have gradually become the mainstream technology for road extraction. The semantic segmentation model based on convolutional neural networks can significantly improve the extraction accuracy through end-to-end training. However, when dealing with complex scenes, such methods are difficult to establish the topological relationship between the road sections separated by occlusion, and face the challenge of insufficient semantic information. At the same time, there is a problem of high computational cost when processing large-scale data. Therefore, it is necessary to design a road extraction method that can effectively capture the global and local semantic features of road targets while improving computational efficiency, and ensure the continuity and accuracy of road extraction results. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a road extraction method based on a hybrid attention mechanism and directional prior, which can effectively capture the global and local semantic features of road targets while improving computational efficiency, and ensure the continuity and accuracy of road extraction results.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] The present invention provides a road extraction method based on a hybrid attention mechanism and a direction prior, comprising the following steps:

[0007] Obtain remote sensing images, input the trained road extraction model based on hybrid attention mechanism and direction prior, and obtain the road extraction results;

[0008] The road extraction model based on the hybrid attention mechanism and direction prior includes an encoder and a decoder; the encoder includes a plurality of sequentially connected hybrid attention modules, the remote sensing image is firstly divided into blocks to generate a plurality of feature blocks, and then processed by a plurality of hybrid attention modules to generate a multi-scale feature map, in each hybrid attention module, the input data is firstly extracted with local features through a plurality of sequentially connected sparse attention layers, and finally the global semantic association is captured through an improved dense attention layer, in which the calculation method of self-attention is reconstructed into a combination of linear activation function and local convolution;

[0009] The decoder includes a direction enhancement fusion module and multiple lightweight hybrid attention modules. The direction enhancement fusion module takes the multi-scale feature map as input, performs convolution operations in multiple directions, obtains corresponding direction information, and fuses all direction information by dynamically adjusting the weights of each direction information to obtain a fused feature map; the fused feature map is further refined by multiple lightweight hybrid attention modules in turn, and finally a pixel-level road extraction result is generated by upsampling. Each lightweight hybrid attention module includes a sparse attention layer and a dense attention layer.

[0010] Furthermore, in the hybrid attention module, the sparse attention layer realizes the dynamic extraction of local features through deformable convolution. The specific expression is as follows:

[0011]

[0012] Among them, y1 is the output of the sparse attention layer, N is the number of sampling points, and w n is the weight of the nth sampling point, x(p0+p n +Δp n ) is the characteristic value of the sampling point position, p0 is the current pixel position, p n represents the nth position in the predefined grid sampling {(-1,-1),(-1,0),...,(0,+1),...,(+1,+1)}, Δp n is the position offset obtained through learning, Δm n It is a dynamically adjusted modulus used to control the addition value at different sampling points.

[0013] Furthermore, in the hybrid attention module, the specific expression of the improved dense attention layer is as follows:

[0014]

[0015] Among them, FLA-Attention is the output of the improved dense attention layer, Q, K, V represent query, key and value respectively, σ is a nonlinear activation function, Conv represents the deep convolution operation of channel grouping, and ⊙ represents the element-wise product operation.

[0016] Furthermore, the nonlinear activation function σ adopts ReLU 3 .

[0017] Furthermore, in the directional enhancement fusion module, the specific expressions for performing convolution operations and fusion in multiple directions are as follows:

[0018]

[0019] Among them, y2 is the fusion direction information, N d is the number of directions, N d0 is the number of sampling points in each direction, w i,j represents the convolution weight of the jth sampling point in direction i, p0 is the current pixel position, x(p0+Δp i,j ) is the characteristic value of the offset sampling point predefined based on the direction, Δp i,j Indicates the relative position offset of the jth sampling point in direction i, b i is the bias in direction i.

[0020] Furthermore, the relative position offset Δp of the jth sampling point in direction i is ij The specific expression is as follows:

[0021]

[0022] Among them, r is the distance of the sampling point, δ is the expansion coefficient, θ i is the angle of direction i, θ i Uniformly distributed in [0,2π).

[0023] Furthermore, in the lightweight hybrid attention module, the channel u of the input feature map c Make the following adjustments:

[0024]

[0025] s c =σ(w2ξ(w1z c ))

[0026] in, is the adjusted channel, σ is the activation function, w1 and w2 are two one-dimensional fully connected layers, used for dimensionality reduction and dimensionality increase respectively, z c is u after global average pooling c , ξ is a nonlinear activation operation.

[0027] Furthermore, u c Perform global average pooling to obtain z c The specific expression is as follows:

[0028]

[0029] Where H and W are u c The height and width of the

[0030] Furthermore, the road extraction model based on the hybrid attention mechanism and direction prior adopts a cross entropy loss function and an AdamW optimizer during training.

[0031] Furthermore, the road extraction model based on the hybrid attention mechanism and direction prior introduces a phased learning rate decay strategy and a data enhancement method during the training process.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The present invention proposes a road extraction method based on a hybrid attention mechanism and directional priors, designs a road extraction model based on a hybrid attention mechanism and directional priors, and obtains road extraction results according to remote sensing images; the model includes an encoder and a decoder, wherein the encoder includes a plurality of sequentially connected hybrid attention modules, the remote sensing image is firstly divided into blocks to generate a plurality of feature blocks, and then processed by a plurality of hybrid attention modules to generate a multi-scale feature map, in each hybrid attention module, the input data is firstly extracted with local features through a plurality of sequentially connected sparse attention layers, and finally the global semantic association is captured through an improved dense attention layer, the hybrid attention module can realize the full-image scale aggregation of pixel association, and provides more comprehensive support for feature modeling, specifically, the sparse attention layer can dynamically adjust the sampling position, capture the local features with elastic adaptability, so as to adapt to the diversity of road forms in complex scenes, in the improved dense attention layer, the calculation method of self-attention is reconstructed into a combination of linear activation function and local convolution, which can reduce the computational complexity and enhance the efficiency of feature expression while maintaining the global modeling capability;

[0034] The decoder includes a direction-enhanced fusion module and multiple lightweight hybrid attention modules. In high-resolution remote sensing images, roads have linear connections and significant directionality. The direction-enhanced fusion module takes a multi-scale feature map as input, performs convolution operations in multiple directions, obtains corresponding direction information, and fuses all direction information by dynamically adjusting the weights of each direction information to obtain a fused feature map, which can significantly improve the model's modeling ability for road morphology information; the fused feature map is further refined by multiple lightweight hybrid attention modules in turn, and finally upsampling is used to generate pixel-level road extraction results; while maintaining high-resolution input, the above method greatly improves the efficiency and accuracy of the model, providing an efficient and robust solution to the problem of road extraction in complex remote sensing images.

[0035] 2. In the present invention, the road extraction model based on the hybrid attention mechanism and direction prior adopts the cross entropy loss function and AdamW optimizer during the training process, and introduces a phased learning rate decay strategy and data enhancement method, which effectively improves the training efficiency and generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of the structure of the road extraction model based on the hybrid attention mechanism and direction prior;

[0037] Figure 2 It is a comparison diagram of the effective receptive field (ERF) of different backbone networks (untrained state) at each stage.

[0038] Among them, (2a), (2b), and (2c) respectively show the effective receptive fields of the three stages of InternImage using sparse attention, (2d), (2e), and (2f) respectively show the effective receptive fields of the three stages of Swin Transformer using window-based dense attention, and (2g), (2h), and (2i) respectively show the effective receptive fields of the three stages using HDRoad (i.e., the method proposed in this embodiment);

[0039] Figure 3 This is a schematic diagram of the sampling position of the first quadrant of the directional enhanced convolution.

[0040] Where δ is the expansion coefficient, Δp represents the relative position offset, θ is the angle of the direction, (3a) corresponds to δ = 1, (3b) corresponds to δ = 2;

[0041] Figure 4 The results of each model on the DeepGlobe dataset are compared.

[0042] Among them, the rows from top to bottom are: rural areas, densely populated urban areas and urban-rural fringe areas; the columns are: (4a) is the input image (test set), (4b) is the true value label, (4c) is D-LinkNet, (4d) is U-Net, (4e) is DeepLabv3+, (4f) is Swin-T (combined with Upernet), (4g) is InternImage-T (combined with Upernet), (4h) is HDRoad-Mobile (a lightweight version of the method in this paper), and (4i) is HDRoad-Standard (a standard version of the method in this paper);

[0043] Figure 5 is the structural diagram of the encoder.

[0044] Among them, Image represents image, Patch Partition represents image partitioning, Patch Merging represents image block merging, Hybrid Attention Block represents hybrid attention module, Sparse Attention represents sparse attention, Dense Attention represents dense attention, Patched Map represents block map, Grouping represents grouping, Deformable Convolution represents deformable convolution, Channel Mixing represents channel mixing, LayerNormalization (LN) represents layer normalization, Multi-head represents multi-head mechanism, Focused Linear Attention represents focused linear attention, Q represents query (Query), K represents key (Key), and V represents value (Value). DETAILED DESCRIPTION

[0045] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0046] Example:

[0047] This embodiment provides a road extraction method based on a hybrid attention mechanism and direction prior, combining the encoder-decoder structure and morphological features to design a Figure 1The road extraction model (HDRoad) based on hybrid attention mechanism and directional prior is shown. In the encoder, the input image is segmented by patches and then sent to multiple hybrid attention modules for multi-scale feature extraction. The hybrid attention module combines sparse and dense attention mechanisms to capture global semantic information while retaining detail features and efficiently extract multi-scale semantic information. Subsequently, the decoder restores the multi-scale feature map output by the encoder. During the decoding process, the morphological prior of the road is embedded through the directional enhanced convolution operation, which can effectively improve the accuracy and continuity of feature restoration; in addition, multi-scale features are fused layer by layer to ensure semantic expression and spatial consistency, and finally generate pixel-level road prediction results. The specific description of the model is as follows:

[0048] Ⅰ. Encoder

[0049] In this embodiment, the encoder includes three hybrid attention modules connected in sequence, which constitute three feature extraction stages, each of which is responsible for extracting features at different resolutions. The first stage: preliminary extraction of local features of the image to generate low-resolution feature maps; the second stage: refine the features at medium resolution, taking into account global and local information; the third stage: capture the detailed features of the road morphology at high resolution. The output features of each stage are multi-scale fused by downsampling.

[0050] The formula of the hybrid attention module is as follows:

[0051] F out =DenseAttention(SparseAttention(F in ))

[0052] Among them, F in and F out Represent the input and output feature maps of the hybrid attention module, respectively.

[0053] like Figure 5 As shown in the figure, the hybrid attention module consists of multiple sparse attention layers connected in sequence and an improved dense attention layer. It is the core module of the encoder. Aiming at the characteristics of high-resolution remote sensing images, it improves the ability to extract semantic features while significantly reducing the computational complexity. Among them, the sparse attention layer is dynamically used to extract relatively localized features from the feature map, and the improved dense attention layer aggregates these local features and expands their relevance over the entire image, ultimately generating a multi-scale feature map at high resolution to provide rich feature information for the decoder.

[0054] (1) Sparse Attention Layer

[0055] The sparse attention layer realizes the dynamic extraction of local features through deformable convolution (DCNv3), which is expressed as follows:

[0056]

[0057] Among them, y1 is the output of the sparse attention layer, N is the number of sampling points, and w n is the weight of the nth sampling point, x(p0+p n +Δp n ) is the characteristic value of the sampling point position, p0 is the current pixel position, p n represents the nth position in the predefined grid sampling {(-1,-1),(-1,0),...,(0,+1),...,(+1,+1)}, Δm n is the dynamically adjusted modulus used to control the addition value at different sampling points, Δp n is the position offset learned by the existing method DCNv3, through Δp n , the sparse attention layer can dynamically adjust the sampling position and capture local features with elastic adaptability, thereby adapting to the diversity of road forms in complex scenes. Specifically, the position offset of DCNv3 is learned by applying an additional convolutional layer to the input feature map. The number of output channels of this convolutional layer is 2N (N is the number of sampling points), which represents the offset of each sampling point in the x and y directions.

[0058] (2) Improve the dense attention layer

[0059] The dense attention layer models global semantic associations through an efficient linearized self-attention mechanism, avoiding the problem in traditional Transformers where computational complexity grows with the square of the input resolution. The specific formula is as follows:

[0060]

[0061] To improve efficiency, this embodiment uses a combination of linear activation function and local convolution to reconstruct self-attention:

[0062]

[0063] Among them, FLA-Attention is the output of the improved dense attention layer, Q, K, V represent query, key and value respectively, σ is a nonlinear activation function, and ReLU is used in this embodiment 3 , Conv represents the channel grouping depth convolution operation. In this embodiment, the convolution kernel size is 5, and ⊙ represents the element-level product operation.

[0064] The above reconstruction significantly reduces the computational cost of dense attention, from O(n 2 ) is reduced to O(n), and the efficiency of feature expression is enhanced while maintaining the global modeling capability. It supports processing high-resolution image input on a single GPU and is suitable for remote sensing road extraction tasks in complex scenes.

[0065] like Figure 2 As shown, when the activated pixel is located at the central intersection of the input image, (2a), (2b), and (2c) respectively show the effective receptive fields of the three stages of InternImage using sparse attention, (2d), (2e), and (2f) respectively show the effective receptive fields of the three stages of Swin Transformer using window-based dense attention, and (2g), (2h), and (2i) respectively show the effective receptive fields of the three stages of HDRoad. Compared with the method that only uses sparse attention (such as InternImage) or the window-based dense attention method (such as Swin Transformer), the hybrid attention module proposed in this embodiment combines the advantages of sparse attention and dense attention, realizes the full-image scale aggregation of pixel associations in a single stage, and provides more comprehensive support for feature modeling.

[0066] Ⅱ.Decoder

[0067] The decoder combines the directional prior of road morphology and achieves accurate restoration and refinement of road features through a lightweight structure.

[0068] (1) Directional Augmented Fusion Module (DAFM)

[0069] In high-resolution remote sensing images, roads show linear connections and significant directionality. In order to effectively capture this feature, this embodiment designs a directional enhancement fusion module to perform convolution operations in a specific direction, extract the corresponding directional information, and fuse all directional information by dynamically adjusting the weights of each directional information to obtain a fused feature map.

[0070] The specific calculation formula for directional information fusion is as follows:

[0071]

[0072] Among them, y2 is the fusion direction information, N d is the number of directions. In this embodiment, 16 directions are set. d0 is the number of sampling points in each direction, w i,j represents the convolution weight of the jth sampling point in direction i, p0 is the current pixel position, x(p0+Δpi,j ) is the characteristic value of the offset sampling point predefined based on the direction, Δp i,j Indicates the relative position offset of the jth sampling point in direction i, b i is the bias in direction i.

[0073] The relative position offset Δp of the jth sampling point in direction i i,j The specific expression is as follows:

[0074]

[0075] Among them, r is the distance of the sampling point, θ i is the angle of direction i, θ i Uniformly distributed in [0,2π), δ is the expansion coefficient. The sampling position of the first quadrant of the directional enhanced convolution is as follows Figure 3 As shown, (3a) corresponds to the expansion coefficient δ=1, and (3b) corresponds to the expansion coefficient δ=2.

[0076] The obtained fusion feature map is:

[0077]

[0078] Among them, N f Indicates the number of feature maps, F i is the feature map output by the i-th layer, and DA-Conv represents the directional information fusion operation. Through layer-by-layer fusion, the directional enhancement fusion module can ensure the spatial and semantic consistency of the high-resolution feature map.

[0079] (2) Lightweight Hybrid Attention Module

[0080] After completing feature fusion, this embodiment uses multiple lightweight hybrid attention modules to further process the features, and finally generates pixel-level segmentation results through upsampling and 1x1 convolution.

[0081] Each lightweight hybrid attention module consists of a sparse attention layer and an improved dense attention layer. The specific formula is the same as that in the hybrid attention module, but the FNN layer in the sparse attention layer and the improved dense attention layer becomes a lighter Squeeze-and-Excitation (SE) module. The FFN layer is placed at the end of each attention layer to perform information fusion operations on the number of channels, that is, the third dimension in addition to x and y. The above replacement operation can reduce the amount of calculation. c Represents a channel in the feature map, whose size is H×W, and u c Perform global average pooling (GAP) to obtain z c , the expression is as follows:

[0082]

[0083] According to the above results, the channel u of the input feature map c Make the following adjustments:

[0084]

[0085] s c =σ(w2ξ(w1z c ))

[0086] in, is the adjusted channel, σ is the activation function, w1 and w2 are two one-dimensional fully connected layers, used for dimensionality reduction and dimensionality increase respectively, z c is u after global average pooling c ξ is a nonlinear activation operation. u is adjusted by global average pooling c , reducing the full connection in three dimensions of FFN to the full connection in one dimension, further reducing the computational complexity.

[0087] In this embodiment, the road extraction model based on the hybrid attention mechanism and direction prior adopts the cross entropy loss function and AdamW optimizer during the training process, and introduces a phased learning rate decay strategy and data enhancement method, which can further improve the training efficiency and generalization ability of the model.

[0088] While maintaining high-resolution input, the above method greatly improves the efficiency and accuracy of the model. Compared with existing methods, the above method has significant advantages in extraction accuracy, model efficiency, and adaptability to complex scenes, and provides an efficient and robust solution to the problem of road extraction in complex remote sensing images. To verify the effectiveness of this method, this embodiment conducted experiments on the public DeepGlobe dataset and compared it with a variety of mainstream algorithms. The DeepGlobe dataset is one of the commonly used public datasets in the field of remote sensing. It contains high-resolution remote sensing images in a variety of scenarios and is widely used in tasks such as road extraction, building detection, and land cover classification. Among them, the data subset for the road extraction task includes 6226 images with a resolution of 1024×1024 and a pixel resolution of 0.5 meters / pixel. These images cover urban-rural fringe areas, rural areas, and densely populated urban areas. The scenes are complex and diverse, and the background interference is significant. The pixel-level labels of the roads are annotated in the dataset, providing the training, verification, and test sets required for standard segmentation tasks.

[0089] The HDRoad model proposed in this embodiment includes two versions: HDRoad-Standard and HDRoad-Mobile. Among them, HDRoad-Standard is the standard version, which adopts a complete hybrid attention module and a direction enhancement fusion module, focusing on optimizing the global modeling and local restoration capabilities of road features, and achieving the industry-leading level in segmentation accuracy. HDRoad-Mobile is a lightweight version, which has carried out module optimization and parameter compression for resource-constrained scenarios, maintaining high segmentation performance while significantly reducing model complexity and computational overhead. The parameter volume of HDRoad-Mobile is only 17.24M, and the floating-point operations (FLOPs) are 119G. Compared with the 59.6M parameters and 391G FLOPs of HDRoad-Standard, it is more efficient and suitable for embedded devices or real-time application needs.

[0090] The experiment was conducted on a single NVIDIA RTX 3090 GPU and implemented using the PyTorch framework. During the training process, the cross entropy loss function was combined with the directional prior regularization term to optimize the model. The AdamW optimizer was used, and the initial learning rate was set to 10 -4 , and the cosine annealing learning rate scheduling strategy is used to improve the training stability. In order to enhance the generalization ability of the model, data enhancement operations such as random rotation, flipping and brightness adjustment are performed on the images during training. In addition, in order to comprehensively evaluate the performance of the algorithm, the experiment uses intersection over union (IoU), precision, recall and F1 value as evaluation indicators, and compares this method with mainstream algorithms (including U-Net, DeepLabV3+, Swin transformer and InternImage).

[0091] (1) Qualitative analysis

[0092] On the DeepGlobe dataset, the segmentation performance of this method in a variety of complex scenarios is significantly better than that of existing mainstream algorithms. Figure 4 The segmentation results of different methods are shown in four typical scenarios, including rural areas, densely populated urban areas, and urban-rural fringe areas. Figure 4In the figure, the rows are rural areas, densely populated urban areas, and urban-rural fringe areas; the columns are (4a) input image (test set), (4b) true value label, (4c) D-LinkNet, (4d) U-Net, (4e) DeepLabv3+, (4f) Swin-T (combined with Upernet), (4g) InternImage-T (combined with Upernet), (4h) HDRoad-Mobile (a lightweight version of our method), (4i) HDRoad-Standard (a standard version of our method). It can be seen from the results that our method has significant advantages in the following aspects:

[0093] ① Extraction of slender roads: In rural areas, roads are usually narrow and irregular in shape, and some areas may be blocked by vegetation, shadows, or buildings. Traditional algorithms (such as U-Net and DeepLabV3+) often have broken roads or blurred boundaries in these scenarios. This method captures the directional characteristics of the road through the Direction Enhanced Convolution Module (DA-Conv), which can effectively maintain the continuity and integrity of the road even when the middle section of the slender road is blocked.

[0094] ② Suppression of complex background: In densely populated urban areas, pseudo roads are easily extracted due to interference from buildings, parking lots, and other linear targets. Mainstream algorithms often mix a lot of noise into the segmentation results in these scenarios. This method uses a hybrid attention mechanism to effectively focus on road area features while suppressing background interference targets, thereby significantly reducing the generation of pseudo roads.

[0095] ③ Adaptability in urban-rural fringe areas: In urban-rural fringe areas, the road morphology is complex, including both unstructured roads in rural areas and standardized roads in urban areas. Traditional algorithms are often unable to adapt to multiple road morphologies at the same time in these scenarios. This method achieves a balance between the extraction of global and local features through a multi-stage feature fusion module, and is more adaptable to diverse road morphologies.

[0096] (2) Quantitative analysis

[0097] As shown in Table 1, on the DeepGlobe dataset, both versions of HDRoad significantly outperform existing mainstream methods. HDRoad-Standard performs particularly well in core evaluation indicators such as intersection over union (IoU) and F1 value, with an IoU of 75.06%, which is 3.58% higher than the existing best model InternImage, 4.23% higher than DeepLabv3+ and 4.32% higher than Swin, and about 9% higher than U-Net and D-LinkNet. The F1 value of HDRoad-Standard also reaches 85.67%, showing excellent performance in both recognition accuracy and completeness of road targets.

[0098] As a lightweight version, HDRoad-Mobile has a significantly reduced model parameter count of 17.24M and FLOPs of only 119G, but its IoU still reaches 72.23%, surpassing the InternImage and Swin models with higher computational complexity. In addition, its F1 value reaches 84.01%, which is only about 1.66 percentage points lower than HDRoad-Standard, showing the advantage of maintaining lightweight without sacrificing segmentation performance.

[0099] Table 1 Quantitative analysis results of each model on the DeepGlobe dataset

[0100]

[0101] In summary, this embodiment proposes a road extraction method based on hybrid attention mechanism and direction prior, constructs a road extraction model HDRoad based on hybrid attention mechanism and direction prior based on encoder-decoder architecture, designs hybrid attention module and direction enhancement fusion module, and realizes efficient and accurate road segmentation in complex scenes. The hybrid attention module can effectively extract global and local features by combining sparse and dense attention; the direction enhancement fusion module embeds the morphological prior of the road, which significantly improves the continuity and integrity of the segmentation results. Experiments on the DeepGlobe dataset show that both versions of the HDRoad model outperform existing mainstream methods. Among them, HDRoad-Standard (standard version) achieves a leading level in segmentation accuracy with an intersection over union (IoU) of 75.06% and an F1 value of 85.67%; HDRoad-Mobile (lightweight version) achieves 72.23% IoU and 84.01% F1 value while maintaining a small number of parameters and computational complexity, showing the advantages of lightweight design.

[0102] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0103] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A road extraction method based on hybrid attention mechanism and direction prior, characterized in that: The following steps are involved: Obtain remote sensing images, input the trained road extraction model based on hybrid attention mechanism and direction prior, and obtain the road extraction results; The road extraction model based on the hybrid attention mechanism and direction prior includes an encoder and a decoder; the encoder includes a plurality of sequentially connected hybrid attention modules, the remote sensing image is firstly divided into blocks to generate a plurality of feature blocks, and then processed by a plurality of hybrid attention modules to generate a multi-scale feature map, in each hybrid attention module, the input data is firstly extracted with local features through a plurality of sequentially connected sparse attention layers, and finally the global semantic association is captured through an improved dense attention layer, in which the calculation method of self-attention is reconstructed into a combination of linear activation function and local convolution; The decoder includes a direction enhancement fusion module and a plurality of lightweight hybrid attention modules. The direction enhancement fusion module takes the multi-scale feature map as input, performs convolution operations in multiple directions, obtains corresponding direction information, and fuses all direction information by dynamically adjusting the weights of each direction information, thereby obtaining a fused feature map. The fused feature map is further refined by multiple lightweight hybrid attention modules in sequence, and finally a pixel-level road extraction result is generated by upsampling. Each lightweight hybrid attention module includes a sparse attention layer and a dense attention layer.

2. The road extraction method based on hybrid attention mechanism and direction prior according to claim 1, characterized in that: In the hybrid attention module, the sparse attention layer realizes the dynamic extraction of local features through deformable convolution. The specific expression is as follows: Among them, y1 is the output of the sparse attention layer, N is the number of sampling points, and w n is the weight of the nth sampling point, x(p0+p n +Δp n ) is the characteristic value of the sampling point position, p0 is the current pixel position, p n represents the nth position in the predefined grid sampling {(-1,-1),(-1,0),...,(0,+1),...,(+1,+1)}, Δp n is the position offset obtained through learning, Δm n It is a dynamically adjusted modulus used to control the addition value at different sampling points.

3. The road extraction method based on hybrid attention mechanism and direction prior according to claim 1, characterized in that: In the hybrid attention module, the specific expression of the improved dense attention layer is as follows: Among them, FLA-Attention is the output of the improved dense attention layer, Q, K, V represent query, key and value respectively, σ is a nonlinear activation function, Conv represents the deep convolution operation of channel grouping, and ⊙ represents the element-wise product operation.

4. The road extraction method based on hybrid attention mechanism and direction prior according to claim 3 is characterized in that: The nonlinear activation function σ adopts ReLU 3 .

5. The road extraction method based on hybrid attention mechanism and direction prior according to claim 1, characterized in that: In the directional enhancement fusion module, the specific expressions for performing convolution operations and fusion in multiple directions are as follows: Among them, y2 is the fusion direction information, N d is the number of directions, N d0 is the number of sampling points in each direction, w i,j represents the convolution weight of the jth sampling point in direction i, p0 is the current pixel position, x(p0+Δp i,j ) is the characteristic value of the offset sampling point predefined based on the direction, Δp i,j Indicates the relative position offset of the jth sampling point in direction i, b i is the bias in direction i.

6. The road extraction method based on hybrid attention mechanism and direction prior according to claim 5, characterized in that: The relative position offset Δp of the jth sampling point in direction i i,j The specific expression is as follows: Among them, r is the distance of the sampling point, δ is the expansion coefficient, θ i is the angle of direction i, θ i Uniformly distributed in [0,2π).

7. The road extraction method based on hybrid attention mechanism and direction prior according to claim 1, characterized in that: In the lightweight hybrid attention module, the channel u of the input feature map c Make the following adjustments: s c =σ(w2ξ(w1z c )) in, is the adjusted channel, σ is the activation function, w1 and w2 are two one-dimensional fully connected layers, used for dimensionality reduction and dimensionality increase respectively, z c is u after global average pooling c , ξ is a nonlinear activation operation.

8. The road extraction method based on hybrid attention mechanism and direction prior according to claim 7, characterized in that: will u c Perform global average pooling to obtain z c The specific expression is as follows: Where H and W are u c The height and width of the 9. The road extraction method based on hybrid attention mechanism and direction prior according to claim 1, characterized in that: The road extraction model based on hybrid attention mechanism and direction prior adopts cross entropy loss function and AdamW optimizer during training.

10. The road extraction method based on hybrid attention mechanism and direction prior according to claim 1, characterized in that: The road extraction model based on the hybrid attention mechanism and direction prior introduces a phased learning rate decay strategy and a data enhancement method during the training process.

Images