Remote sensing image road extraction method and storage medium
By introducing a dual sparse attention embedding module and a global average pooling module into the road extraction method of remote sensing images, the problems of disconnection, breakage and complex background interference in the road extraction of remote sensing images in the prior art are solved, and efficient and accurate road extraction results are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-01-30
- Publication Date
- 2026-06-05
AI Technical Summary
Existing remote sensing image road extraction methods struggle to handle complex background interference, occlusion, and multi-scale characteristics in high-resolution images, resulting in disconnected, broken, or topologically incorrect road networks. Furthermore, they suffer from high computational complexity, limiting efficiency and accuracy.
An encoder-decoder network employs a dual sparse attention embedding module and a global average pooling module. The dual sparse attention embedding module models the sparsity of roads and multi-scale contextual information, while the global average pooling module performs spatial compression to reduce computational complexity. Furthermore, binary cross-entropy and segmentation loss functions guide network training.
It enables the accurate capture of fine-grained local connectivity and global topological semantics in high-resolution remote sensing images, reduces computational complexity, and improves the extraction accuracy of road segmentation masks and network performance.
Smart Images
Figure CN122157005A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of remote sensing image information extraction technology, and in particular to a method and storage medium for extracting roads from remote sensing images. Background Technology
[0002] With the rapid development of remote sensing technology, high-resolution satellite and aerial imagery have become crucial data sources for applications such as geographic information acquisition, urban planning, and intelligent traffic management. Among these, road networks, as the backbone of modern urban and regional infrastructure, require precise and efficient automatic extraction of road information for updating map data, supporting navigation systems, and building smart cities. Traditional road extraction methods primarily rely on mathematical morphology, texture analysis, or image processing techniques based on hand-designed features. While these methods can achieve certain results in specific scenarios, their generalization ability is poor, and they are extremely sensitive to image quality, lighting variations, and the diversity of road morphology. They are particularly ill-equipped to handle complex background interference, severe occlusion, and the multi-scale characteristics of roads themselves in high-resolution imagery, and thus cannot meet the demands of today's large-scale, automated processing.
[0003] In recent years, the development of deep learning technology, especially the emergence of encoder-decoder architectures such as fully convolutional neural networks (FCNs) and U-Net, has brought revolutionary progress to semantic segmentation of remote sensing images. U-Net and its variants, such as UNet++ and D-LinkNet, can effectively fuse shallow detail information with deep semantic information through their symmetrical encoder-decoder structure and skip connections, demonstrating significantly better performance than traditional methods in road extraction tasks. However, models based on traditional convolutional neural networks (CNNs) inherently rely on local convolution operations, have limited receptive fields, and are difficult to model long-range dependencies.
[0004] Road targets are typical "elongated" ground features, and their accurate identification and reconstruction rely not only on the discrimination of local pixels but also on the understanding of the overall topological connectivity and global semantic integrity of the road. Therefore, when roads are broken, obscured by the shadows of trees or buildings, or adjacent to features with similar spectral characteristics, such models are prone to misclassification and missed detection, leading to disconnected, broken, or topologically incorrect road networks. To overcome the inherent limitations of CNNs, researchers have attempted to introduce self-attention mechanisms, particularly Transformer models, into segmentation architectures to capture global contextual information. Many works, such as RoadFormer, CMTFNet, and UCTransNet, have improved the model's ability to perceive global features by embedding different forms of attention modules in the U-Net encoder, decoder, or skip connections.
[0005] However, these methods face two main challenges: First, high computational complexity. Standard self-attention mechanisms have quadratic complexity relative to the number of input tokens, while high-resolution remote sensing images typically contain massive numbers of pixels, leading to unacceptable computational and memory overhead when applied directly. Although some studies have employed mechanisms such as deformable attention for approximation, their sampling points are often obtained through free learning, lacking prior guidance on road structure, causing them to deviate from the target area and limiting efficiency and accuracy. Second, the one-sidedness of feature modeling. Existing methods either focus on capturing details through local convolutions while lacking a global perspective, or focus on establishing long-range dependencies through global attention while ignoring the continuous, fine-grained connectivity relationships within the road. They cannot collaboratively and explicitly model the two closely related semantic information of road local connectivity and global integrity, which are of different granularities. This lack of modeling makes performance improvement a bottleneck when dealing with multi-scale, irregular road scenes affected by complex backgrounds. Summary of the Invention
[0006] The purpose of this invention is to provide a method and storage medium for road extraction from remote sensing images. This method fully considers the diversity of input images and the influence of road target characteristics on feature encoding and decoding in road remote sensing image sample acquisition and segmentation tasks using dual sparse attention masks. It can effectively handle road extraction problems from remote sensing images with large scale variations, irregular shapes, sparse structures, and background occlusion. This invention is achieved through the following technical solutions.
[0007] In a first aspect, the present invention provides a method for road extraction from remote sensing images, comprising:
[0008] The remote sensing images of multiple types of roads to be detected are input into a pre-trained encoding and decoding network to obtain a road segmentation mask. The encoding and decoding network includes a first residual module, a first branch and a second branch connected to the first residual module in parallel, and a global average pooling module and a segmentation module connected sequentially to the first branch and the second branch in parallel. The first branch includes a dual sparse attention embedding module and a full road feature fusion module connected sequentially, and the second branch includes a second residual module.
[0009] The first residual module is used to extract features from remote sensing image samples of multiple types of roads to obtain multi-scale distributed abstract features.
[0010] The dual sparse attention embedding module is used to extract features from multi-scale distributed abstract features to obtain road connectivity features and road integrity features, and the full road feature fusion module is used to fuse road connectivity features and road integrity features to obtain a full road fusion feature map;
[0011] The second residual module is used to decode the deep features in the multi-scale distributed abstract features to obtain the decoder feature map;
[0012] The global average pooling module is used to spatially compress the full-road fusion feature map and the decoder feature map to obtain a dual sparse attention mask;
[0013] The segmentation module is used to perform 1×1 convolution and binary classification on the dual sparse attention mask to obtain the road segmentation mask.
[0014] Optionally, the first residual module includes multiple parallel-connected residual units, each of which performs feature extraction on remote sensing image samples of multiple types of roads to obtain multi-scale distributed abstract features; wherein, the deep features in the multi-scale distributed abstract features are the features extracted by the last residual unit on the remote sensing image samples of multiple types of roads.
[0015] Optionally, the loss function of the encoding / decoding network includes binary cross-entropy and a segmentation loss function, calculated as follows:
[0016] ,
[0017] ,
[0018] ,
[0019] In the formula, This is an index for remote sensing image samples of various road types, with values ranging from 1 to... , The total number of remote sensing image samples of various road types. The loss function of the encoding / decoding network, It is the binary cross-entropy. For the segmentation loss function, For the first Road segmentation masks for multiple types of remote sensing image samples of roads. For the first A pre-acquired artificial segmentation mask.
[0020] Optionally, each residual module includes sequentially connected convolutional layers, group normalization layers, and ReLU activation functions.
[0021] Optionally, the multi-scale distributed abstract features are calculated using the following formula:
[0022] ,
[0023] ,
[0024] In the formula, This is the index of the residual cell, with a value ranging from 1 to... , This represents the total number of residual units. For the first Multi-scale distributed abstract features output by each residual unit For the first Multi-scale distributed abstract features output by +1 residual unit Represents the linear rectification activation function. This represents the group normalization function. Represents the convolution function. This indicates a max pooling operation.
[0025] Optionally, the dual sparse attention embedding module includes a connectivity attention unit and an integrity attention unit connected in parallel, and the connectivity attention unit and the integrity attention unit interact with each other.
[0026] The process of inputting multi-scale distributed abstract features into a dual sparse attention embedding module to obtain road connectivity features and road integrity features includes: simultaneously inputting the multi-scale distributed abstract features into a connectivity attention unit and an integrity attention unit to obtain road connectivity features and road integrity features respectively; and then interactively updating the road connectivity features and road integrity features to obtain interactively updated road connectivity features and road integrity features respectively.
[0027] Optionally, the interactively updated road connectivity features are calculated using the following formula: ,
[0028] ,
[0029] In the formula, The index for the number of updates, with a value range of 1- , This represents the total number of updates. , All are window indices, with values ranging from 1 to... and 1- , This represents the total number of windows in the connectivity self-attention computation phase. This represents the total number of connectivity cross-attention computation stages. For the process Passing the exam The updated road connectivity features For the first The updated connectivity features are calculated using secondary connectivity self-attention computation. For the first After layer normalization, the window and the first The updated road connectivity features For the first After layer normalization, the window and the first The updated road integrity features. For the connectivity self-attention computation stage The center grid pixels of each window For the connectivity cross-attention calculation stage The center grid pixels of each window For the connectivity self-attention computation stage Query in one window Deformable offset. For the connectivity cross-attention calculation stage Query in one window Deformable offset. For pixels, , and These are the first stages of connectivity self-attention computation. The first window and the first The query after the update ,key Sum The weight, , and These are the first two stages of connectivity cross-attention computation. The first window and the first The query after the update ,key Sum The weight, For the first Secondary connectivity cross-attention computation updates the connectivity features. For the first After layer normalization, the window and the first The updated road connectivity features For the connectivity attention unit +1 update the output road connectivity features Representation layer normalization, This indicates connectivity-based multi-head self-attention. This represents a multilayer perceptron.
[0030] Optionally, the interactively updated road integrity features are calculated using the following formula:
[0031] ,
[0032] ,
[0033] ,
[0034] In the formula, For the first The updated road integrity features are calculated using the secondary integrity cross-attention method. For the first The residual unit of the first The updated road integrity features. For all residual elements +1 update to road integrity features For the first The residual unit of the first Query after +1 update , and For each of the residual units, the th +1 updated key Sum The weight, For the normalized level of the grade The updated road integrity features. For the integrity attention unit +1 update the output road integrity feature.
[0035] Optionally, the double sparse attention mask is calculated using the following formula:
[0036] ,
[0037] In the formula, For a double sparse attention mask, For decoder feature maps, and All are weighting coefficients. and These are the road connectivity features and road integrity features after interactive updates, respectively. This indicates a global average pooling operation.
[0038] In a second aspect, the present invention provides a computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, when the computer program / instructions are executed by a processor, they implement the steps of the remote sensing image road extraction method described in the first aspect.
[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0040] The remote sensing image road extraction method introduced in this invention effectively models the sparsity, irregular shape, and multi-scale contextual information of roads by introducing a dual sparse attention embedding module into the encoding / decoding network to replace the traditional U-Net skip connections. Through interactive updates of road connectivity and road integrity features, updated road connectivity and integrity features are obtained respectively, achieving accurate capture of fine-grained local road connectivity and overall global topological semantics while reducing computational complexity and encoding / decoding network complexity, thus improving the performance of the encoding / decoding network. The use of a global average pooling module to spatially compress the fused feature map and decoder feature map of the entire road alleviates the information loss caused by the inconsistency in traditional encoding / decoding semantics. By defining the loss function of the encoding / decoding network using binary cross-entropy and segmentation loss function, the training and updates of the entire encoding / decoding network can be effectively supervised and guided, forming an end-to-end learnable jointly optimized network framework, effectively improving the accuracy of road segmentation mask extraction. Attached Figure Description
[0041] Figure 1 The diagram shown is a schematic flowchart of a remote sensing image road extraction method in one embodiment of the present invention.
[0042] Figure 2 The diagram shown is a schematic of the encoding / decoding network structure in one embodiment of the present invention;
[0043] Figure 3 The diagram shown is a comparison of simulation results of different segmentation networks on a multi-type road remote sensing dataset in one embodiment of the present invention.
[0044] Figure 4 The diagram shown illustrates the extraction effect of the remote sensing image road extraction method in a large-scale road dataset according to one embodiment of the present invention. Detailed Implementation
[0045] The following description, in conjunction with the accompanying drawings and specific embodiments, provides further details. In this description, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature.
[0046] Example 1
[0047] This embodiment describes a method for road extraction from remote sensing images, such as... Figure 1 As shown, it includes the following:
[0048] The remote sensing images of multiple types of roads to be detected are input into a pre-trained encoding and decoding network to obtain a road segmentation mask. The encoding and decoding network includes a first residual module, a first branch and a second branch connected to the first residual module in parallel, and a global average pooling module and a segmentation module connected sequentially to the first branch and the second branch in parallel. The first branch includes a dual sparse attention embedding module and a full road feature fusion module connected sequentially, and the second branch includes a second residual module.
[0049] The first residual module is used to extract features from remote sensing image samples of multiple types of roads to obtain multi-scale distributed abstract features.
[0050] The dual sparse attention embedding module is used to extract features from multi-scale distributed abstract features to obtain road connectivity features and road integrity features, and the full road feature fusion module is used to fuse road connectivity features and road integrity features to obtain a full road fusion feature map;
[0051] The second residual module is used to decode the deep features in the multi-scale distributed abstract features to obtain the decoder feature map;
[0052] The global average pooling module is used to spatially compress the full-road fusion feature map and the decoder feature map to obtain a dual sparse attention mask;
[0053] The road segmentation mask is obtained by performing 1×1 convolution and Sigmoid classification on the dual sparse attention mask.
[0054] In one specific embodiment of the present invention, Figure 2 This is a schematic diagram of the encoding / decoding network structure. Figure 2 The module uses a dual sparse attention embedding, a full-path feature fusion module, and a global average pooling module. ResNet encoding and decoding are the first and second residual modules, respectively. 1×1 convolution and sigmoid classification are the operations performed by the segmentation module.
[0055] Example 2
[0056] Based on Example 1, this example describes the specific implementation process of a road extraction method from remote sensing images, which includes the following:
[0057] I. Data Acquisition and Preprocessing
[0058] In one specific embodiment of the present invention, the various types of road remote sensing images used for training and testing are all from publicly available datasets such as Massachusetts, DeepGlobe, SpaceNet, and large-scale road datasets. Road labels are generated from the original GeoJSON file and converted into the various types of road remote sensing image formats and their label datasets used in the experiment; the specific process is as follows:
[0059] (1) Use the GDAL library to read the geographic coordinate transformation parameters from the original GeoJSON file, and use the GeoPandas function to load the road vector data in the same GeoJSON format; then, by creating a three-channel blank canvas of the same size, extract the continuous vertex coordinates of each road line geometric object in the road vector data, and accurately convert the geographic coordinates into pixel coordinates;
[0060] (2) Using OpenCV, white line segments are drawn on the canvas to generate a dataset of multiple types of roads and their labels that are perfectly aligned with the original image space. The resize function is then used to unify the size of the dataset. .
[0061] II. Feature Extraction, Update and Fusion
[0062] 2.1 Feature Extraction
[0063] In one specific embodiment of the present invention, the first residual module includes multiple parallel-connected residual units, each of which performs feature extraction on remote sensing image samples of multiple types of roads to obtain multi-scale distributed abstract features; wherein, the deep features in the multi-scale distributed abstract features are the features extracted by the last residual unit from the remote sensing image samples of multiple types of roads.
[0064] In one specific embodiment of the present invention, each residual unit includes a sequentially connected convolutional layer, a group normalization layer, and a ReLU activation function. The first residual unit is used to extract features from the acquired multi-type road remote sensing image samples to obtain multi-scale distributed abstract features, calculated using the following formula:
[0065] ,
[0066] ,
[0067] In the formula, This is the index of the residual cell, with a value ranging from 1 to... , This represents the total number of residual units. For the first Multi-scale distributed abstract features output by each residual unit For the first Multi-scale distributed abstract features output by +1 residual unit Represents the linear rectification activation function. This represents the group normalization function. Represents the convolution function. This indicates a max pooling operation.
[0068] 2.2 Feature Update and Fusion
[0069] In one specific embodiment of the present invention, the dual sparse attention embedding module includes a connectivity attention unit and an integrity attention unit connected in parallel, and the connectivity attention unit and the integrity attention unit have data interaction;
[0070] The process of inputting multi-scale distributed abstract features into a dual sparse attention embedding module to obtain road connectivity features and road integrity features includes: simultaneously inputting the multi-scale distributed abstract features into a connectivity attention unit and an integrity attention unit to obtain road connectivity features and road integrity features respectively; and then interactively updating the road connectivity features and road integrity features to obtain interactively updated road connectivity features and road integrity features respectively.
[0071] In one specific embodiment of the present invention, road connectivity features and road integrity features are input into the full road feature fusion module to obtain a full road fusion feature map. Specifically, the updated road connectivity features and road integrity features are input into the full road feature fusion module to obtain a full road fusion feature map.
[0072] In one specific embodiment of the present invention, the interactively updated road connectivity features are calculated using the following formula:
[0073] The updated road connectivity features are calculated using the following formula:
[0074]
[0075] ,
[0076] ,
[0077] In the formula, The index for the number of updates, with a value range of 1- , This represents the total number of updates. , All are window indices, with values ranging from 1 to... and 1- , This represents the total number of windows in the connectivity self-attention computation phase. This represents the total number of connectivity cross-attention computation stages. For the process Passing the exam The updated road connectivity features For the first The updated connectivity features are calculated using secondary connectivity self-attention computation. For the first After layer normalization, the window and the first The updated road connectivity features For the first After layer normalization, the window and the first The updated road integrity features. For the connectivity self-attention computation stage The center grid pixels of each window For the connectivity cross-attention calculation stage The center grid pixels of each window For the connectivity self-attention computation stage Query in one window Deformable offset. For the connectivity cross-attention calculation stage Query in one window Deformable offset. For pixels, , and These are the first stages of connectivity self-attention computation. The first window and the first The query after the update ,key Sum The weight, , and These are the first two stages of connectivity cross-attention computation. The first window and the first The query after the update ,key Sum The weight, For the first Secondary connectivity cross-attention computation updates the connectivity features. For the first After layer normalization, the window and the first The updated road connectivity features For the connectivity attention unit +1 update the output road connectivity features Representation layer normalization, This indicates connectivity-based multi-head self-attention. This represents a multilayer perceptron.
[0078] In one specific embodiment of the present invention, the interactively updated road integrity feature is calculated using the following formula:
[0079] ,
[0080] ,
[0081] ,
[0082] In the formula, For the first The updated road integrity features are calculated using the secondary integrity cross-attention method. For the first The residual unit of the first The updated road integrity features. For all residual elements +1 update to road integrity features For the first The residual unit of the first Query after +1 update , and For each of the residual units, the th +1 updated key Sum The weight, For the normalized level of the grade The updated road integrity features. For the integrity attention unit +1 update the output road integrity feature.
[0083] In one specific embodiment of the present invention, a dual sparse attention mask is calculated using the fused connectivity and integrity features and the decoder feature map. Then, the dual sparse attention mask is subjected to 1×1 convolution dimensionality reduction and sigmoid classification to obtain the road segmentation mask. The dual sparse attention mask is calculated using the following formula:
[0084] ,
[0085] In the formula, For a double sparse attention mask, For decoder feature maps, and All are weighting coefficients. and These are the road connectivity features and road integrity features after interactive updates, respectively. This indicates a global average pooling operation.
[0086] In one specific embodiment of the present invention, the loss function of the encoding / decoding network is obtained through the road segmentation mask and a pre-acquired manual segmentation mask. This effectively supervises and guides the training and updating of the entire encoding / decoding network, forming an end-to-end learnable joint optimization network framework, which effectively improves the accuracy of road segmentation mask extraction. The calculation formula is as follows:
[0087] ,
[0088] ,
[0089] ,
[0090] In the formula, This is an index for remote sensing image samples of various road types, with values ranging from 1 to... , The total number of remote sensing image samples of various road types. The loss function of the encoding / decoding network, It is the binary cross-entropy. For the segmentation loss function, For the first Road segmentation masks for multiple types of remote sensing image samples of roads. For the first A pre-acquired artificial segmentation mask.
[0091] III. Experiment
[0092] 3.1 Simulation Conditions
[0093] The simulation experiments used Massachusetts, DeepGlobe, SpaceNet, and a large-scale road dataset. The Massachusetts road dataset contains 1108 training images, 14 validation images, and 49 test images. Each image is 1500×1500 pixels in size, with a spatial resolution of 1 meter per pixel, covering an area of over 2600 square kilometers in Massachusetts, encompassing diverse scenes including rural, suburban, and urban areas. The DeepGlobe road dataset is the benchmark dataset for the Satellite Image Understanding Challenge, containing 6226 pixel-level labeled images from India, Indonesia, and Thailand. Each image is 1024×1024 pixels in size, with a spatial resolution of 0.5 meters per pixel. The SpaceNet road dataset uses 2549 images from four cities: Shanghai, Paris, Khartoum, and Las Vegas, as the training dataset. Line segment labeling is used to represent the centerlines of unpaved, paved, and dirt roads, respectively. Each image is 1300×1300 pixels in size, with a spatial resolution of 0.3 meters per pixel.
[0094] The experiments used overall accuracy (OA), precision, recall, F1 score, and intersection-over-union (IoU) statistics as evaluation metrics. Each metric was tested five times, and the average was used as the final result. The results were compared with DANet, UNet++, D-LinkNet, RecurrUNet, DSCNet, OARENet, SegFormer, RoadFormer, and UCTransNet. Simulation experiments were performed using PyTorch 1.8.1 on a server with an NVIDIA A40 GPU, an Intel® Xeon® Platinum 8358P (2.60GHz) processor, and 80GB of video memory. Furthermore, in this embodiment, the network training iterations were fixed at 200, the initial learning rate was set to 0.0001, the Adam optimizer was used to train the network, and the learning rate was halved when the validation set loss did not decrease within 10 training epochs.
[0095] 3.2 Analysis of Simulation Experiment Results
[0096] Table 1 presents the quantitative results of simulation experiments on the Massachusetts, DeepGlobe, and SpaceNet road datasets for the method of this invention and its comparative methods. Experimental results show that, under the same parameter settings, the method of this invention, by introducing a dual sparse attention embedding module, a ResNet residual network, and a full road feature fusion decoding module, significantly improves the F1-score and mean IoU. The proposed method for road extraction from remote sensing images in Table 1 represents the method proposed in this invention. Compared with existing Transformer encoding / decoding segmentation models (SegFormer, RoadFormer, and UCTransNet in Table 1) and U-Net variants (DANet, UNet++, D-LinkNet, RecurrUNet, DSCNet, and OARENet in Table 1), it improves the F1-score by 4.57 / 10.43 / 8.45 / 24.4 / 16.72 / 14.95 / 16.37 / 13.52 / 2.07 respectively, and the IoU value by 5.97 / 13.29 / 10.82 / 28.73 / 20.77 / 18.69 / 20.37 / 16.92 / 1.83 respectively. This demonstrates that the proposed method for road extraction from remote sensing images significantly surpasses the comparative methods in improving road segmentation accuracy.
[0097] Table 1. Simulation comparison results of different segmentation networks on various types of road remote sensing datasets.
[0098] A comparative illustration of simulation results of different segmentation networks on various types of road remote sensing datasets is shown below. Figure 3 As shown, Figure 3 In the diagram, (a) shows the segmentation results of manual segmentation on three datasets; (b) shows the segmentation results of DANet on three datasets; (c) shows the segmentation results of UNet++ on three datasets; (d) shows the segmentation results of D-LinkNet on three datasets; (e) shows the segmentation results of RecurrUNet on three datasets; (f) shows the segmentation results of DSCNet on three datasets; (g) shows the segmentation results of OARENet on three datasets; (h) shows the segmentation results of SegFormer on three datasets; (i) shows the segmentation results of RoadFormer on three datasets; (j) shows the segmentation results of UCTransNet on three datasets; and (k) shows the segmentation results of the remote sensing image road extraction method proposed in this invention on three datasets. Figure 3As can be seen, the remote sensing image road extraction method proposed in this invention can effectively handle problems such as road breaks, occlusions, and shadows, and the extraction results are continuous and complete, which is closer to the effect of manual segmentation.
[0099] Figure 4 The image shows the road extraction results of the method of this invention on a large-scale road dataset. It can be seen that the extracted roads are relatively clear and without any breaks.
[0100] The method of this invention has the ability to learn deep full road features simultaneously on connectivity details and integrity semantics, and can greatly reduce computational complexity without affecting performance. It has excellent performance when applied to road network extraction from high-resolution remote sensing images.
[0101] Example 3
[0102] This embodiment describes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the remote sensing image road extraction method as described in Embodiment 1 or 2.
[0103] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0104] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0105] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0106] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0107] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.
Claims
1. A method for road extraction from remote sensing images, characterized in that, include: The remote sensing images of various types of roads to be detected are input into a pre-trained encoding and decoding network to obtain road segmentation masks; The encoding / decoding network includes a first residual module, a parallel first branch and a second branch connected to the first residual module, and a global average pooling module and a segmentation module connected sequentially to the parallel first branch and the second branch. The first branch includes a sequentially connected dual sparse attention embedding module and a full-path feature fusion module, and the second branch includes a second residual module. The first residual module is used to extract features from remote sensing image samples of multiple types of roads to obtain multi-scale distributed abstract features. The dual sparse attention embedding module is used to extract features from multi-scale distributed abstract features to obtain road connectivity features and road integrity features, and the full road feature fusion module is used to fuse road connectivity features and road integrity features to obtain a full road fusion feature map; The second residual module is used to decode the deep features in the multi-scale distributed abstract features to obtain the decoder feature map; The global average pooling module is used to spatially compress the full-road fusion feature map and the decoder feature map to obtain a dual sparse attention mask; The segmentation module is used to perform 1×1 convolution and binary classification on the dual sparse attention mask to obtain the road segmentation mask.
2. The method for road extraction from remote sensing images according to claim 1, characterized in that, The first residual module includes multiple parallel-connected residual units, each of which extracts features from remote sensing image samples of various types of roads to obtain multi-scale distributed abstract features. Among these, the deep features in the multi-scale distributed abstract features are the features extracted by the last residual unit from the remote sensing image samples of various types of roads.
3. The method for road extraction from remote sensing images according to claim 1, characterized in that, The loss function of the encoding / decoding network includes binary cross-entropy and segmentation loss function, and the calculation formula is as follows: ， ， ， In the formula, This is an index for remote sensing image samples of various road types, with values ranging from 1 to... , The total number of remote sensing image samples of various road types. These are the weighting coefficients. The loss function of the encoding / decoding network, It is the binary cross-entropy. For the segmentation loss function, For the first Road segmentation masks for multiple types of remote sensing image samples of roads. For the first A pre-acquired artificial segmentation mask.
4. The method for road extraction from remote sensing images according to claim 2, characterized in that, Each residual unit consists of sequentially connected convolutional layers, group normalization layers, and ReLU activation functions.
5. The method for road extraction from remote sensing images according to claim 4, characterized in that, The multi-scale distributed abstract features are calculated using the following formula: ， ， In the formula, This is the index of the residual cell, with a value ranging from 1 to... , This represents the total number of residual units. For the first Multi-scale distributed abstract features output by each residual unit For the first Multi-scale distributed abstract features output by +1 residual unit Represents the linear rectification activation function. This represents the group normalization function. Represents the convolution function. This indicates a max pooling operation.
6. The method for road extraction from remote sensing images according to claim 1, characterized in that, The dual sparse attention embedding module includes a connectivity attention unit and an integrity attention unit connected in parallel, and the connectivity attention unit and the integrity attention unit interact with each other. The process of inputting multi-scale distributed abstract features into a dual sparse attention embedding module to obtain road connectivity features and road integrity features includes: simultaneously inputting the multi-scale distributed abstract features into a connectivity attention unit and an integrity attention unit to obtain road connectivity features and road integrity features respectively; and then interactively updating the road connectivity features and road integrity features to obtain interactively updated road connectivity features and road integrity features respectively.
7. The method for road extraction from remote sensing images according to claim 6, characterized in that, The updated road connectivity features are calculated using the following formula: ， ， In the formula, The index for the number of updates, with a value range of 1- , This represents the total number of updates. , All are window indices, with values ranging from 1 to... and 1- , This represents the total number of windows in the connectivity self-attention computation phase. This represents the total number of connectivity cross-attention computation stages. For the process Passing the exam The updated road connectivity features For the first The updated connectivity features are calculated using secondary connectivity self-attention computation. For the first After layer normalization, the window and the first The updated road connectivity features For the first After layer normalization, the window and the first The updated road integrity features. For the connectivity self-attention computation stage The center grid pixels of each window For the connectivity cross-attention calculation stage The center grid pixels of each window For the connectivity self-attention computation stage Query in one window Deformable offset. For the connectivity cross-attention calculation stage Query in one window Deformable offset. For pixels, , and These are the first stages of connectivity self-attention computation. The first window and the first The query after the update ,key Sum The weight, , and These are the first two stages of connectivity cross-attention computation. The first window and the first The query after the update ,key Sum The weight, For the first Secondary connectivity cross-attention computation updates the connectivity features. For the first After layer normalization, the window and the first The updated road connectivity features For the connectivity attention unit +1 update the output road connectivity features Representation layer normalization, This indicates connectivity-based multi-head self-attention. This represents a multilayer perceptron.
8. The method for road extraction from remote sensing images according to claim 7, characterized in that, The updated road integrity features are calculated using the following formula: ， ， ， In the formula, For the first The updated road integrity features are calculated using the secondary integrity cross-attention method. For the first The residual unit of the first The updated road integrity features. For all residual elements +1 update to road integrity features For the first The residual unit of the first Query after +1 update , and For each of the residual units, the th +1 updated key Sum The weight, For the normalized level of the grade The updated road integrity features. For the integrity attention unit +1 update the output road integrity feature.
9. The remote sensing image road extraction method according to claim 8, characterized in that it is dual... The sparse attention mask is calculated using the following formula: ， In the formula, For a double sparse attention mask, For decoder feature maps, and All are weighting coefficients. and These are the road connectivity features and road integrity features after interactive updates, respectively. This indicates a global average pooling operation.
10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the remote sensing image road extraction method according to any one of claims 1-9.