Remote sensing image road extraction method and device

Abstract

The invention relates to the technical field of natural geography, and particularly provides a remote sensing image road extraction method and device, and the method comprises the following steps: S1, data preparation and feature coding; s2, constructing a road feature reasoning enhancement module based on graph convolution; s3, decoder construction and multi-scale feature fusion are carried out; s4, generating a road probability graph; and S5, model training and performance evaluation. Compared with the prior art, the method has the advantages that the road network which is more complete, better in connectivity and clearer in boundary can be extracted, and the method has important theoretical research value and practical application prospect.

Description

Technical Field

[0001] The present invention relates to the field of physical geography technology, and specifically provides a method and device for extracting roads from remote sensing images. Background Art

[0002] High-resolution remote sensing images have become a key data source for obtaining surface information. Road extraction is crucial for applications such as urban planning, traffic management, and navigation updates. Especially in the context of smart cities and refined management, the demand for high-precision road extraction technology is becoming increasingly urgent.

[0003] However, a major challenge in extracting roads from remote sensing imagery is object occlusion. Roads are often obscured by trees, buildings, and shadows, resulting in incomplete or lost road feature information in the image. Automated extraction methods often produce road results with defects such as gaps, holes, and blurred boundaries, severely impacting the completeness and accuracy of the extraction.

[0004] Traditional road extraction methods mainly rely on preset geometric or spectral features, which are acceptable in simple unobstructed scenes. However, under complex occlusion conditions, the performance drops significantly due to feature loss and interference.

[0005] In recent years, deep learning methods, such as convolutional neural networks (CNNs), have made significant progress in road extraction tasks, demonstrating powerful feature learning capabilities. However, even deep learning methods with improved network structures, multi-task learning, or the incorporation of auxiliary information (such as orientation and topology) still have limitations when dealing with severe occlusions. These methods often struggle to effectively utilize global contextual information to infer the road structure of the occluded portion, resulting in a disruption in road connectivity. Summary of the Invention

[0006] The present invention aims to overcome the above-mentioned deficiencies in the prior art and provides a highly practical method for extracting roads from remote sensing images.

[0007] A further technical task of the present invention is to provide a remote sensing image road extraction device that is rationally designed, safe and applicable.

[0008] The technical solution adopted by the present invention to solve its technical problem is:

[0009] A method for extracting roads from remote sensing images comprises the following steps:

[0010] S1, data preparation and feature encoding;

[0011] S2. Build a road feature inference enhancement module based on graph convolution;

[0012] S3, decoder construction and multi-scale feature fusion;

[0013] S4, generation of road probability map;

[0014] S5. Model training and performance evaluation.

[0015] Furthermore, in step S1, the remote sensing image dataset Massachusetts Roads Dataset and the corresponding road label map are obtained, and the data are preprocessed, including uniformly cropping to a fixed size and normalizing pixel values ​​to the [0,1] interval;

[0016] Data augmentation technology is applied to improve the generalization ability of the model. The dataset is divided into training set, validation set and test set in the ratio of 70%:15%:15%. An encoder based on the residual network ResNet-50 is constructed and initialized with the ImageNet pre-trained weights. The encoder reduces the resolution through the initial convolution layer and the maximum pooling layer, and then gradually extracts deep features through four residual stages. Each residual stage contains multiple bottleneck structure residual blocks.

[0017] Furthermore, in step S2, it includes:

[0018] S2-1, Node and graph structure definition;

[0019] S2-2, adjacency relationship construction;

[0020] S2-3. Graph convolution feature enhancement.

[0021] Furthermore, in step S2-1, the deepest feature map output by the encoder is H′=16,W′=16,C=2048; flatten in the spatial dimension and regard it as a set of N=H′×W′ nodes. The dimension of the feature vector of each node is C, and the node feature matrix is ​​obtained N = 256;

[0022] In step S2-2, in order to capture the non-local dependency between road pixels, the adjacency relationship of the graph is constructed and the adjacency relationship between any two nodes V is calculated. i , V j The cosine similarity of the features, for each node V i Select the K nodes with the highest similarity as its neighbors and build a sparse adjacency matrix based on this Among them, A ij =sim(v i , v j ), through symmetric processing A=(A+A T ) / 2 ensures the undirectedness of the graph;

[0023] In step S2-3, symmetric normalization is applied to the adjacency matrix A to calculate Where D is the degree matrix, a graph convolutional network containing multiple graph convolutional layers is used to process the node features. Each graph convolutional network performs the operation Among them, H (0) =X,W (l) is a learnable weight matrix, and the graph convolutional network module outputs the enhanced node feature matrix The feature matrix integrates the contextual information based on the graph structure to reason about the connectivity and structure of the road.

[0024] Furthermore, in step S3, the decoder adopts a multi-level structure symmetrical to the encoder and receives the low-resolution deep feature map F′ enhanced by graph convolution. k As input, through a series of continuous upsampling operations, the spatial dimension of the feature map is gradually enlarged. After each upsampling step, the current feature map will be fused with the feature map (F3, F2, F1) with higher spatial resolution but shallower semantics saved at the corresponding level of the encoder through jump connections. The fused feature map is then processed by the convolution block to learn effective feature combinations, refine the representation and adjust the number of channels. This process is repeated layer by layer until the feature map is restored to the resolution of the original input image, and finally a high-resolution feature map D4 that integrates multi-scale context information is generated.

[0025] Furthermore, in step S4, after the final output of the decoder, a 1×1 convolutional layer is applied to map the number of feature channels to 1, representing the predicted logit of the road category, and the logit is converted into a pixel-level road prediction probability map through the Sigmoid activation function. Each pixel value in the image represents the probability that the location belongs to the road.

[0026] Furthermore, in step S5, it includes:

[0027] S5-1. Use a hybrid loss function to guide model training, combining pixel-level accuracy and regional consistency, and use the weighted sum of binary cross entropy loss BCE Loss and Dice loss: L total =w1×L BCE +w2×L Dice , W1=W2=0.5,L BCE Focus on the accuracy of each pixel classification, L Dice Pay attention to the overlap between the predicted area and the real area;

[0028] S5-2. Use the AdamW optimizer and a learning rate of 1e-4 to iteratively optimize on the training set, set the batch size to 12 and the number of training rounds to 200. After each training round, use the validation set to evaluate the model performance and save the model weights with the best performance.

[0029] S5-3. Use an independent test set to fully evaluate the final selected best model and calculate key indicators, including intersection over union, precision, recall, and F1 score.

[0030] A remote sensing image road extraction device, comprising: at least one memory and at least one processor;

[0031] The at least one memory is configured to store a machine-readable program;

[0032] The at least one processor is used to call the machine-readable program to execute a remote sensing image road extraction method.

[0033] Compared with the prior art, the method and device for extracting roads from remote sensing images of the present invention have the following outstanding beneficial effects:

[0034] (1) By introducing a graph convolutional network, a graph structure is constructed at the feature level, connections are established using node similarity, and information is propagated and aggregated through graph convolution. This enables the model to explicitly model long-range dependencies between road pixels and between roads and similar contextual environments, effectively reasoning about occluded or weakly featured road parts, and significantly enhancing the expressiveness and global consistency of road features.

[0035] (2) The information aggregation mechanism of graph convolution enables broken road segments to "perceive" each other through the connection paths on the graph, making it easier to connect them during prediction. This effectively solves the problem of incomplete and broken road extraction caused by factors such as occlusion and shadows, and improves the connectivity of the extraction results.

[0036] (3) The ResNet-based encoder can effectively extract road features in complex backgrounds. The graph convolution module helps distinguish roads from easily confused objects in the background by associating similar features. The introduction of Dice loss makes the model more robust to the common imbalance in the number of road / background pixels in remote sensing images, and comprehensively improves the model's road extraction performance in various complex scenarios.

[0037] In summary, the method of the present invention effectively overcomes the limitations of traditional methods in processing complex remote sensing images, especially when facing occlusion and blurred boundaries, through an innovative graph convolution feature enhancement mechanism. It can extract a more complete road network with better connectivity and clearer boundaries, and has important theoretical research value and practical application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 The present invention is a flowchart of a method for extracting roads from remote sensing images. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention will be further described in detail below in conjunction with specific embodiments. Obviously, the embodiments described are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0041] A best embodiment is given below:

[0042] like Figure 1 As shown, a remote sensing image road extraction method in this embodiment has the following steps:

[0043] S1, data preparation and feature encoding;

[0044] A road extraction network with an encoder-decoder architecture is constructed, where the encoder adopts a deep convolutional neural network based on the residual network ResNet and is initialized with pre-trained weights to hierarchically extract a set of multi-scale feature maps {F1, F2, ..., F k}.

[0045] The network structure is specifically ResNet-50, and ImageNet pre-trained weights are loaded; the encoder contains the initial convolution and pooling layers and four main residual stages, each of which consists of multiple bottleneck structures (Bottleneck) residual blocks. Through gradual convolution and downsampling operations (usually implemented by convolution with a step size of 2 at the beginning of the stage), a sequence of feature maps {F1, F2, F3, F k}, where F K Has the smallest spatial resolution and the highest level of semantic abstraction.

[0046] S2. Build a road feature inference enhancement module based on graph convolution;

[0047] The deepest feature map F output by the encoder k ∈RH′×W′×C Flatten in the spatial dimension and regard it as a set of N = H′×W′ nodes. The dimension of the feature vector of each node is C, and the initial node feature matrix X∈R is obtained N×C ; Based on the cosine similarity calculation between node features, a sparse adjacency matrix A∈R representing the strength of the association between nodes is constructed N×N , thereby explicitly converting feature graph data into a graph structure;

[0048] The process of constructing the sparse adjacency matrix A specifically includes: calculating the adjacency matrix between any two nodes V i , V j The eigenvector X i , X j Cosine similarity between For each node Vi, select the K (K=10) nodes (K-Nearest Neighbors, KNN) with the highest similarity as its neighbors, or set a similarity threshold τ and only retain the edges with similarity greater than τ. Construct the adjacency matrix A based on the selected neighbors or edges that meet the threshold, where A ij =sim(v i , v j ) or 1, otherwise A ij = 0, optionally, perform symmetric processing on A = (A + A T ) / 2 and perform symmetric normalization Where D is the degree matrix.

[0049] S3, decoder construction and multi-scale feature fusion;

[0050] A graph convolutional network (GCN) containing multiple layers of graph convolutional layers is applied to process the graph data (node ​​feature matrix X and adjacency matrix A or its normalized form) constructed in step S2; each layer of GCN updates the representation of the central node by aggregating the feature information of neighboring nodes, achieving effective transmission and aggregation of information between nodes, thereby capturing long-distance dependencies, integrating global context information based on the graph structure, and outputting the enhanced node feature matrix H (final) ,This feature particularly enhances the connectivity and structural integrity of the road;

[0051] The application of graph convolutional network processing specifically includes: taking the node feature matrix X as the initial input H of GCN (0) ; For the Lth graph convolution layer, its forward propagation operation is Where σ is a nonlinear activation function (such as ReLU), W (l) is the learnable weight matrix of this layer; by stacking L layers (e.g., L=2) of graph convolutional layers, the receptive field of each node can be extended to its L-hop neighbors, thereby effectively aggregating a wider range of contextual information and outputting the final enhanced node feature matrix H(final) ,This feature is more robust in inferring the road ownership in partially occluded or complex background areas.

[0052] S4, generation of road probability map;

[0053] The node feature matrix H after graph convolution enhancement in step S3 (final) Reshape into spatial feature map F′ k , and input it to the decoder; the decoder gradually amplifies the spatial resolution of the feature map through a series of upsampling operations (such as transposed convolution or bilinear interpolation), and after each upsampling stage, uses a skip connection to connect the current feature map with the feature map (F) with higher spatial resolution output by the encoder at the corresponding level. k-1 ,...,F1) performs feature fusion by channel splicing or element addition to integrate multi-scale information and accurately restore the spatial details and boundaries of the road;

[0054] The decoder fuses multi-scale features through skip connections. Specifically, after each upsampling stage of the decoder, the upsampled feature map U(F′) is l+1 ) and the feature map F outputted by the encoder at the corresponding level (with the same spatial resolution) l Concatenation is performed in the channel dimension, i.e. Concat(U(F′ l+1 ),F l ); the concatenated features are then processed through one or more convolutional layers (usually including 3x3 convolution, batch normalization, and activation functions) to learn effective feature combinations, adjust the number of channels, and refine the feature representation, thereby effectively incorporating the high-frequency spatial detail information retained by the encoder into the decoding path.

[0055] S5, model training and performance evaluation;

[0056] Based on the high-resolution feature map D4 finally output by the decoder, a 1x1 convolution layer is applied to map the number of feature channels to 1, and a pixel-level road prediction probability map is generated through the Sigmoid activation function. A method combining binary cross entropy loss (L BCE ) and Dice loss (L Dice )’s mixed loss function L total =w1×L BCE +w2×L Dice The network model is trained and optimized end-to-end to simultaneously improve pixel classification accuracy and segmentation region overlap.

[0057] Binary cross entropy loss where y iis the true label of pixel i, p i is the predicted probability;

[0058] Dice loss L Dice =1-DSC, where Dice coefficient DSC=(2∑y i p i +∈) / (∑y i +∑p i +∈), ∈ is a smoothing term to prevent division by zero;

[0059] Hybrid loss: a weighted combination of the two:

[0060] L total =α×L BCE +β×L Dice ;

[0061] α and β are hyperparameters used to adjust the relative importance of the two losses. For example, you can set α to 0.5 and β to 0.5 or adjust based on the validation set performance. This hybrid loss function optimizes both pixel-level accuracy and region overlap during training, enabling the model to improve segmentation performance for incomplete and blurred roads while preserving detail.

[0062] Based on the above method, a remote sensing image road extraction device in this embodiment includes: at least one memory and at least one processor;

[0063] The at least one memory is configured to store a machine-readable program;

[0064] The at least one processor is used to call the machine-readable program to execute a remote sensing image road extraction method.

[0065] The above-mentioned specific implementation methods are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementation methods. Any technical solutions that conform to the above-mentioned specific implementation methods of the present invention and any appropriate changes or substitutions made thereto by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0066] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for extracting roads from remote sensing images, characterized in that: The steps are as follows: S1, data preparation and feature encoding; S2. Build a road feature inference enhancement module based on graph convolution; S3, decoder construction and multi-scale feature fusion; S4, generation of road probability map; S5. Model training and performance evaluation.

2. The method for extracting roads from remote sensing images according to claim 1, characterized in that: In step S1, the remote sensing image dataset Massachusetts Roads Dataset and the corresponding road label map are obtained, and the data is preprocessed, including uniformly cropping to a fixed size and normalizing pixel values ​​to the [0, 1] interval; Data augmentation technology is applied to improve the generalization ability of the model. The dataset is divided into training set, validation set and test set in the ratio of 70%:15%:15%. An encoder based on the residual network ResNet-50 is constructed and initialized with the ImageNet pre-trained weights. The encoder reduces the resolution through the initial convolution layer and the maximum pooling layer, and then gradually extracts deep features through four residual stages. Each residual stage contains multiple bottleneck structure residual blocks.

3. The method for extracting roads from remote sensing images according to claim 2, characterized in that: In step S2, it includes: S2-1, Node and graph structure definition; S2-2, adjacency relationship construction; S2-3. Graph convolution feature enhancement.

4. The method for extracting roads from remote sensing images according to claim 3, wherein: In step S2-1, the deepest feature map output by the encoder is H′=16,W′=16,C=2048; flatten in the spatial dimension and regard it as a set containing N=H′×W′ nodes. The dimension of the feature vector of each node is C, and the node feature matrix is ​​obtained N = 256; In step S2-2, in order to capture the non-local dependency between road pixels, the adjacency relationship of the graph is constructed and the adjacency relationship between any two nodes V is calculated. i , V j The cosine similarity of the features, for each node V i Select the K nodes with the highest similarity as its neighbors and build a sparse adjacency matrix based on this Among them, A ij =sim(v i , v j ), through symmetric processing A=(A+A T ) / 2 ensures the graph is undirected; In step S2-3, symmetric normalization is applied to the adjacency matrix A to calculate Where D is the degree matrix, a graph convolutional network containing multiple graph convolutional layers is used to process the node features. Each graph convolutional network performs the operation Among them, H (0) =X,W (l) is a learnable weight matrix, and the graph convolutional network module outputs the enhanced node feature matrix The feature matrix integrates the contextual information based on the graph structure to reason about the connectivity and structure of the road.

5. The method for extracting roads from remote sensing images according to claim 4, characterized in that: In step S3, the decoder adopts a multi-level structure symmetrical to the encoder and receives the low-resolution deep feature map F′ enhanced by graph convolution k As input, through a series of continuous upsampling operations, the spatial dimension of the feature map is gradually enlarged. After each upsampling step, the current feature map will be fused with the feature map (F3, F2, F1) with higher spatial resolution but shallower semantics saved at the corresponding level of the encoder through jump connections. The fused feature map is then processed by the convolution block to learn effective feature combinations, refine the representation and adjust the number of channels. This process is repeated layer by layer until the feature map is restored to the resolution of the original input image, and finally a high-resolution feature map D4 that integrates multi-scale context information is generated.

6. The method for extracting roads from remote sensing images according to claim 5, characterized in that: In step S4, after the final output of the decoder, a 1×1 convolutional layer is applied to map the number of feature channels to 1, representing the predicted logit of the road category, and the logit is converted into a pixel-level road prediction probability map through the Sigmoid activation function. Each pixel value in the image represents the probability that the location belongs to the road.

7. The method for extracting roads from remote sensing images according to claim 6, characterized in that: In step S5, it includes: S5-1. Use a hybrid loss function to guide model training, combining pixel-level accuracy and regional consistency, and use the weighted sum of binary cross entropy loss BCE Loss and Dice loss: L total =w1×L BCE +w2×L Dice , W1=W2=0.5,L BCE Focus on the accuracy of each pixel classification, L Dice Pay attention to the overlap between the predicted area and the real area; S5-2. Use the AdamW optimizer and a learning rate of 1e-4 to iteratively optimize on the training set, set the batch size to 12 and the number of training rounds to 200. After each training round, use the validation set to evaluate the model performance and save the model weights with the best performance. S5-3. Use an independent test set to fully evaluate the final selected best model and calculate key indicators, including intersection over union, precision, recall, and F1 score.

8. A remote sensing image road extraction device, characterized in that: include: at least one memory and at least one processor; The at least one memory is configured to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 7.

Images