Multi-task collaborative remote sensing image road topological structure extraction method, system and equipment

Abstract

The invention relates to the technical field of remote sensing image processing, in particular to a multi-task collaborative remote sensing image road topological structure extraction method, system and device, and the method comprises the steps of data preparation, model training and road extraction. A multi-task cooperation mechanism is introduced, a segmentation task and a topology modeling task are fused, and unified modeling of road mask segmentation and topology structure reconstruction is realized through an end-to-end cooperation optimization process from feature extraction, key point detection, dynamic prompt generation to topology repair formation, so that the road mask segmentation and topology structure reconstruction efficiency is improved while high segmentation precision is kept. The method has the advantages of being clear in structure, efficient in calculation, high in generalization ability and the like, and can be widely applied to the fields of intelligent transportation, urban planning, disaster emergency, automatic map drawing and the like.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a method, system and device for extracting road topology from remote sensing images through multi-task collaboration. Background Technology

[0002] Remote sensing image recognition and interpretation are primarily used for road navigation and disaster emergency response. Manual interpretation methods for road segmentation and extraction from remote sensing images are costly and time-consuming. However, with the development of artificial intelligence in recent years, especially significant breakthroughs in computer vision, and combined with the high resolution and rapid update speed of remote sensing images, the ability to quickly extract roads from remote sensing images has become possible.

[0003] Traditional processing techniques for road extraction from remote sensing images include the following two methods: one is based on edge detection methods, using operators such as Canny and Sobel to extract road edges, but it is difficult to distinguish between roads and non-road linear features (such as walls and rivers); the other is based on morphological methods, connecting broken roads through dilation and erosion, but it has poor processing effect on complex topological structures (such as overpasses), and it relies on manually designed features, has weak generalization ability, and low recall rate.

[0004] With the development of deep learning technology, more and more scholars have begun to use semantic segmentation networks such as Convolutional Neural Network (CNN), Fully Convolutional Network (FCN), U-Net, and DeepLab to automatically extract road regions from remote sensing images end-to-end.

[0005] However, existing deep learning-based road extraction methods still mainly focus on pixel-level segmentation accuracy as the optimization goal, and lack sufficient modeling and constraint on the overall topology of the road network. On the one hand, in areas with low contrast, severe occlusion, or uneven resolution, problems such as road breaks, endpoint drift, and missed detection of fine roads are prone to occur, resulting in poor topological connectivity. On the other hand, some methods rely on manually set connection rules or thresholds in the post-processing stage, which has limited generalization ability and makes it difficult to simultaneously achieve high segmentation accuracy and high topological integrity.

[0006] It should be noted that the information disclosed in this background section is intended only to enhance the understanding of the overall background of the present invention, and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0007] To address the technical problems of severe road fragmentation, poor topological connectivity, and insufficient ability to identify low-contrast fine roads in existing remote sensing image road extraction techniques, this invention provides a multi-task collaborative method for extracting road topology from remote sensing images. Based on the SAMRoadPlus model, this method improves the connectivity and overall extraction accuracy of the topological structure of roads in remote sensing images by improving its architecture and training strategy. The method includes the following steps: Data preparation steps: Obtain publicly available remote sensing road datasets, which include remote sensing road images and labeled data, and preprocess the obtained remote sensing road datasets to obtain processed remote sensing road datasets. Model training steps: Construct a multi-task collaborative remote sensing image road extraction model, input the processed remote sensing road dataset into the multi-task collaborative remote sensing image road extraction model for feature extraction, and output multi-scale feature maps of different lengths; Using multi-scale feature maps as input, a high-resolution feature branch is constructed through the HRNetKeypoint architecture, and an asymmetric Gaussian kernel is introduced to generate a target heatmap. Extract key road points from remotely sensed road images; Based on the target heatmap, positive prompt information is generated, and the multi-scale feature map, target heatmap and positive prompt information are input into the segmentation and decoding module to generate the preliminary road segmentation mask for the current iteration. The topology modeling module constructs a road skeleton and road map on the initial road segmentation mask, identifies broken edges and generates negative prompts, feeds the negative prompts back to the segmentation decoding module to correct and refine the initial road segmentation mask, outputs the updated road segmentation mask, and saves it to obtain the trained multi-task collaborative remote sensing image road extraction model. Road extraction steps: Input the real-time remote sensing road dataset into the trained multi-task collaborative remote sensing image road extraction model, and output the remote sensing road image.

[0008] Furthermore, the preprocessing of the remote sensing road dataset includes data augmentation operations, which include, but are not limited to, performing random flipping, deformation scaling, and color and brightness adjustments on the remote sensing road image and the labeled data simultaneously.

[0009] Furthermore, the step of inputting the processed remote sensing road dataset into a multi-task collaborative remote sensing image road extraction model for feature extraction to output multi-scale feature maps of different lengths includes the following steps: The input remote sensing road image is used as an encoder for feature extraction. The input remote sensing road image is divided into 256×256 image blocks and converted into embedding vectors through linear projection. After processing by a multi-layer Transformer encoder, features are extracted at different levels, and four feature maps with strides of 4, 8, 16 and 32 are output to capture semantic information from fine-grained to coarse-grained. After feature extraction, a local feature enhancement convolutional layer is added to enhance the feature representation ability of narrow roads and small targets and improve the detection accuracy of low-contrast roads.

[0010] Furthermore, the step of using multi-scale feature maps as input and constructing a keypoint detection branch through the HRNetKeypoint architecture to extract road keypoints from remotely sensed road images includes: The process involves taking multi-scale feature maps as input, constructing high-resolution feature branches through the HRNetKeypoint architecture, and introducing asymmetric Gaussian kernels to generate target heatmaps. Combined with the target heatmaps, a heatmap regression method is used to detect key points. A heatmap response value is predicted for each potential road key point location, and the higher the response value, the greater the probability that the location is a key point.

[0011] Furthermore, the step of generating positive prompt information based on the target heatmap and inputting the multi-scale feature map, target heatmap, and positive prompt information into the segmentation decoding module to generate the preliminary road segmentation mask for the current iteration includes the following steps: Local maxima points are extracted from the generated target heatmap as positive prompts; these points correspond to key locations on the road. The positive prompt information is encoded by mapping the coordinates of the positive prompt points to the corresponding feature map scale and performing position encoding to obtain the positive prompt embedding vector. The target heatmap is scale-aligned with the multi-scale feature map, and the target heatmap is used as a spatial prior weight to weight and enhance the features of road-related areas, thus obtaining heatmap-guided features. Multi-scale feature maps are fused from top to bottom / bottom to top to obtain fused features. Positive cue embedding vectors are introduced into the fused features. Cue-guided feature updates are achieved through cross attention or gating mechanisms to obtain cue-guided decoding features. The prompt-guided decoding features are decoded and upsampled layer by layer to output a road segmentation probability map; the road segmentation probability map is processed by an activation function and thresholded to obtain the preliminary road segmentation mask for the current iteration.

[0012] Furthermore, the process of constructing a road skeleton and road map on the initial road segmentation mask using the topology modeling module, identifying broken edges and generating negative prompts, feeding the negative prompts back to the segmentation decoding module to correct and refine the initial road segmentation mask, and outputting the updated road segmentation mask includes the following steps: Connectivity analysis and morphological preprocessing are performed on the preliminary road segmentation mask, including but not limited to hole filling, lightweight closure operation of fractures and gaps, burr removal and small connected component removal, in order to reduce the impact of noise on skeletonization. Perform a thinning / skeletonization operation on the preprocessed road segmentation mask to obtain a road centerline skeleton with a single pixel width; The road centerline skeleton is pruned to remove pseudo-branches and isolated skeleton segments with a length less than a preset threshold. The road centerline skeleton is then smoothed and broken point corrected to obtain a purified road skeleton. Calculate the neighborhood connectivity of the skeleton pixels on the purified road skeleton, extract the endpoint pixels with a degree of 1 as endpoint nodes, and the pixels with a degree of ≥3 as intersection nodes. The curvature threshold can be combined to extract the turning nodes to form a set of road graph nodes. Starting from a road graph node, path tracing is performed along the cleaned road skeleton pixels. The skeleton path between any two adjacent nodes is defined as a road edge, and the geometric sequence, length, direction and other attributes of each edge are recorded. A road graph is constructed using the set of road graph nodes as the set of vertices and the set of road edges as the set of edges. The positions of nodes and edges in the road graph are then mapped back to the coordinates of the original remote sensing road image.

[0013] The topology modeling module identifies broken edges in the road network, i.e. road segments that should be connected but are not. The endpoints of the broken edges and their surrounding areas are used as negative cue information input to the segmentation and decoding module to guide the model to focus on these error-prone areas. The segmentation module corrects the initial road segmentation mask based on positive and negative prompts and outputs the updated road segmentation mask. The topology modeling module optimizes road connectivity based on the updated road segmentation mask. The two modules iterate and optimize until convergence, and then output the final road segmentation result.

[0014] Furthermore, the topology modeling module includes a two-level graph attention network, which uses the detected road key points as graph nodes and connects adjacent nodes to form edges through a distance threshold. The features of each edge are composed of the concatenation of the feature vectors of the two nodes and the direction vector between the two nodes. The first-level graph attention network calculates the importance weight of each edge through the attention mechanism, learns the correlation strength between nodes, and outputs a preliminary edge weight prediction. The second-level graph attention network receives the edge weight prediction results output by the first-level graph attention network, further refines the edge weight prediction, and integrates the information from both levels through cross-layer attention connections. Analyze edges whose weights are below the edge weight threshold, combine image features to determine whether they are broken edges, connect and repair real broken edges, and improve the road topology. Among them, the distance threshold is adaptively set according to the average road width and node spatial distribution in the training set to limit the connection range between nodes; the edge weight threshold is automatically selected through the performance index of the validation set to distinguish weakly connected edges from real broken edges. Furthermore, the segmentation decoding module includes a lightweight decoder and a multi-scale skip connection structure, which transmits multi-scale feature maps to the corresponding layers of the decoder through skip connections, preserving multi-scale spatial detail information; a Bi-GRU bidirectional gated recurrent unit is used to fuse features at different scales, and the gating mechanism can adaptively select important features and suppress redundant information; negative cue information is embedded into the decoding process to enhance feature response in fracture edge regions, guiding the multi-task collaborative remote sensing image road extraction model to focus on correcting the segmentation results at these error-prone locations; through multi-layer transposed convolution, the feature map is gradually restored to the original remote sensing road image resolution, outputting the final road segmentation mask.

[0015] Furthermore, the present invention also provides a multi-task collaborative remote sensing image road topology extraction system, comprising: Multi-scale feature extraction module: This module receives input remote sensing road images and uses the SAMRoadPlus structure to extract multi-scale feature maps, outputting multi-scale feature maps of different scale lengths. Key point detection module: It takes multi-scale feature maps as input, constructs a key point detection branch through the HRNetKeypoint architecture, and extracts road key points in remote sensing road images; Dynamic prompt generation module: Generates positive prompt information based on the target heatmap and identifies road break edges as negative prompt information; Topology modeling module: Based on the detected road key points in remote sensing road images, a road topology map is constructed, and edge weight prediction and broken edge repair are achieved through a two-level graph attention network; Segmentation and decoding module: Receives multi-scale feature maps, target heatmaps, and negative cue information, and combines multi-scale features and dynamic cue information to generate the final remote sensing road segmentation result.

[0016] Furthermore, the present invention also provides an electronic device, including at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the processor to perform the above-described multi-task collaborative remote sensing image road topology extraction method.

[0017] The multi-task collaborative remote sensing image road topology extraction method provided by this invention has the following advantages over existing technologies: (1) Segmentation and topology co-optimization: A dynamic prompting mechanism is used to establish information feedback between the segmentation decoding module and the topology modeling module, so as to guide and optimize the topology structure through the segmentation results, thereby improving the coherence and structural consistency of the road network. Multi-scale feature enhancement adopts SAM ViT-L combined with local convolution enhancement mechanism to effectively capture multi-level information from fine-grained to global semantics, thereby improving the recognition ability of narrow roads and low-contrast areas.

[0018] (2) Improved accuracy of key point detection: Based on the HRNetKeypoint key point detection branch, the detection accuracy of key areas such as road intersections and turning points is significantly improved, and the stability of key point positioning is enhanced.

[0019] (3) Enhanced topology repair capability: By constructing road topology relationships and predicting edge weights through a two-layer GATv2 graph attention network, road broken edges are automatically identified and repaired, thereby improving the topological integrity and connectivity of the road network.

[0020] (4) Improved decoding fusion mechanism: Bi-GRU gated fusion and multi-scale jump connection structure are introduced into the segmentation decoding module to achieve adaptive fusion of features at different scales and improve the spatial continuity and detail preservation of the segmentation results.

[0021] In summary, this invention integrates segmentation and topology modeling tasks by introducing a multi-task collaborative mechanism. From feature extraction, key point detection, dynamic prompt generation to topology repair, an end-to-end collaborative optimization process is formed, realizing unified modeling of road mask segmentation and topology reconstruction. While maintaining high segmentation accuracy, it significantly improves the topological connectivity and structural consistency of the road network. It has advantages such as clear structure, high computational efficiency, and strong generalization ability, and can be widely applied in fields such as intelligent transportation, urban planning, disaster emergency response, and automatic map making. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Unless otherwise specified, the positional relationships in the drawings described below are based on the direction in which the components are drawn in the figures.

[0023] Figure 1 , Figure 2 This is a schematic diagram of the network framework structure of a multi-task collaborative remote sensing road topology extraction method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the framework structure of a multi-scale feature extraction module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the framework structure of a topology modeling module provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the framework structure of a segmentation and decoding module provided in an embodiment of the present invention for segmentation and decoding; Figure 6 , Figure 7 A comparison chart of test results provided for an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] In the description of this invention, it should be noted that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more. Additionally, the term "comprising" and any variations thereof mean "at least comprising."

[0026] Example 1 Please see Figure 1 and Figure 2 , Figure 1 , Figure 2 This is a schematic diagram of the network framework structure of a multi-task collaborative remote sensing road topology extraction method provided in an embodiment of the present invention.

[0027] To address the technical problems of severe road fragmentation, poor topological connectivity, and insufficient ability to identify low-contrast fine roads in existing remote sensing image road extraction technologies, or to achieve at least one or more of the aforementioned advantages, an embodiment of the present invention provides a multi-task collaborative remote sensing image road topology extraction method.

[0028] As shown in the figure, this method, based on the SAMRoadPlus model, improves the connectivity and overall extraction accuracy of the topological structure of roads in remote sensing images by improving its architecture and training strategy. The method includes the following steps: Data preparation steps: Obtain publicly available remote sensing road datasets, which contain remote sensing road images and corresponding annotation data. This embodiment preferably uses the publicly available CityScale dataset and SpaceNet dataset, both of which cover remote sensing road scenes with multiple terrains and resolutions, and provide high-precision pixel-level annotations.

[0029] For example, the CityScale dataset contains 5,000 high-resolution 1024×1024 pixel images, covering road networks in different terrains such as plains and hilly cities, with an annotation accuracy of over 95%. Road types include arterial roads, secondary roads, and local roads. The SpaceNet dataset contains 3,000 2048×2048 pixel images, focusing on urban-rural fringe areas, suburbs, and some disaster-affected areas. Road features include complex situations such as narrow roads, roads obscured by vegetation, and broken roads, with an image resolution of 0.5 meters.

[0030] Preferably, the acquired remote sensing road dataset can be divided into a training set, a validation set, and a test set using random sampling in an 8:1:1 ratio. The training set is used for model training; the validation set is used for performance monitoring and hyperparameter tuning during training, model selection, and early stopping based on validation metrics to determine optimal weights; the test set is used for a one-time objective evaluation of data not used in training after the model and hyperparameters are fixed, without participating in any training or hyperparameter tuning. The final result is a training set of 6400 images, a validation set of 800 images, and a test set of 800 images, avoiding data bias that could affect the model's generalization ability.

[0031] The segmented remote sensing road dataset is preprocessed to obtain a processed remote sensing road dataset, which facilitates subsequent model training, accelerates convergence, and improves cross-scene robustness.

[0032] In practice, the input remote sensing road imagery is first divided into sub-image sets of 256×256 pixels. and subgraph set satisfy:

[0033]

[0034]

[0035] And standardize. :

[0036] in, These are the horizontal pixel coordinates (positive to the right). These are vertical pixel coordinates (positive for downward). and These are the horizontal coordinates of the left and right boundaries of the current cropping window. and Use the vertical index for the top / bottom boundaries of the window. The mean calculated for the channel and , The standard deviation vector and .

[0037] To improve the model's adaptability to different scenarios and imaging conditions, synchronous data augmentation operations are performed on the segmented remote sensing road dataset, including but not limited to random flipping, deformation scaling, and color and brightness adjustment of the remote sensing road images and labeled data, in order to obtain the processed remote sensing road dataset.

[0038] Model training steps: A multi-task collaborative remote sensing image road extraction model was constructed, and the processed remote sensing road dataset was input into the multi-task collaborative remote sensing image road extraction model for training. The model was then saved to obtain the trained multi-task collaborative remote sensing image road extraction model.

[0039] In specific implementation, such as Figure 3 , Figure 4 As shown, feature extraction is first performed on the input remotely sensed road image to output multi-scale feature maps with different strides. SAM ViT-L is used as the encoder to divide the input remotely sensed road image into 256×256 image patches, which are then converted into embedding vectors through linear projection. After processing by a multi-layer Transformer encoder, features are extracted at different levels, outputting four feature maps at different scales with strides of 4, 8, 16, and 32, respectively. , , , This allows for the capture of semantic information ranging from fine-grained to coarse-grained.

[0040] After feature extraction, a local feature enhancement convolutional layer is added to represent narrow roads. The extracted feature maps are convolved using a 3×3 convolution kernel, and the fused enhanced features are obtained through a dilated convolution module U and channel attention z. And finally merged to obtain the fused features. This enhances the feature representation capability for small targets such as narrow roads and improves the detection accuracy for low-contrast roads. The fused features... Expressed as:

[0041] Using multi-scale feature maps as input, a keypoint detection branch is constructed through the HRNetKeypoint architecture to extract road keypoints from remotely sensed road images. In practice, the HRNetKeypoint architecture is used to construct a high-resolution feature branch, which includes four parallel multi-resolution sub-networks (with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, respectively). The parallel multi-resolution sub-networks process multi-scale feature maps to maintain high-resolution feature representation.

[0042] Introducing asymmetric Gaussian kernels to generate target heatmaps It adapts to road direction features, with a heatmap resolution of 1 / 4 of the input image. It also combines Focal Loss (balancing the foreground / background ratio) and Repulsion Loss (suppressing interference between key points) for joint optimization, improving the recognition accuracy of hard-to-detect key points.

[0043] Then target heatmap Expressed as:

[0044] Keypoint detection is performed using heatmap regression. A heatmap response value is predicted for each potential road keypoint location. The higher the response value, the greater the probability that the location is a keypoint.

[0045] Specifically, unlike conventional road extraction processes that only output road masks, this scheme adds a key point heatmap regression branch on top of the encoded features. It uses heatmap regression to detect key road points; that is, on a 1 / 4 resolution key point heatmap, it regresses the confidence response value for each pixel location as a key road point. The preferred key road points include road endpoints, intersection nodes, obvious turning points, and skeleton bifurcation points.

[0046] During training, the centerline can be obtained from the ground truth road mask through skeletonization, and the key points mentioned above can be extracted based on the neighborhood degree and curvature of the skeleton pixels, serving as key point supervision annotations. During inference, the key point detection branch outputs a predicted heatmap; the higher the response value, the greater the probability that the location is a key point. This transforms the key point localization problem into dense pixel-level regression, improving robustness in occluded, low-contrast, and narrow road scenes, and providing structural priors for subsequent cue-guided segmentation and topology repair.

[0047] Compared to the traditional symmetric Gaussian kernel, the asymmetric Gaussian kernel can better adapt to the directional characteristics of roads, with the variance along the road direction being greater than that in the vertical direction, thereby improving the detection accuracy of key points such as road intersections and turning points.

[0048] Based on the generated target heatmap, positive prompts are generated, and the multi-scale feature map, target heatmap, and positive prompts are input into the segmentation and decoding module to generate the preliminary road segmentation mask for the current iteration.

[0049] In practice, local maxima are extracted from the generated target heatmap as positive prompts. These local maxima refer to the highest response points in the image relative to their neighborhood, corresponding to key locations on the road.

[0050] Subsequently, the multi-scale feature map, target heatmap, and positive prompt information are input into the segmentation and decoding module to generate the preliminary road segmentation mask for the current iteration, which includes the following steps: Hint Encoding: The positive hint information is encoded by mapping the coordinates of the positive hint points to the corresponding feature map scale and performing position encoding to obtain the positive hint embedding vector; Heatmap Prior Alignment and Enhancement: The target heatmap and the multi-scale feature map are scale aligned (upsampled / downsampled to the same resolution), and the target heatmap is used as a spatial prior weight to weight and enhance the features of road-related areas to obtain heatmap guidance features; Multi-scale fusion and cue fusion: Multi-scale feature maps are fused from top to bottom / bottom to top (including but not limited to feature concatenation, element-wise addition, or attention fusion) to obtain fused features; positive cue embedding vectors are introduced into the fused features, and cue-guided feature updates are achieved through cross-attention or gating mechanisms to obtain cue-guided decoding features; Mask decoding output: The prompt guidance decoding features are decoded and upsampled layer by layer to output a road segmentation probability map; the road segmentation probability map is processed by an activation function and thresholded to obtain the preliminary road segmentation mask for the current iteration.

[0051] The topology modeling module then constructs a road skeleton and road map on the initial road segmentation mask, identifies the broken edges and generates negative prompts, which are fed back to the segmentation decoding module to correct and refine the initial road segmentation mask and output the updated road segmentation mask.

[0052] The specific implementation includes the following steps: Mask preprocessing: Connectivity analysis and morphological processing are performed on the preliminary road segmentation mask, including hole filling, lightweight closure operation of fractures and gaps, burr removal and small connected component removal, in order to reduce the impact of noise on skeletonization. Road skeleton extraction: Perform refinement / skeletonization operations on the preprocessed road segmentation mask to obtain a road centerline skeleton with a width of one pixel; Skeleton purification and pruning: The road centerline skeleton is pruned to remove pseudo-branches and isolated skeleton segments with lengths less than a preset threshold. The road centerline skeleton is then smoothed and discontinuities are corrected to obtain the purified road skeleton. The preset threshold is the skeleton pruning length threshold used to identify pseudo-branches; it is adaptively determined based on the image ground resolution / road width estimate. Node detection: On the cleaned road skeleton, the neighborhood connectivity of skeleton pixels is calculated. Endpoint pixels with a degree of 1 are extracted as endpoint nodes, and pixels with a degree ≥ 3 are extracted as intersection nodes. A curvature threshold can be used to extract turning nodes, forming a set of road graph nodes. The curvature threshold is used to determine the turning nodes of the skeleton. Select the distances along the skeleton path before and after. A point of skeleton pixels and Construct direction vector , Calculate local rotation angle and discrete curvature ,when At that time, the skeleton point p is determined to be a turning point, where The preset curvature threshold is adaptively set based on the remote sensing image resolution and the estimated road width. Edge tracing and edge generation: Starting from the road graph node, path tracing is performed along the cleaned road skeleton pixels. The skeleton path between any two adjacent nodes is defined as a road edge. The geometric sequence (pixel chain / polyline), length, direction and other attributes of each edge are recorded. Road graph construction: Using the set of road graph nodes as the set of vertices and the road edges as the set of edges, a road graph is constructed, and the positions of the nodes and edges in the road graph are mapped back to the coordinates of the original remote sensing road image.

[0053] Break edge identification: Break edges in the road network are identified through the topology modeling module; these are road segments that should be connected but are not connected in the initial road segmentation mask. The endpoints of the break edges and their surrounding areas are input as negative cue information into the segmentation decoding module, guiding the multi-task collaborative remote sensing image road extraction model to focus on these error-prone areas. Specifically, this includes the following steps: Break edge identification: Based on the road map and road skeleton, break edges / suspected break areas are identified. Specifically, this may include: detecting nearest neighbor endpoint pairs (the distance between endpoints is less than a threshold but they are not connected), detecting gaps caused by occlusion (there are holes between the skeletons and they are in the same direction), detecting weak segments with low confidence (corresponding to low segmentation probability and poor continuity), etc., to obtain a candidate set of break edges. Negative prompt generation: For each candidate fracture edge, negative prompt information is generated in its gap area / misconnection area. The negative prompt information can be negative point prompts or negative box prompts, and its coordinates and scale are unified to the input space of the segmentation decoding module to suppress misconnections, eliminate false roads and guide subsequent mask refinement.

[0054] Mask Correction: The segmentation module corrects the initial road segmentation mask based on positive and negative prompts and outputs the updated road segmentation mask.

[0055] Furthermore, the topology modeling module optimizes road connectivity based on the updated road segmentation mask, and the two iteratively optimize until convergence, outputting the final road segmentation result.

[0056] In some preferred embodiments, the topology modeling module includes a two-level graph attention (GATv2) network, which uses the detected road key points as graph nodes and connects adjacent nodes to form edges through a distance threshold. The features of each edge are composed of the concatenation of the feature vectors of the two nodes and the direction vector between the two nodes.

[0057] The first-level graph attention network calculates the importance weight of each edge through the attention mechanism, learns the correlation strength between nodes, and outputs a preliminary edge weight prediction. The second-level graph attention network receives the edge weight prediction results output by the first-level graph attention network, further refines the edge weight prediction, and integrates the information of the two levels through cross-layer attention connections. Analyze edges whose weights are below the edge weight threshold, combine image features to determine whether they are broken edges, connect and repair real broken edges, and improve the road topology. Understandably, the distance threshold is adaptively set based on the average road width and node spatial distribution in the training set to limit the connection range between nodes; the edge weight threshold is automatically selected through the performance indicators of the validation set to distinguish weakly connected edges from true broken edges.

[0058] In some preferred embodiments, such as Figure 5 As shown, the segmentation and decoding module includes a lightweight decoder and a multi-scale skip connection structure. Multi-scale feature maps are passed to the corresponding layers of the decoder via skip connections, preserving multi-scale spatial details. A Bi-GRU bidirectional gated recurrent unit is used to fuse features at different scales. The gating mechanism adaptively selects important features and suppresses redundant information. Negative cue information is embedded into the decoding process to enhance feature responses in fracture edge regions, guiding the multi-task collaborative remote sensing image road extraction model to focus on correcting segmentation results at these error-prone locations. Through multi-layer transposed convolution, the feature maps are gradually restored to the original remote sensing road image resolution, outputting the final road segmentation result.

[0059] Road extraction steps: The real-time remote sensing road dataset is input into the trained multi-task collaborative remote sensing image road extraction model, which outputs remote sensing road images.

[0060] By introducing a multi-task collaborative mechanism, the segmentation task and the topology modeling task are integrated. From feature extraction, key point detection, dynamic prompt generation to topology repair, an end-to-end collaborative optimization process is formed, realizing unified modeling of road mask segmentation and topology reconstruction. While maintaining high segmentation accuracy, it significantly improves the topological connectivity and structural consistency of the road network, and has the advantages of clear structure, high computational efficiency and strong generalization ability.

[0061] Example 2 To further verify the effectiveness of the multi-task collaborative remote sensing road topology extraction method (hereinafter referred to as "this method") provided in the above embodiments, a comparative experiment was conducted between this method and traditional edge detection methods and morphological methods under the same experimental conditions.

[0062] 1. Experimental Data and Environment We used the existing CityScale and SpaceNet public road datasets, with an image spatial resolution of approximately 0.5m, both providing pixel-level road annotations. The datasets were randomly divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used to train the model of our proposed method, the validation set was used for parameter tuning and model selection, and the test set was used only for final performance evaluation. All three methods were compared on the same test set.

[0063] The experimental platform consisted of a workstation environment with two NVIDIA RTX 3090 GPUs. The deep learning part was implemented using the PyTorch 2.0 framework, while the traditional edge detection and morphological algorithms were implemented using OpenCV and Python. For the method of this invention, the optimizer was AdamW, the batch size was set to 16, the initial learning rate was 0.001, and the total number of training epochs was 150. Early stopping was triggered by validation set metrics to determine the optimal model parameters.

[0064] 2. Evaluation Indicators To comprehensively evaluate the accuracy and topological integrity of road extraction, this embodiment uses the following metrics: (1) Overall accuracy (OA) at the pixel level; (2) Precision, Recall, and F1 score; (3) Average intersection-union ratio (mIoU) of road categories.

[0065] The above metrics reflect the degree of overlap between the predicted mask and the actual annotation at the pixel level.

[0066] To measure the connectivity and structural consistency of the road network, this embodiment constructs a road map structure based on the predicted road skeleton and calculates the following metrics: (1) Connectivity: The proportion of road endpoints that remain connected in the actual road map that remain connected in the predicted road map; (2) Number of broken paths: The number of times that actually connected roads are incorrectly interrupted in the prediction results; (3) Number of incorrect connections: The number of connections in the prediction results that incorrectly connect two roads that should not be connected; Among them, the higher the connectivity rate and the fewer the number of broken lines and erroneous connections, the more complete the road topology and the closer it is to the real road network.

[0067] IV. Experimental Results and Analysis Under the unified data and environmental conditions described above, this embodiment employs three methods to extract roads from the test set and statistically analyzes various indicators. The experimental results are summarized in Table 1 (segmentation accuracy comparison) and Table 2 (topology structure indicator comparison), as follows: Table 1 Comparison of Segmentation Accuracy

[0068] As shown in Table 1, our proposed method outperforms traditional edge detection and morphological methods in pixel-level accuracy metrics such as OA, F1, and mIoU. Particularly on SpaceNet datasets containing narrow roads and occluded scenes, our method effectively improves the recall rate for narrow roads through multi-scale feature extraction and local convolution enhancement mechanisms, enabling more accurate localization of key road points while maintaining overall segmentation accuracy.

[0069] Table 2 Comparison of Topology Indicators

[0070] As shown in Table 2, the method of this invention significantly outperforms the comparative methods in all road topology evaluation metrics. Specifically, the connectivity rate of this method is 76.38%, which is approximately 40.75 percentage points higher than the edge detection method (35.63%) and 30.56 percentage points higher than the morphological method (45.82%). This indicates that on the same test set, this invention can significantly reduce breaks in the road network, ensuring that more truly connected road endpoints remain connected in the prediction results.

[0071] Meanwhile, the method of this invention results in only 16 fractures. Compared to 25 edge detection methods, respectively. 36 morphological methods Reduced by approximately 9 items. and 20 items· The number of erroneous connections also decreased from 19 in the comparison method. and 18 articles· Reduced to 9 items This effectively suppressed false connections between different roads.

[0072] Furthermore, the average path deviation of the method of the present invention is 3.15m, which is reduced by approximately 2.12m and 4.38m compared to the edge detection method (5.27m) and morphological method (7.53m), respectively, indicating that the predicted road centerline is closer to the actual road direction in spatial location.

[0073] In summary, the method of the present invention significantly reduces road breaks and erroneous connections while ensuring high road connectivity, and improves the accuracy of road geometric positions. The road topology is more complete and closer to the real road network, thus verifying the effectiveness and superiority of the method of the present invention.

[0074] 5. Result diagram To more intuitively demonstrate the comparison effect, this embodiment selects several representative areas from the test set to create a comparison diagram. For example... Figure 6 and Figure 7 As shown, from left to right, the images are: the original remote sensing road image, the real road annotation, the road extraction result of the edge detection method, the road extraction result of the morphological method, and the road extraction result of this method.

[0075] Example 3 In another specific embodiment, the present invention also provides a multi-task collaborative remote sensing image road topology extraction system, used to implement the multi-task collaborative remote sensing image road topology extraction method described in Embodiment 1, comprising: Multi-scale feature extraction module: This module receives input remote sensing road images and uses the SAMRoadPlus structure to extract multi-scale feature maps, outputting multi-scale feature maps of different scale lengths. Key point detection module: Constructs a key point detection branch using the HRNetKeypoint architecture to extract road key points from remotely sensed road images; Dynamic prompt generation module: Generates positive prompts based on the target heatmap and identifies road break edges as negative prompts. Topology modeling module: Based on the detected road key points in remote sensing road images, a road topology map is constructed, and edge weight prediction and broken edge repair are achieved through a two-level graph attention network (GATv2). Segmentation and decoding module: Receives multi-scale feature maps, target heatmaps, and negative cue information, and combines multi-scale features and dynamic cue information to generate the final remote sensing road segmentation result.

[0076] Example 4 This embodiment also provides an electronic device, including at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by at least one processor, the instructions being executed by at least one processor to cause the processor to perform the multi-task collaborative remote sensing image road topology extraction method of the above embodiment.

[0077] In summary, this invention integrates segmentation and topology modeling tasks by introducing a multi-task collaborative mechanism. From feature extraction, key point detection, dynamic prompt generation to topology repair, an end-to-end collaborative optimization process is formed, realizing unified modeling of road mask segmentation and topology reconstruction. While maintaining high segmentation accuracy, it significantly improves the topological connectivity and structural consistency of the road network. It has advantages such as clear structure, high computational efficiency, and strong generalization ability, and can be widely applied in fields such as intelligent transportation, urban planning, disaster emergency response, and automatic map making.

[0078] Furthermore, those skilled in the art should understand that although many problems exist in the prior art, each embodiment or technical solution of the present invention can be improved in only one or a few aspects, without necessarily solving all the technical problems listed in the prior art or the background art simultaneously. Those skilled in the art should understand that any content not mentioned in a claim should not be construed as a limitation on that claim.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-task collaborative method for extracting road topology from remote sensing images, based on the SAMRoadPlus model, improves the connectivity and overall extraction accuracy of road topology in remote sensing images by improving its architecture and training strategy, characterized in that: Includes the following steps: Data preparation steps: Obtain publicly available remote sensing road datasets, which include remote sensing road images and labeled data, and preprocess the obtained remote sensing road datasets to obtain processed remote sensing road datasets. Model training steps: Construct a multi-task collaborative remote sensing image road extraction model, input the processed remote sensing road dataset into the multi-task collaborative remote sensing image road extraction model for feature extraction, and output multi-scale feature maps of different lengths; Using multi-scale feature maps as input, a high-resolution feature branch is constructed through the HRNetKeypoint architecture, and an asymmetric Gaussian kernel is introduced to generate a target heatmap. Extract key road points from remotely sensed road images; Based on the target heatmap, positive prompt information is generated, and the multi-scale feature map, target heatmap and positive prompt information are input into the segmentation and decoding module to generate the preliminary road segmentation mask for the current iteration. The topology modeling module constructs a road skeleton and road map on the initial road segmentation mask, identifies broken edges and generates negative prompts, feeds the negative prompts back to the segmentation decoding module to correct and refine the initial road segmentation mask, outputs the updated road segmentation mask, and saves it to obtain the trained multi-task collaborative remote sensing image road extraction model. Road extraction steps: Input the real-time remote sensing road dataset into the trained multi-task collaborative remote sensing image road extraction model, and output the remote sensing road image.

2. The method for extracting road topology from remote sensing images using multi-task collaborative methods according to claim 1, characterized in that: The preprocessing of the remote sensing road dataset includes data augmentation operations, which include, but are not limited to, performing random flipping, deformation scaling, and color and brightness adjustments on the remote sensing road images and labeled data simultaneously.

3. The method for extracting road topology from remote sensing images using multi-task collaborative methods according to claim 1, characterized in that: The process of inputting the processed remote sensing road dataset into a multi-task collaborative remote sensing image road extraction model for feature extraction, in order to output multi-scale feature maps of different lengths, includes the following steps: The input remote sensing road image is used as an encoder for feature extraction. The input remote sensing road image is divided into 256×256 image blocks and converted into embedding vectors through linear projection. After processing by a multi-layer Transformer encoder, features are extracted at different levels, and four feature maps with strides of 4, 8, 16 and 32 are output to capture semantic information from fine-grained to coarse-grained. After feature extraction, a local feature enhancement convolutional layer is added to enhance the feature representation ability of narrow roads and small targets and improve the detection accuracy of low-contrast roads.

4. The method for extracting road topology from remote sensing images using multi-task collaborative methods according to claim 1, characterized in that: The step of taking multi-scale feature maps as input and constructing a keypoint detection branch through the HRNetKeypoint architecture to extract road keypoints from remotely sensed road images includes: The process involves taking multi-scale feature maps as input, constructing high-resolution feature branches through the HRNetKeypoint architecture, and introducing asymmetric Gaussian kernels to generate target heatmaps. Combined with the target heatmaps, a heatmap regression method is used to detect key points. A heatmap response value is predicted for each potential road key point location, and the higher the response value, the greater the probability that the location is a key point.

5. The method for extracting road topology from remote sensing images using multi-task collaborative methods according to claim 4, characterized in that: The process of generating positive prompt information based on the target heatmap and inputting the multi-scale feature map, the target heatmap, and the positive prompt information into the segmentation decoding module to generate the preliminary road segmentation mask for the current iteration includes the following steps: Local maxima points are extracted from the generated target heatmap as positive prompts; these points correspond to key locations on the road. The positive prompt information is encoded by mapping the coordinates of the positive prompt points to the corresponding feature map scale and performing position encoding to obtain the positive prompt embedding vector. The target heatmap is scale-aligned with the multi-scale feature map, and the target heatmap is used as a spatial prior weight to weight and enhance the features of road-related areas, thus obtaining heatmap-guided features. Multi-scale feature maps are fused from top to bottom / bottom to top to obtain fused features. Positive cue embedding vectors are introduced into the fused features. Cue-guided feature updates are achieved through cross attention or gating mechanisms to obtain cue-guided decoding features. The prompt-guided decoding features are decoded and upsampled layer by layer to output a road segmentation probability map; the road segmentation probability map is processed by an activation function and thresholded to obtain the preliminary road segmentation mask for the current iteration.

6. The method for extracting road topology from remote sensing images using multi-task collaborative methods according to claim 5, characterized in that: The process of constructing a road skeleton and road map on the initial road segmentation mask using the topology modeling module, identifying broken edges and generating negative prompts, feeding these negative prompts back to the segmentation decoding module to correct and refine the initial road segmentation mask, and outputting the updated road segmentation mask includes the following steps: Connectivity analysis and morphological preprocessing are performed on the preliminary road segmentation mask, including but not limited to hole filling, lightweight closure operation of fractures and gaps, burr removal and small connected component removal, in order to reduce the impact of noise on skeletonization. Perform a thinning / skeletonization operation on the preprocessed road segmentation mask to obtain a road centerline skeleton with a single pixel width; The road centerline skeleton is pruned to remove pseudo-branches and isolated skeleton segments with a length less than a preset threshold. The road centerline skeleton is then smoothed and broken point corrected to obtain a purified road skeleton. Calculate the neighborhood connectivity of the skeleton pixels on the purified road skeleton, extract the endpoint pixels with a degree of 1 as endpoint nodes, and the pixels with a degree of ≥3 as intersection nodes. Combine the curvature threshold to extract the turning nodes and form a set of road graph nodes. Starting from a road graph node, path tracing is performed along the cleaned road skeleton pixels. The skeleton path between any two adjacent nodes is defined as a road edge, and the geometric sequence, length, direction and other attributes of each edge are recorded. A road graph is constructed using the set of road graph nodes as the set of vertices and the set of road edges as the set of edges. The positions of nodes and edges in the road graph are then mapped back to the coordinates of the original remote sensing road image. The topology modeling module identifies broken edges in the road network, i.e. road segments that should be connected but are not. The endpoints of the broken edges and their surrounding areas are used as negative cue information input to the segmentation and decoding module to guide the model to focus on these error-prone areas. The segmentation module corrects the initial road segmentation mask based on positive and negative prompts and outputs the updated road segmentation mask. The topology modeling module optimizes road connectivity based on the updated road segmentation mask. The two modules iterate and optimize until convergence, and then output the final road segmentation result.

7. The method for extracting road topology from remote sensing images using multi-task collaborative methods according to claim 6, characterized in that: The topology modeling module includes a two-level graph attention network, which uses the detected road key points as graph nodes and connects adjacent nodes to form edges through a distance threshold. The features of each edge are composed of the concatenation of the feature vectors of the two nodes and the direction vector between the two nodes. The first-level graph attention network calculates the importance weight of each edge through the attention mechanism, learns the correlation strength between nodes, and outputs a preliminary edge weight prediction. The second-level graph attention network receives the edge weight prediction results output by the first-level graph attention network, further refines the edge weight prediction, and integrates the information from both levels through cross-layer attention connections. Analyze edges whose weights are below the edge weight threshold, combine image features to determine whether they are broken edges, connect and repair real broken edges, and improve the road topology. Among them, the distance threshold is adaptively set according to the average road width and node spatial distribution in the training set to limit the connection range between nodes; the edge weight threshold is automatically selected through the performance index of the validation set to distinguish weakly connected edges from real broken edges.

8. The method for extracting road topology from remote sensing images using multi-task collaborative methods according to claim 7, characterized in that: The segmentation and decoding module includes a lightweight decoder and a multi-scale skip connection structure. Multi-scale feature maps are passed to the corresponding layers of the decoder via skip connections, preserving multi-scale spatial details. A Bi-GRU bidirectional gated recurrent unit is used to fuse features at different scales. The gating mechanism adaptively selects important features and suppresses redundant information. Negative cue information is embedded into the decoding process to enhance feature responses in fracture edge regions, guiding the multi-task collaborative remote sensing image road extraction model to focus on correcting the segmentation results at these error-prone locations. Through multi-layer transposed convolution, the feature maps are gradually restored to the original remote sensing road image resolution, outputting the final road segmentation mask.

9. A multi-task collaborative remote sensing image road topology extraction system, characterized in that: include: Multi-scale feature extraction module: This module receives input remote sensing road images and uses the SAMRoadPlus structure to extract multi-scale feature maps, outputting multi-scale feature maps of different scale lengths. Key point detection module: It takes multi-scale feature maps as input, constructs a key point detection branch through the HRNetKeypoint architecture, and extracts road key points in remote sensing road images; Dynamic prompt generation module: Generates positive prompt information based on the target heatmap and identifies road break edges as negative prompt information; Topology modeling module: Based on the detected road key points in remote sensing road images, a road topology map is constructed, and edge weight prediction and broken edge repair are achieved through a two-level graph attention network; Segmentation and decoding module: Receives multi-scale feature maps, target heatmaps, and negative cue information, and combines multi-scale features and dynamic cue information to generate the final remote sensing road segmentation result.

10. An electronic device, characterized in that: The method includes at least one processor and a memory communicatively connected to the processor, wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, causes the processor to perform the multi-task collaborative remote sensing image road topology extraction method as described in any one of claims 1-8.

Images