A multi-task road extraction method with boundary prior constraints

Abstract

The application discloses a kind of boundary priori constraint multi-task road extraction method, it is related to remote sensing image processing technical field, comprising the following steps: obtaining original remote sensing image, original remote sensing image is pretreated, and multi-scale feature extraction is carried out, and the output multi-scale coding feature map;The channel feature weight and spatial feature weight of multi-scale coding feature map are calculated and decoupled weighting is carried out, and the output decoupled coding feature map;Feature enhancement and multi-scale modeling are carried out to multi-scale coding feature map, and the output boundary prediction map and road segmentation prediction map;Road segmentation prediction map is carried out superpixel segmentation, and the feature vector corresponding to superpixel graph node is output;The feature vector corresponding to superpixel graph node is carried out topological relation reasoning and attention weight calculation, and the output connectivity prediction map;Boundary prediction map, road segmentation prediction map and connectivity prediction map are carried out weighted fusion processing, and connectivity repair is carried out, and the output road extraction result.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a multi-task road extraction method with boundary prior constraints. Background Technology

[0002] Road extraction from remote sensing imagery is a core supporting technology in fields such as urban planning, intelligent transportation, disaster emergency response, and geographic information updates, directly determining the accuracy of road network modeling, the timeliness of navigation data, and the efficiency of emergency response. With the rapid popularization of high-resolution remote sensing satellites and aerial photogrammetry, imagery details are richer, but this also brings challenges such as complex backgrounds, dense occlusion, and large scale differences. Traditional road extraction methods based on thresholding, edge detection, mathematical morphology, and machine learning are no longer sufficient to meet the requirements for high accuracy and robustness. Deep learning-based image segmentation methods, with their powerful feature representation and end-to-end learning capabilities, have become the mainstream technical approach for remote sensing road extraction, among which semantic segmentation models, represented by encoder-decoder architectures, are the most widely used.

[0003] Current mainstream technologies still primarily rely on single-task semantic segmentation methods, with UNet and DeepUNet being typical examples. UNet, a classic encoder-decoder model originating in medical image segmentation, has been widely adapted for remote sensing road extraction tasks. The network employs a symmetrical encoder-decoder structure: the encoder downsamples layer by layer through continuous convolution and pooling operations, extracting deep semantic features such as road texture and contours; the decoder restores spatial resolution layer by layer through deconvolution or upsampling, and makes skip connections with corresponding layers of the encoder to compensate for details lost during downsampling, ultimately outputting a binary road segmentation map. DeepUNet is a deep encoder-decoder segmentation model optimized for high-resolution remote sensing images. It deepens the network layers and enhances multi-scale feature extraction capabilities compared to UNet. The model increases the receptive field through deeper encoding modules, adapting to the coexistence of wide, narrow, and branch roads in remote sensing images; simultaneously, it improves skip connections and feature fusion methods, strengthening the combination of high and low-level features, thus improving the recall rate of road regions to some extent.

[0004] The two single-task segmentation methods described above can achieve basic road extraction in conventional scenarios, but they have significant limitations in complex high-resolution remote sensing scenarios. For example, strong shadows cast by buildings, vegetation, and bridges can obscure road pixel features, and single-task models are prone to misclassifying shadowed areas as background, causing road interruptions and breaks, making it difficult to recover complete connected paths. Summary of the Invention

[0005] To solve the above problems, such as Figure 1As shown, this invention proposes a Boundary-Prior Multi-Task Network (BPM-Net) for road extraction with boundary prior constraints, which includes a shared encoder, a three-branch decoder, a boundary prior fusion module, and a connectivity repair module.

[0006] Shared encoders are used to extract multi-scale features from remote sensing images and to decouple those features. The three-branch decoder includes a boundary decoder, a road decoder, and a connectivity decoder, which are used to generate boundary prediction maps, road segmentation prediction maps, and connectivity prediction maps, respectively. The boundary prior fusion module is used to fuse the three prediction graphs, including boundary consistency loss, adaptive fusion weights, and topological constraint loss. The connectivity repair module is used to repair the connectivity of the prediction results to obtain the final road extraction results.

[0007] This invention provides a multi-task road extraction method with boundary prior constraints, comprising the following steps: S1: Acquire the original remote sensing image, preprocess the original remote sensing image, and output the preprocessed remote sensing image. S2: Perform multi-scale feature extraction on the preprocessed remote sensing image and output a multi-scale coded feature map; S3: Calculate the channel feature weights and spatial feature weights of the multi-scale encoded feature map, decouple and weight them, and output the decoupled encoded feature map; S4: Perform feature enhancement and multi-scale modeling on the multi-scale encoded feature map, and output the boundary prediction map and the road segmentation prediction map; S5: Perform superpixel segmentation on the road segmentation prediction map, and then perform mean pooling based on the decoupled encoded feature map to output the feature vectors corresponding to the superpixel map nodes. S6: Perform topological relationship reasoning and attention weight calculation on the feature vectors corresponding to the superpixel graph nodes, and output a connectivity prediction graph; S7: Perform weighted fusion processing on the boundary prediction map, the road segmentation prediction map, and the connectivity prediction map, and output the fused road feature map; S8: Perform connectivity repair on the fused road feature map and output the road extraction results.

[0008] In summary, the present invention has at least the following beneficial effects: 1. This invention incorporates the boundary prediction map as an explicit prior constraint into the loss function, constructing a boundary consistency loss that forces the road segmentation result to maintain spatial alignment with the boundary prediction, thus suppressing boundary ambiguity from the training source. Simultaneously, the boundary decoder incorporates a Sobel convolution enhancement module, employing fixed 3×3 horizontal and vertical gradient convolution kernels to accurately extract gradient features from road edges, strengthening feature responses in weak boundary regions. Combined with an adaptive fusion weight mechanism, a multilayer perceptron (MLP) dynamically allocates feature weights based on boundary confidence, increasing the proportion of boundary features at boundary locations and enhancing road semantic features in non-boundary regions. This allows the model to concentrate computational resources on boundary optimization, significantly improving road edge localization accuracy and segmentation contour smoothness.

[0009] 2. This invention uses a Graph Attention Network (GAT) to infer the road topology map constructed by SLIC superpixels. Under boundary prior constraints, it learns the connectivity relationships between road segments, achieving explicit modeling of the road network structure. A topological constraint loss is introduced during training to ensure high spatial consistency between connectivity prediction and road segmentation results, avoiding topological misalignment and incorrect connections. During the inference phase, a connectivity repair module is activated to automatically detect broken road endpoints, evaluate connection reliability, and complete breakpoint repair. This dual approach of prediction and post-processing ensures the integrity and continuity of the road network, effectively solving the problems of road segment discrepancies and topological fragmentation.

[0010] 3. The road decoder of this invention employs a multi-scale dilated convolution module, using four parallel branches with dilation rates of 1, 2, 4, and 8 to extract features from different receptive fields in parallel, taking into account multi-scale road information such as narrow roads, wide roads, and intersections. It combines a boundary-guided dynamic feature fusion strategy to replace the traditional fixed-weight fusion method, adaptively adjusting the multi-scale feature ratio according to the boundary position. Simultaneously, a joint loss function is used to simultaneously optimize the three tasks of road segmentation, boundary detection, and connectivity prediction, achieving multi-task collaborative learning and gradient balancing, comprehensively improving the accuracy, completeness, and intersection-union ratio of the overall road network extraction.

[0011] 4. This invention addresses the problem of road occlusion and breaks caused by building and vegetation shadows. It employs a closed-loop solution based on a connectivity prediction graph and a connectivity repair mechanism: first, a connectivity decoder identifies potential connectivity relationships within the shadowed area; then, intelligent repair is achieved through breakpoint detection, directional consistency judgment, and connection reliability scoring, restoring the occluded road segments without introducing additional prior data. Simultaneously, boundary prior constraints and multi-scale feature enhancement enhance the model's resistance to shadow interference, ensuring stable output even under complex lighting and dense occlusion scenarios, significantly improving the robustness of road extraction. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is the overall architecture diagram of the BPM-Net of this invention; Figure 2 This is an architecture diagram of the shared encoder of the present invention; Figure 3 This is an architecture diagram of the feature separation module of the present invention; Figure 4 This is an architecture diagram of the boundary decoder of the present invention; Figure 5 This is an architecture diagram of the road decoder of the present invention; Figure 6 This is an architecture diagram of the connectivity decoder of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] The following is in conjunction with the appendix Figures 1 to 6 The present invention will be described in further detail below.

[0016] A multi-task road extraction method with boundary prior constraints includes the following steps: Step 1: Data preprocessing.

[0017] The original remote sensing images are acquired, and their pixel values ​​are mapped to a standard numerical range to eliminate numerical differences caused by different lighting, imaging devices, and shooting conditions, making the training of this model more stable.

[0018] The remote sensing images are uniformly adjusted to an input size acceptable to this model to ensure scale consistency in the feature extraction and decoding processes; at the same time, the images are cropped at a fixed size during the training phase to make the model adaptable to road extraction tasks of images with different resolutions.

[0019] Furthermore, data augmentation is employed to expand the diversity of training samples, alleviate overfitting, and improve the model's generalization ability and robustness in complex scenarios. Specifically, a combination of random cropping, horizontal flipping, vertical flipping, color jittering, and the addition of Gaussian noise is used to perform random transformations on the remote sensing images.

[0020] Step 2: Extraction of multi-scale features.

[0021] The preprocessed remote sensing images are input into the shared encoder, such as... Figure 2 As shown, the shared encoder uses a ResNeXt-50 backbone network with parameters configured as 32×4d and loaded with ImageNet pre-trained weights. The ResNeXt-50 network is a deep residual network improved on the ResNet-50 network. By introducing the concepts of grouped convolution and cardinality, it improves the expressive power of this model without increasing computational complexity.

[0022] The ResNeXt-50 backbone network consists of an initial convolutional layer and four residual stages. The initial convolutional layer uses a 7×7 convolutional kernel with a stride of 2, padding of 3, and 64 output channels. It is followed by a batch normalization layer and a ReLU activation function, and then downsampling is performed through a 3×3 max pooling layer with a stride of 2.

[0023] The four residual stages contain 3, 4, 6, and 3 residual blocks respectively. Each residual block uses 32 groups of grouped convolutions, with each group containing 4 channels, i.e., a parameter configuration of 32×4d. The structure of each residual block includes: the first layer is a 1×1 dimensionality-reducing convolution with C input channels and 128 output channels; the second layer is a 3×3 grouped convolution with 128 input channels, 128 output channels, and 32 groups; the third layer is a 1×1 dimensionality-increasing convolution with 128 input channels and 4C output channels. Within each residual block, the input and output features are element-wise added through skip connections, followed by a batch normalization layer and a ReLU activation function.

[0024] The output feature map sizes and number of channels for the four residual stages are as follows: the first coded feature map F1 is 1 / 4 the size of the input image and has 256 channels; the second coded feature map F2 is 1 / 8 the size of the input image and has 512 channels; the third coded feature map F3 is 1 / 16 the size of the input image and has 1024 channels; and the fourth coded feature map F4 is 1 / 32 the size of the input image and has 2048 channels. These multi-scale feature maps are subsequently input into the three-branch decoder for prediction of different tasks.

[0025] The ResNeXt-50 backbone network loads ImageNet pre-trained weights, which include learnable parameters from the initial convolutional layers and four residual stages. Transfer learning is used to accelerate network convergence and improve the generalization ability of this model. After loading the pre-trained weights, the network parameters are fine-tuned using stochastic gradient descent with a low learning rate to avoid compromising the excellent initialization of the pre-trained parameters.

[0026] Step 3: Calculation of channel feature weights.

[0027] like Figure 3 As shown, the channel attention submodule calculates channel feature weights along the channel dimension to enhance important channel features. The structure of the channel attention submodule includes: a global average pooling layer, a fully connected layer, a ReLU activation function, a fully connected layer, and a Sigmoid activation function.

[0028] Specifically, the channel attention submodule receives the feature map output by the encoder. Where C is the number of channels, H is the height, and W is the width. First, the spatial dimensions of the feature map are aggregated using a global average pooling layer to obtain a C-dimensional channel feature vector. The calculation formula is as follows: Where c represents the channel index, and i and j represent the row and column indices of the spatial location, respectively.

[0029] Next, the C-dimensional channel feature vectors are processed through the first fully connected layer. Mapped to dimensional features, where This represents the dimensionality reduction ratio, set in this embodiment. The weight matrix of the first fully connected layer is: The bias vector is .

[0030] After the ReLU activation function, the feature dimension is restored to C dimensions through a second fully connected layer. The weight matrix of the second fully connected layer is as follows. The bias vector is Finally, the output is mapped to... using the Sigmoid activation function. Range, to obtain the channel attention weight vector .

[0031] The formula for calculating the channel attention weight vector can be expressed as: in, This represents the Sigmoid activation function. This represents the ReLU activation function.

[0032] The obtained channel attention weight vector Compared with the original feature map Perform channel-by-channel multiplication to obtain the channel-weighted feature map. The calculation formula is as follows: .

[0033] Step 4: Calculation of spatial feature weights.

[0034] The spatial attention submodule computes spatial feature weights along the spatial dimension to enhance important spatial location features. The structure of the spatial attention submodule includes: max pooling and average pooling along the channel dimension, concatenation operations, convolutional layers, and a sigmoid activation function.

[0035] Specifically, the spatial attention submodule receives the feature map output by the encoder. First, max pooling and average pooling are performed along the channel dimension to obtain two results. Spatial feature map and The formula for calculating max pooling is: The formula for calculating average pooling is: Where i and j represent the row and column indices of the spatial location, respectively.

[0036] Next, the two spatial feature maps are concatenated along the channel dimension to obtain a 2-channel concatenated feature map. Then, through Convolutional layers and the sigmoid activation function generate a spatial attention weight map. . The output channel of the convolutional layer is 1, and the padding is 3.

[0037] Spatial attention weight map Compared with the original feature map Perform element-wise multiplication to obtain the spatially weighted feature map. The calculation formula is as follows: Where i and j represent the row and column indices of the spatial location, respectively.

[0038] Step 5: Decoupling of feature maps.

[0039] Channel weighted feature map Spatial weighted feature map Element-wise addition yields the decoupled feature map. The formula for calculating the decoupling feature map is: in, This indicates element-wise multiplication.

[0040] Through the feature decoupling process, the feature separation module can weight channel features and spatial features separately, enabling the network to learn feature representations of different scales and dimensions more flexibly. This decoupling strategy allows the three-branch decoder to utilize differentiated feature representations according to the needs of each task, thereby improving the performance of multi-task learning.

[0041] After feature extraction and feature separation, the shared encoder outputs three encoded feature maps at different scales, which are then fed into the three-branch decoder. Specifically, the feature maps output by the shared encoder include: The third encoded feature map: derived from the third residual stage of the ResNeXt-50 network, with a feature map size of 1 / 16 of the input image and 1024 channels, denoted as... , where H and W are the height and width of the original image.

[0042] Fourth encoded feature map: from the fourth residual stage of the ResNeXt-50 network, the feature map size is 1 / 32 of the input image, and the number of channels is 2048, denoted as... .

[0043] The fifth encoded feature map: the fused features after processing by the feature separation module, which fuses features of different scales through skip connections. The feature map size is 1 / 16 of the input image, and the number of channels is 1024. .

[0044] Three encoded feature maps are input into the boundary decoder, road decoder, and connectivity decoder, respectively, to generate the boundary prediction map, road segmentation prediction map, and connectivity prediction map. The third encoded feature map... The main input is fed into the road decoder for extracting road semantic information; the fourth encoding feature map The main input is fed into the boundary decoder for boundary gradient feature extraction; the fifth part encodes the feature map. The main input is fed into the connectivity decoder for topology analysis and reasoning.

[0045] Through feature output design, the shared encoder can provide feature representations of appropriate scale and granularity for different decoding tasks, thereby improving the efficiency and accuracy of multi-task learning.

[0046] Step 6: Generation of the boundary prediction map.

[0047] like Figure 4As shown, the boundary decoder extracts gradient information of road edges from encoded features to generate a boundary prediction map. The boundary decoder employs a Sobel convolution enhancement module, which enhances the gradient features of the road boundary region by applying Sobel convolution kernels in both horizontal and vertical directions.

[0048] The Sobel convolution enhancement module includes two parallel Sobel convolution branches, which extract boundary gradient features in the horizontal and vertical directions, respectively. The Sobel convolution kernels are 3×3 in size, and the kernel parameters are fixed as Sobel operators, and they are not used in network training.

[0049] Specifically, the horizontal Sobel convolution kernel is defined as follows: The vertical Sobel convolution kernel is defined as follows: in, This represents the horizontal Sobel convolution kernel. This represents a vertical Sobel convolution kernel. The kernel value represents the weight of pixels at different positions, with the four positions above, below, left, and right of the center pixel having greater weights, used to detect gradient changes.

[0050] The Sobel convolutional enhancement module works as follows: First, it receives the fourth encoded feature map from the output of the shared encoder. Then, apply horizontal Sobel convolution kernels respectively. and vertical Sobel convolution kernel Perform a convolution operation on each channel of the feature map to obtain the horizontal gradient feature map. and vertical gradient feature map .

[0051] Horizontal gradient feature map The calculation formula is: Vertical gradient feature map The calculation formula is: in, and This represents the offset of the convolution kernel, with values ​​of -1, 0, and 1.

[0052] Obtain the horizontal gradient feature map and vertical gradient feature map Then, a feature fusion layer is used to fuse the gradient features from the two directions. The feature fusion layer employs a concatenation operation, concatenating the horizontal and vertical gradient feature maps along the channel dimension to obtain a fused gradient feature map. .

[0053] Then, through The convolutional layer performs channel-wise dimensionality reduction on the fused gradient feature map, reducing the number of channels from 4096 to 256, resulting in the dimensionality-reduced gradient feature map. . The convolutional layers are followed by batch normalization layers and ReLU activation functions to improve the linear expressiveness of features. The Sobel convolutional kernel can effectively detect edge gradients in images. Because it uses a convolutional kernel with fixed parameters, it does not need to participate in network training and can serve as prior knowledge to guide the network's learning.

[0054] Next, a bilinear upsampling layer is used to upsample the dimensionality-reduced gradient feature map to the original resolution. The upsampled gradient feature map is obtained. Bilinear upsampling uses a sampling factor of 32 and employs bilinear interpolation to estimate pixel values. Bilinear upsampling is a commonly used upsampling method that maintains spatial continuity and avoids jagged edges.

[0055] Next, through The convolutional layer maps the upsampled gradient feature map to a single-channel prediction map. The convolutional layer has 256 input channels and 1 output channel, used to compress 256-dimensional features into a single-channel probability value. The weight matrix of the convolutional layer is , bias is .

[0056] Then, the predicted values ​​are mapped to using the Sigmoid activation function. Range. The definition of the Sigmoid activation function is: in, Indicates the input value. This indicates the output value.

[0057] Finally, the boundary prediction map is obtained. The boundary prediction map represents the probability that each pixel belongs to the road boundary. During the inference phase, a threshold is set. In this embodiment, it is set to 0.5, which determines pixels with a probability value greater than the threshold as road boundaries, and others as non-boundaries.

[0058] By designing a boundary decoder, this model can fully utilize the edge gradient detection capability of the Sobel operator and combine it with the feature extraction capability of deep learning to generate a high-precision road boundary prediction map, providing a reliable foundation for subsequent boundary prior fusion.

[0059] Step 7: Generation of road segmentation prediction map.

[0060] like Figure 5 As shown, the road decoder extracts semantic information about roads from encoded features and generates a road segmentation prediction map. The road decoder employs a multi-scale dilated convolution module, using parallel dilated convolutions with different dilation rates to capture road features in different receptive fields, adapting to roads of varying widths.

[0061] The multi-scale dilated convolution module comprises four parallel dilated convolution branches, each using the same 3×3 convolution kernel but with different dilation rates of 1, 2, 4, and 8. This design enables the network to simultaneously capture road features at small, medium, and large scales, improving its adaptability to roads of varying widths.

[0062] Specifically, the structure of the four parallel dilated convolution branches is as follows: First branch: dilation rate d1=1, kernel size k=3×3, input channels 1024, output channels 256. This branch corresponds to a normal convolution operation, with a receptive field size of 3×3 pixels.

[0063] The second branch has a dilation rate of d²=2, a kernel size of k=3×3, 1024 input channels, and 256 output channels. This branch inserts a zero-value hole between the elements of the kernel, and has a receptive field size of 5×5 pixels.

[0064] The third branch has a dilation rate of d3=4, a kernel size of k=3×3, 1024 input channels, and 256 output channels. This branch inserts three zero-value holes between the elements of the kernel, and has a receptive field size of 9×9 pixels.

[0065] The fourth branch has a dilation rate of d4=8, a kernel size of k=3×3, 1024 input channels, and 256 output channels. This branch inserts 7 zero-value holes between the elements of the kernel, and the receptive field size is 17×17 pixels.

[0066] The formula for calculating dilated convolution is: in, denoted by , where i represents the pixel value at position i, j in the intermediate feature map, X represents the first encoded feature map F1, d represents the dilation rate, and M represents the 3×3 convolution kernel; In this embodiment, the input feature map of the multi-scale dilated convolution module is the third encoded feature map from the shared encoder. Each branch performs a dilated convolution operation on the input feature map, resulting in four intermediate feature maps. .

[0067] Each dilated convolutional branch is followed by a batch normalization layer and a ReLU activation function to normalize the feature distribution and introduce nonlinearity, thereby improving the network's expressive power.

[0068] The four intermediate feature maps are dilated and convolved, then fused through a feature fusion layer. The feature fusion layer uses a concatenation operation to stitch the four intermediate feature maps along the channel dimension to obtain the fused feature map. .

[0069] Then, through Convolutional layers perform feature compression and fusion on the fused feature maps. The convolutional layer has 1024 input channels and 256 output channels, and is used to perform channel mixing and dimensionality reduction on the concatenated features. The convolutional layer is followed by a batch normalization layer and a ReLU activation function.

[0070] The formula for calculating a convolutional layer is: in, This represents the compressed feature map. express The weight matrix of the convolutional layer, This represents the bias vector.

[0071] The specific steps for generating the road segmentation prediction map are as follows: First, road features at different scales are extracted using a multi-scale dilated convolution module, resulting in four intermediate feature maps. In this step, dilated convolutions with different dilation rates can capture features of different receptive fields, improving the network's adaptability to roads of different widths.

[0072] Then, the four intermediate feature maps are concatenated through a feature fusion layer to obtain a fused feature map. In this step, the stitching operation preserves the integrity of features at different scales, avoiding feature loss caused by premature compression.

[0073] Next, through Convolutional layers compress and fuse features from the fused feature maps to obtain compressed feature maps. . Convolutional layers enable information exchange between channels, improving the ability to discriminate features.

[0074] Then, the compressed feature map is upsampled by a factor of 16 using a bilinear upsampling layer to obtain the upsampled feature map. Bilinear upsampling can maintain the continuity of spatial location and avoid the sawtooth effect.

[0075] Next, through The convolutional layer maps the upsampled feature map to a single-channel prediction map. The convolutional layer has 256 input channels and 1 output channel, used to compress 256-dimensional features into a single-channel probability value. The weight matrix of the convolutional layer is , bias is .

[0076] Finally, the predicted values ​​are mapped to using the Sigmoid activation function. The range is used to obtain the road segmentation prediction map. The road segmentation prediction map represents the probability that each pixel belongs to a road; the higher the probability value, the more likely that the pixel is a road.

[0077] During the inference phase, a threshold is set. In this embodiment, the threshold is set to 0.5, which determines pixels with probability values ​​greater than the threshold as roads, and others as non-roads. Through thresholding, the probability map can be converted into a binary road mask.

[0078] By designing a road decoder, this model can capture road features from different receptive fields through multi-scale dilated convolutions, improving adaptability and segmentation accuracy for roads of varying widths. Simultaneously, through a feature fusion strategy, the network can fully utilize multi-scale features, enhancing the accuracy and robustness of road extraction.

[0079] Step 8: Generation of the connectivity prediction graph.

[0080] like Figure 6 As shown, the connectivity decoder is used to analyze the topology of the road network and generate a connectivity prediction graph. The connectivity decoder uses a graph attention network (GAT) to model and infer road topological relationships by constructing the graph structure, calculating graph attention weights, and predicting connectivity.

[0081] The construction of a graph structure consists of two steps: superpixel segmentation and graph structure generation.

[0082] First, the road segmentation prediction map is generated using the SLIC (Simple Linear Iterative Clustering) superpixel segmentation algorithm. Superpixel segmentation is performed. SLIC is a k-means clustering-based superpixel segmentation algorithm that can quickly generate regular superpixels.

[0083] Specifically, the SLIC superpixel segmentation algorithm works as follows: Step S1, generate the road segmentation prediction map. from Pixel grid conversion Using CIELAB color space vectors while preserving boundary coordinate information, a five-dimensional feature vector set is constructed. , where each feature vector , Represents CIELAB color space values. , Represents the pixel coordinates of the nth vector feature.

[0084] Step S2: Uniformly initialize on the image grid Cluster centers, This represents the number of superpixels, set in this embodiment. Each cluster center , and Indicates the grid position.

[0085] Step S3, for each cluster center, in its Search for neighboring pixels within the neighborhood. Represents the superpixel spacing, calculated , Represents the dimension of the feature vector. .

[0086] Step S4, for each cluster center Calculate the features of each vector in the neighborhood. Feature distance Feature distance includes color distance and spatial distance ; Color distance and spatial distance The definition is as follows: Total distance is defined as: in, Representing cluster centers CIELAB luminance component, Representing cluster centers CIELAB's a-channel component, Representing cluster centers CIELAB's b-channel component, and Representing cluster centers pixel coordinates, This represents the compactness parameter, which is set in this embodiment. It is used to control the relative importance of color distance and spatial distance.

[0087] Step S5: Assign each pixel to the nearest cluster center to form a superpixel segmentation result.

[0088] Step S6: Update the cluster center position to the mean of all pixels within the superpixel.

[0089] Step S7: Repeat steps S3 to S6 until the cluster center positions converge or the maximum number of iterations is reached. In this embodiment, the maximum number of iterations is set to 10.

[0090] Using the SLIC superpixel segmentation algorithm, we obtain There are 10 superpixels, each of which is a node in the graph structure. The number of superpixels... The number of superpixels can be adjusted according to the image resolution and road complexity. In this embodiment, the number of superpixels ranges from 2000 to 4000, and is preferably set to 3000 in this embodiment.

[0091] Next, construct the graph structure. ,in, Represents a set of nodes. Represents the set of edges. Set of nodes. Include Each superpixel node has a superpixel node. The feature is the average of the features of all pixels within that superpixel. (Edge set) It includes the adjacency relationship between nodes, that is, when two superpixels are adjacent, there is an edge between them.

[0092] Specifically, node features The calculation formula is: in, Denotes the set of pixels within the k-th superpixel. This represents the number of pixels within the k-th superpixel. Representing the same coordinate The feature vector corresponding to the fifth encoded feature map F5.

[0093] Graph attention networks are used for feature aggregation and reasoning on graph structures. By calculating attention weights between nodes, they achieve adaptive aggregation of neighborhood node features. A graph attention network consists of three graph attention layers, each employing eight attention heads.

[0094] Specifically, the structure of each layer of the graph attention network is as follows: First, a linear transformation layer projects the features of each node into different attention head subspaces, where the output dimension of each attention head is 32. Then, an attention computation layer calculates the attention weights between nodes. Finally, a feature aggregation layer performs a weighted summation of the features of neighboring nodes based on the attention weights.

[0095] The definition of graph attention computation is as follows: For the graph ,node The characteristics are ,in Indicates the feature dimension. In the... In the layered attention layer, the features are first projected through a linear transformation layer: in, Let represent the original attention score of node n to its neighbor node m in the z-th attention head of layer l. This represents the linear transformation matrix of the z-th attention head in the l-th layer. The transpose of the learning vector representing the attention weights. This indicates a splicing operation. This represents the activation function, with a negative slope set to 0.2. This represents the input feature vector of node n in the l-th layer.

[0096] Then, the original attention score is normalized using the Softmax function to obtain the normalized attention score. : Where N(n) represents the set of neighboring nodes of node n. This represents the original attention score of node n for a node g within its neighborhood.

[0097] Next, the features of the neighboring nodes are weighted and summed based on the attention scores to obtain the aggregated node features: in, Indicates the first The aggregated features of node n in the z-th attention head of layer; For multiple attention heads, the outputs of each attention head are concatenated or averaged to obtain the final node features: In this embodiment, the following settings are provided: .

[0098] In this embodiment, the graph attention network comprises three graph attention layers, with the feature dimensions of each layer set as follows: the first layer has an input feature dimension of 1024 and an output feature dimension of 256; the second layer has an input feature dimension of 256 and an output feature dimension of 128; and the third layer has an input feature dimension of 128 and an output feature dimension of 64. Each layer uses eight attention heads, and the output dimensions of each attention head are 32, 16, and 8, respectively.

[0099] After passing through a 3-layer graph attention network, the final features of each node are obtained. ,in This represents the number of network layers.

[0100] By mapping nodes to pixels, the features of each node are mapped back to the original pixel space. Specifically, for each pixel location... Find the superpixel to which the pixel belongs. and the features of the superpixel node Assigning the value to this pixel position yields the connectivity feature map. .

[0101] Next, through Convolutional layers map connectivity feature maps to single-channel prediction maps. The convolutional layer has 64 input channels and 1 output channel, which is used to compress 64-dimensional features into a single-channel probability value. The weight matrix of the convolutional layer is , bias is .

[0102] Then, the predicted values ​​are mapped to using the Sigmoid activation function. The range is used to obtain the connectivity prediction graph. A connectivity prediction map represents the probability that each pixel is connected to its surrounding pixels; the higher the probability value, the more likely that the pixel is connected in the road network.

[0103] During the inference phase, connectivity prediction graph This is used to guide road break repair. Specifically, when a road break point is detected, the confidence level in the connectivity prediction graph is used to determine whether there are potential connections between the break points. If the connectivity confidence level is higher than a preset threshold... In this embodiment, it is set to 0.75, which establishes a connection between the fracture points and fills the fracture area.

[0104] By designing a connectivity decoder, this model can model the topological relationships of roads through graph structures, understand the connection patterns of the road network, and generate a connectivity prediction graph for fault repair. This graph-based approach can effectively model the topological structure of roads, improving the connectivity and integrity of road extraction.

[0105] Step 9: Fusion processing of the three prediction maps.

[0106] The boundary prior fusion module is used to fuse the boundary prediction map, road segmentation prediction map, and connectivity prediction map. It achieves multi-task collaborative optimization through boundary consistency loss, adaptive fusion weights, and topological constraint loss. The core idea of ​​the boundary prior fusion module is to use the boundary prediction map as prior knowledge to guide feature enhancement of the road segmentation prediction map in the boundary region, while using the connectivity prediction map as topological constraints to ensure that the road segmentation prediction map maintains a good topological structure.

[0107] Boundary consistency loss is used to constrain the consistency between the boundary prediction map and the road segmentation prediction map, ensuring the accuracy of boundary detection and the boundary precision of road segmentation. The mathematical expression for boundary consistency loss is as follows: in, Indicates position The local neighborhood, in this embodiment, is adopted Neighborhood, This represents the coordinates of other pixels within the neighborhood of the center pixel.

[0108] Specifically, the working principle of boundary consistency loss is as follows: First, calculate the road segmentation prediction map. gradient The gradient reflects the variation of the road segmentation prediction map between pixels. The gradient value is larger in the boundary region and smaller in the interior of the road or background region.

[0109] Road segmentation prediction map The gradient calculation uses the Sobel operator, including the horizontal gradient. and vertical gradient : The formula for calculating the gradient magnitude is: Then, in position of Neighborhood Inside, calculate the weighted sum of gradient magnitudes. The expression for the weighted sum is: In this formula, As weights, pixels with high road probability contribute more to the local gradient. This design ensures that the boundary consistency loss focuses primarily on the boundary features of the road region, rather than the noise gradient of the background region.

[0110] Next, the boundary prediction map Weighted sum of local gradients Compare the differences and calculate the absolute value difference. The mathematical expression for the absolute value difference is: If the boundary prediction map A larger value indicates a higher confidence level that the location is a road boundary; simultaneously, if the local gradient weighted sum... The value is also relatively large, indicating that the gradient magnitude of the road segmentation prediction map at that location is large, and the consistency between the two is good, with little difference. The difference is relatively small. Conversely, if the difference between the boundary prediction map and the local gradient weighted sum is large, it indicates that the boundary prediction and road segmentation prediction are inconsistent, and gradient descent is needed to reduce the difference.

[0111] Finally, the absolute differences at all locations are summed to obtain the boundary consistency loss. During training, by minimizing This improves the consistency between the boundary prediction map and the road segmentation prediction map, ensuring the accuracy of the boundary position of the road segmentation prediction map.

[0112] The parameters for boundary consistency loss are set as follows: neighborhood size is set to... That is, including location The gradient is calculated using the Sobel operator, and the gradient weights are... The probability values ​​of the road segmentation prediction map are used directly.

[0113] By designing a boundary consistency loss, this model can use the boundary prediction map as an explicit prior constraint to guide the learning of the road segmentation prediction map in the boundary region, thereby improving the boundary localization accuracy, reducing boundary ambiguity, and enhancing the distinction between roads and the background.

[0114] The adaptive fusion weights are calculated using a multi-layer perceptron (MLP). The fusion weights of boundary features and road features are dynamically adjusted based on the confidence level of the boundary prediction map. Boundary features are given higher weights in boundary regions, while road features are given higher weights in non-boundary regions.

[0115] A multilayer perceptron (MLP) consists of two fully connected layers that map the features of the boundary prediction map to adaptive fusion weights. The specific structure of the MLP is as follows: The first fully connected layer maps the 256-dimensional features to 128-dimensional features, followed by a ReLU activation function. The weight matrix of the first fully connected layer is as follows: The bias vector is .

[0116] The second fully connected layer maps the 128-dimensional features to 1-dimensional features, followed by a Sigmoid activation function. The weight matrix of the second fully connected layer is as follows: The bias vector is .

[0117] The mathematical expression for the Sigmoid activation function is: in, Indicates the input value. Indicates the output value, and the output range is within between.

[0118] Specifically, the input to the MLP is the fused gradient feature map output by the boundary decoder. In each spatial location The feature vector is MLP maps 256-dimensional feature vectors to scalar weight values. .

[0119] The calculation process of MLP is as follows: First, the 256-dimensional feature vector is mapped to a 128-dimensional feature vector through the first fully connected layer: in, This represents the output of the first hidden layer. This represents the weight matrix of the first fully connected layer. This represents the bias vector of the first fully connected layer.

[0120] Then, the 128-dimensional feature vector is mapped to a 1-dimensional scalar through a second fully connected layer: in, This represents the output of the second fully connected layer, before it has passed through an activation function. This represents the weight matrix of the second fully connected layer. This represents the bias vector of the second fully connected layer.

[0121] Next, the scalar is mapped to using the Sigmoid activation function. scope: in, Indicates position Adaptive fusion weights.

[0122] The adaptive fusion weight map is obtained through the above calculation process. Then, the fused weight map is upsampled to the original resolution using bilinear upsampling to obtain the upsampled fused weight map. .

[0123] The formula for calculating the fusion weights is summarized as follows: MLP stands for Multilayer Perceptron Mapping Function, which consists of two fully connected layers and corresponding activation functions.

[0124] The MLP weight matrix is ​​defined as follows: in, Indicates the first The weight vector of each neuron.

[0125] in, Indicates the first Bias of each neuron.

[0126] in, This represents the weight vector of the output neuron.

[0127] in, This indicates the weight bias of the output neuron.

[0128] During training, the weight matrix of the MLP , and bias vector , The gradient descent algorithm is used to update the MLP, enabling it to learn a suitable fusion weight strategy.

[0129] Obtain adaptive fusion weights Then, feature fusion is performed. The fusion formula is: in, This represents the fused feature map. Represents the boundary decoder features. Indicates the characteristics of the road decoder. This represents element-wise multiplication. Indicates adaptive fusion weights, and These are complementary weights.

[0130] Specifically, the adaptive fusion process works as follows: For spatial location If the confidence level of the boundary prediction is high, that is If it is larger, then: This means that in the boundary region, the fused features are mainly dominated by the boundary decoder features, making full use of the boundary gradient features extracted by the boundary decoder to improve the boundary localization accuracy.

[0131] For spatial location If the confidence level of the boundary prediction is low, i.e. If it is smaller, then: This means that in non-boundary areas, the fused features are mainly dominated by the road decoder features, making full use of the semantic features extracted by the road decoder to improve road segmentation accuracy.

[0132] Through this adaptive fusion strategy, the network can dynamically adjust the feature fusion method according to different regional characteristics. It can concentrate computing resources on enhancing boundary features in boundary regions and concentrate computing resources on extracting semantic features in non-boundary regions, thereby improving the overall road extraction performance.

[0133] Topology constraint loss is used to constrain the consistency between the connectivity prediction graph and the road segmentation prediction graph, ensuring that the connectivity prediction graph can correctly reflect the topology of the road segmentation prediction graph, avoiding misconnections and missing connections, and enhancing the integrity of the road network.

[0134] The mathematical expression for topological constraint loss is as follows: in, Indicates position The neighborhood location is used in this embodiment, which is an 8-neighborhood, that is, 8 neighborhood locations including the four directions of up, down, left, right and the four directions of diagonal. Represents a connectivity function used to calculate position. and location The connectivity between them.

[0135] Specifically, the connectivity function Connect is defined as follows: The principle of connectivity functions is: if the road prediction probabilities of two locations are both high, then the connectivity between these two locations is high; conversely, if the road prediction probability of at least one of the two locations is low, then the connectivity between these two locations is low.

[0136] To reduce computational complexity, this embodiment uses a simplified connectivity function, calculating connectivity only in the horizontal and vertical directions: in, For position The 4-neighborhood refers to the neighboring positions in the four directions: up, down, left, and right.

[0137] The working principle of topology constraint loss is as follows: For each location If connectivity prediction A higher value, close to 1, indicates higher expected connectivity between the road segmentation prediction map at that location and its neighboring locations. It is also relatively high, close to 1. At this point, the first term... This is the main function; minimizing this term increases the connectivity function value, thus improving the consistency between connectivity prediction and road segmentation prediction.

[0138] Conversely, if connectivity prediction A lower value, close to 0, indicates the expected connectivity of the road segmentation prediction map between that location and its neighboring locations. It is also low, close to 0. At this point, the second term... This is the main function; minimizing this term reduces the connectivity function value, thus improving the consistency between connectivity prediction and road segmentation prediction.

[0139] The parameters for the topological constraint loss are set as follows: the neighborhood type is 4-neighborhood, and the connectivity function is calculated using the mean.

[0140] By designing a topology constraint loss, this model can use the connectivity prediction graph as a topology constraint to constrain the topology of the road segmentation prediction graph, ensuring that the road segmentation prediction graph has good connectivity, improving the consistency between connectivity prediction and road segmentation prediction, improving the accuracy of topology inference, avoiding false connections and missing connections, and enhancing the integrity of the road network.

[0141] Step 10: Joint optimization training of the model.

[0142] The multi-task network with boundary prior constraints employs a joint optimization training strategy. It simultaneously optimizes the road segmentation, boundary detection, and connectivity prediction tasks through a joint loss function, achieving multi-task collaborative learning. Joint optimization training includes defining the joint loss function, setting hyperparameters, and implementing a training strategy. By appropriately allocating loss weights and adjusting training parameters, the model is ensured to converge to the optimal solution.

[0143] The joint loss function consists of three parts, including road segmentation loss. Boundary prior loss and topological constraint loss Joint loss function The definition is as follows: in, This represents the boundary loss weighting coefficient, which is set to 0.5 in this embodiment; This represents the topology loss weighting coefficient, which is set to 0.3 in this embodiment.

[0144] Road segmentation loss A weighted sum of the Binary Cross Entropy Loss (BCELoss) and the Dice loss is used. The road segmentation loss is defined as follows: in, This represents the Dice loss weighting coefficient, which is set to 0.5 in this embodiment.

[0145] Binary cross-entropy loss The definition is as follows: in, Indicates the total number of pixels. This indicates the actual label.

[0146] Specifically, the binary cross-entropy loss works as follows: for each pixel, if the true label... If the pixel represents a road, then the expected predicted value is... Approaching 1, loss item This will cause the predicted value to increase; if the true label If the pixel indicates that it is not a road, then the expected predicted value is... Close to 0, loss item This will cause the predicted value to decrease.

[0147] Dice loss The definition is as follows: in, The smoothing term is represented in this embodiment as... This is used to avoid the denominator being zero.

[0148] Specifically, Dice loss measures the degree of overlap between the ground truth labels and the predicted map. The higher the overlap, the larger the Dice coefficient. Since 1-Dice is used as the loss, the higher the overlap, the smaller the loss. Dice loss is sensitive to sample imbalance and can effectively handle the sample imbalance problem between road areas and non-road areas.

[0149] The hyperparameter settings for network training are as follows: The loss weighting coefficient is used to balance the contributions of different loss terms, ensuring that each loss is optimized within a reasonable range. In this embodiment, it is set as follows: = 0.5: Weight coefficient of boundary prior loss, adjustable range is 0.2-0.8.

[0150] = 0.3: Weighting coefficient for topological constraint loss, adjustable from 0.1 to 0.5.

[0151] = 0.5: The weighting coefficient of the Dice loss.

[0152] The selection of loss weight coefficients is based on empirical adjustment, and the optimal values ​​are determined by comparing performance on the validation set. If the boundary loss weights... If the boundary loss weights are too large, the model will overemphasize boundary prediction, potentially leading to a decline in the overall performance of road segmentation; if the boundary loss weights are too large... If the value is too small, the constraint effect of the boundary prediction map on road segmentation will be weakened, which may lead to blurred boundaries.

[0153] This model is trained using the SGD (Stochastic Gradient Descent) optimizer. The optimizer's parameters are configured as follows: Initial learning rate: lr initial = 0.01.

[0154] Momentum coefficient: momentum = 0.9.

[0155] Weight decay: weight decay = 1e-5.

[0156] Learning rate decay strategy: StepLR, multiply the learning rate by 0.1 at epoch 60 and epoch 80.

[0157] Specifically, the parameter update formula for the SGD optimizer is: in, Indicates the first The velocity vector of the step. Indicates the first The parameter vector of the step, This represents the gradient of the joint loss function with respect to the parameters. Indicates the first The learning rate of each step.

[0158] The learning rate decay strategy is defined as follows: in, This represents the attenuation factor, which is set in this embodiment. ; Indicates the decay period, which is set in this embodiment. The next iteration.

[0159] Training parameters include the number of training epochs, batch size, and data augmentation settings. In this embodiment, the following settings are configured: Training epochs: 100.

[0160] Batch size: batch size = 8.

[0161] Cumulative gradient steps: gradient steps = 2, equivalent batch size is 16.

[0162] Data augmentation: random cropping, horizontal flipping, vertical flipping, color jittering, and Gaussian noise addition.

[0163] The random cropping size is 512×512, used to crop the 1024×1024 input image to a suitable training size. Horizontal and vertical flips are performed with a probability of 0.5 each, to increase data diversity. Color jitter includes brightness, contrast, and saturation adjustments, ranging from 0.8 to 1.2 times the original values. Gaussian noise is added with a mean of 0 and a standard deviation of 0.01 to enhance the model's robustness.

[0164] The online training adopts a phased training strategy, including a pre-training phase, a warm-up phase, a main training phase, and a fine-tuning phase.

[0165] During the pre-training phase, ResNeXt-50 weights pre-trained on ImageNet are loaded to initialize the parameters of the shared encoder. These ImageNet pre-trained weights are trained on the ImageNet dataset, which contains approximately 1.2 million images across 1000 classes. Loading these pre-trained weights accelerates network convergence and improves the model's generalization ability.

[0166] Specifically, the loading process for pre-trained weights is as follows: Load the ImageNet pre-trained ResNeXt-50 weight file.

[0167] The pre-trained weights are mapped to the network parameter dictionary. For parameters with matching names, the pre-trained weights are loaded directly; for parameters with mismatched names, they are randomly initialized.

[0168] Freeze the pre-trained parameters and do not update these parameters in the first 10 epochs; only train the newly added decoder module.

[0169] After 10 epochs, all parameters are unfrozen, and the encoder and decoder are trained simultaneously.

[0170] This hierarchical training strategy avoids disrupting the excellent initialization of pre-trained parameters while ensuring that newly added modules can learn from pre-trained features.

[0171] The warm-up phase uses a small learning rate for the first 5 epochs to stabilize the training process. The learning rate settings for the warm-up phase are as follows: in, This indicates the warm-up period, which is set to 5 epochs in this embodiment. During the warm-up phase, the learning rate increases linearly from 0 to an initial learning rate of 0.01.

[0172] The purpose of the warm-up phase is to avoid excessively large gradient updates at the start of training, which could lead to training instability or loss divergence. By gradually increasing the learning rate, this model can smoothly transition from the initial state to the main training phase.

[0173] The main training phase is the training process from the 6th epoch to the 80th epoch. During this phase, a fixed learning rate of 0.01 is used, and the joint loss function is minimized through the SGD optimizer.

[0174] During the main training phase, the network parameter update formula is: in, .

[0175] The main training phase is the primary stage of network learning. Through iterative learning with a large amount of training data, the model parameters gradually converge to the optimal solution.

[0176] The learning rate decay phase consists of two phases, occurring at the 60th and 80th epochs. The learning rate decay formula is as follows: in, This indicates the decay count; it is 1 after the 60th epoch and 2 after the 80th epoch.

[0177] Specifically, the learning rate is set as follows: epochs 1-60: lr = 0.01, epochs 61-80: lr = 0.001, epochs 81-100: lr = 0.0001.

[0178] By decaying the learning rate, the network can perform fine-tuning of parameters in the later stages of training, thereby improving the performance of the final model.

[0179] The fine-tuning phase, spanning epochs 90-100, involves finely adjusting the model parameters using an extremely small learning rate of 0.0001. During this phase, the loss weights are dynamically adjusted based on performance on the validation set. and .

[0180] Specifically, the dynamic adjustment strategy is as follows: In the 90th epoch, the boundary consistency metric and topology consistency metric on the validation set are calculated: in, This represents the neighborhood connectivity of the road segmentation prediction graph. This represents the average of the absolute values ​​of all pixels.

[0181] If the boundary consistency index is higher than the preset threshold, which is set to 0.3 in this embodiment, then the threshold is increased. Up to 0.6; if the topology consistency index is higher than the preset threshold, which is set to 0.4 in this embodiment, then increase. Up to 0.4.

[0182] Through this dynamic adjustment strategy, the model can adaptively balance the optimization of different tasks during the fine-tuning phase, thereby improving the final performance.

[0183] Early stopping is a strategy used to prevent model overfitting. Specifically, if the overall loss on the validation set exceeds a certain threshold over 10 consecutive epochs, early stopping is employed. If the performance no longer declines, training is terminated early, and the model that performs best on the validation set is selected as the final model.

[0184] Set a patience counter The initial value is 0. After each training round, the overall loss of the validation set is calculated: if the current validation loss is higher than the historical best validation loss, the patience counter is incremented by 1; if the current validation loss is lower than or equal to the historical best validation loss, the patience counter is reset to 0. When the patience counter accumulates to 10, the early stopping strategy is triggered, and training is terminated.

[0185] By employing an early stopping strategy, the model can avoid overfitting and improve its generalization ability.

[0186] Step 11: Connectivity restoration.

[0187] The connectivity repair module is used for post-processing of the road segmentation prediction map to repair road breaks caused by shadow occlusion, noise interference, and other reasons, thereby improving the connectivity of the road network. Connectivity repair includes four steps: road break point detection, potential connection finding, connection confidence assessment, and break repair.

[0188] Road breakpoint detection is used to identify break locations, or road endpoints, in the road segmentation prediction map. A road endpoint is a boundary pixel of a road region that has only one or two adjacent pixels belonging to the road.

[0189] Specifically, the process of detecting road fracture points is as follows: First, the road segmentation prediction map Binarization is performed to obtain the binarized road mask. Binarization is performed using a thresholding method, by setting a threshold value. Pixels with a probability value greater than the threshold are identified as roads and marked as 1; otherwise, they are identified as non-roads and marked as 0.

[0190] The formula for binarization is: Then, the binarized road mask Connectivity analysis is performed to identify connected road regions. This analysis uses either Depth-First Search (DFS) or Union-Find algorithms to label adjacent road pixels as belonging to the same connected region.

[0191] Next, for each connected component, its set of boundary pixels is calculated. A boundary pixel is a pixel in the road region that has at least one neighboring pixel belonging to a non-road region. The set of boundary pixels is defined as follows: in, Indicates position 8 neighborhoods.

[0192] Then, for each boundary pixel, the number of its road neighbor pixels is calculated. The number of road neighbor pixels refers to the number of pixels belonging to the road in the 8-neighborhood of that boundary pixel.

[0193] The formula for calculating the number of pixels in the road's neighborhood is: If the number of pixels in the road neighborhood If so, the boundary pixel is marked as a road endpoint. A road endpoint indicates the end position of a road, which may be due to a road break or the end of a road.

[0194] The above steps yield the set of road endpoints. ,in This indicates the number of road endpoints.

[0195] Potential connectivity is used to find possible connections between road endpoints. For each road endpoint, other road endpoints in its neighborhood are searched to determine if potential connecting paths exist.

[0196] Specifically, the process of finding potential connections is as follows: First, for each road endpoint Define its search radius In this embodiment, the search radius is set. The number of pixels can be adjusted based on road density and image resolution.

[0197] Then, in the search radius Inside, search for all other road endpoints. , The Euclidean distance is less than the search radius.

[0198] The formula for calculating Euclidean distance is: in, , Endpoints coordinates , Indicates endpoint The coordinates.

[0199] Get the set of all possible endpoint pairs ,in, Indicates the number of possible endpoint pairs.

[0200] Next, for each endpoint pair According to the connectivity prediction graph Calculate connection reliability. The calculation formula is: in, Endpoints To the endpoint The set of pixels in a straight line between them. This indicates the number of pixels in the straight line.

[0201] The calculation of Line(p,q) uses the Bresenham line algorithm, which can quickly generate discrete pixel points between two endpoints.

[0202] The Bresenham line algorithm efficiently generates a set of line pixels between endpoint pairs.

[0203] Then, for each endpoint pair Calculate road orientation consistency. Road orientation consistency refers to the consistency of endpoints. and endpoints Whether the local road directions are consistent. The local road directions are obtained by calculating the main direction vector of the road region within the neighborhood of the endpoints.

[0204] Specifically, for endpoints In its Perform the following calculations within the neighborhood: Find the set of all road pixels in the neighborhood. : in, express Neighborhood.

[0205] if If empty, skip that endpoint.

[0206] Calculate the covariance matrix of road pixels : Calculate the eigenvalues ​​of the covariance matrix and The eigenvector corresponding to the largest eigenvalue is the local road direction vector.

[0207] The endpoints are obtained through the above calculations. Local road direction vector and endpoints Local road direction vector .

[0208] The formula for calculating road direction consistency is: in, Represents the vector dot product. and Represents the vector magnitude.

[0209] The range of values ​​for road direction consistency is: The larger the value, the more consistent the directions.

[0210] The connection confidence assessment is used to comprehensively judge the connection confidence between endpoint pairs based on connectivity confidence and road direction consistency, and to decide whether to establish a connection.

[0211] The comprehensive formula for calculating connection trustworthiness is: in, and This represents the weighting coefficient, which is set in this embodiment. , This indicates that connectivity confidence is more important than directional consistency.

[0212] For each endpoint pair If the connection is trustworthy Greater than the threshold In this embodiment, the following settings are provided. If a connection can be established between the endpoints, then the endpoints are added to the set of trusted connection pairs. .

[0213] The set of trusted connections is defined as follows: To avoid duplicate connections, for a set of trusted connection pairs Each connection pair in If a join already exists in the set If so, then only one of them will be kept.

[0214] Fracture repair is used to establish connections between the endpoints of a trusted pair of connections, filling in fractured areas.

[0215] Specifically, the fracture repair process is as follows: First, for the set of trusted connection pairs Each connection pair in The connection path is generated using the Bresenham line algorithm, resulting in a set of line pixels. .

[0216] Then, all pixels on the path are marked as road pixels. Specifically, for each pixel location... Set a binary road mask And set the probability prediction value. .

[0217] Repaired road mask and prediction chart The calculation formula is: in, Represents a logical OR operation. This indicates an indicator function, which is 1 if the condition is true and 0 otherwise.

[0218] Next, morphological processing is performed to smooth the connected regions. Morphological processing includes closing and opening operations.

[0219] The definition of the closing operation is: Wherein, Erosion represents the erosion operation, Dilation represents the dilation operation, and SE represents the structuring element. In this embodiment, a circular kernel with a radius of 2 pixels is used.

[0220] The definition of opening operation is: First, a closing operation is performed to fill the small holes in the connected regions; then an opening operation is performed to remove isolated noise points in the connected regions.

[0221] Through the above steps, the final road extraction result is obtained. .

[0222] The technical effects of the connectivity repair module include: First, accurately locating road breaks through road breakpoint detection; second, finding possible connection paths between breakpoints through potential connection search; third, avoiding erroneous connections and improving connection accuracy through connection reliability judgment; and fourth, restoring the connectivity of the road network and improving the integrity of road extraction through break repair and morphological processing.

[0223] The above are merely preferred embodiments of the invention and are not intended to limit the invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A multi-task road extraction method with boundary prior constraints, characterized in that, Includes the following steps: S1: Acquire the original remote sensing image, preprocess the original remote sensing image, and output the preprocessed remote sensing image. S2: Perform multi-scale feature extraction on the preprocessed remote sensing image and output a multi-scale coded feature map; S3: Calculate the channel feature weights and spatial feature weights of the multi-scale encoded feature map and decouple and weight them, outputting the decoupled encoded feature map; perform feature enhancement and multi-scale modeling on the multi-scale encoded feature map, outputting the boundary prediction map and the road segmentation prediction map; S4: Perform superpixel segmentation on the road segmentation prediction map, and then perform mean pooling based on the decoupled encoded feature map to output the feature vectors corresponding to the superpixel map nodes. S5: Perform topological relationship reasoning and attention weight calculation on the feature vectors corresponding to the superpixel graph nodes, and output a connectivity prediction graph; S6: Perform weighted fusion processing on the boundary prediction map, the road segmentation prediction map, and the connectivity prediction map, and output the fused road feature map; S7: Perform connectivity repair on the fused road feature map and output the road extraction results.

2. The multi-task road extraction method with boundary prior constraints according to claim 1, characterized in that, The specific steps for extracting multi-scale features from the preprocessed remote sensing image and outputting a multi-scale coded feature map are as follows: The preprocessed remote sensing image is input into the backbone network, undergoes initial convolution, batch normalization and ReLU activation, and then downsampled by max pooling to output the initial downsampled feature map; The initial downsampled feature map is then subjected to feature learning through four residual stages, and the number of channels is gradually increased by downsampling to output a multi-scale coding map; the multi-scale coding map includes a first coding feature map F1, a second coding feature map F2, a third coding feature map F3 and a fourth coding feature map F4.

3. The multi-task road extraction method with boundary prior constraints according to claim 2, characterized in that, The specific steps for calculating the channel feature weights of the multi-scale encoded feature map are as follows: Let the multi-scale feature map be F: Where C represents the number of channels in the feature map, H represents the height of the feature map, and W represents the width of the feature map; The spatial dimensions of the multi-scale encoded feature maps are aggregated using a global average pooling layer, outputting a C-dimensional channel feature vector. : Where c represents the channel index, i represents the row index of the feature map, and j represents the column index of the feature map; The C-dimensional channel feature vector is processed through the first fully connected layer. The features are mapped to C / r dimensionality, where r represents the dimensionality reduction ratio. After nonlinear transformation and invalid feature filtering using the ReLU activation function, the C / r dimensional features are restored to C dimensionality through a second fully connected layer. Finally, the features are mapped to the range [0,1] using the Sigmoid activation function, and the channel attention weight vector is output. ; in, This represents the Sigmoid activation function. Represents the ReLU activation function. This represents the weight matrix of the first fully connected layer. This represents the bias vector of the first fully connected layer. This represents the weight matrix of the second fully connected layer. This represents the bias vector of the second fully connected layer; The channel attention weight vector Channel-wise multiplication is performed with the multi-scale feature map F to obtain the channel-weighted feature map. .

4. The multi-task road extraction method with boundary prior constraints according to claim 3, characterized in that, The specific steps for calculating the spatial feature weights of the multi-scale encoded feature map are as follows: Max pooling and average pooling are performed on the multi-scale feature map F along the channel dimension to output a spatial feature map with dimension H×W. and ; The maximum pooling formula is: The average pooling formula is: The spatial feature map and Then, the data is stitched together along the channel dimension to output a 2-channel stitched feature map. ; The splicing feature map Spatial attention weight map is generated using convolutional layers and a sigmoid activation function. ; The spatial attention weight map With the multi-scale feature map Perform element-wise multiplication to obtain the spatially weighted feature map. .

5. The multi-task road extraction method with boundary prior constraints according to claim 3, characterized in that, The specific steps for performing feature enhancement on the multi-scale encoded feature map and outputting the boundary prediction map are as follows: Through horizontal convolution kernels and vertical convolution kernel Perform a convolution operation on each channel of the second encoded feature map F2 to output a horizontal gradient feature map. and vertical gradient feature map ; The horizontal gradient feature map The calculation formula is: The vertical gradient feature map The calculation formula is: Where u and v represent the offset of the convolution kernel; The horizontal gradient feature map and the vertical gradient feature map The features are stitched together along the channel dimension to generate a fused gradient feature map. ; For the fused gradient feature map Perform channel dimensionality reduction and upsampling to generate an upsampled gradient feature map; The gradient feature map obtained above is mapped to a single-channel boundary prediction map. .

6. The multi-task road extraction method with boundary prior constraints according to claim 5, characterized in that, The specific steps for performing multi-scale modeling on the multi-scale encoded feature map and outputting the road segmentation prediction map are as follows: The first encoded feature map F1 is dilated by performing a dilated convolution operation to generate four intermediate feature maps. The formula for calculating the dilated convolution is: in, denoted by , where i represents the pixel value at position i, j in the intermediate feature map, X represents the first encoded feature map F1, d represents the dilation rate, and M represents the 3×3 convolution kernel; The four intermediate feature maps are concatenated along the channel dimension to generate a fused feature map. ; For the fused feature map Feature compression and upsampling are performed to generate upsampled feature maps. ; The feature map after the above adoption Mapped to a single-channel road segmentation prediction map .

7. The multi-task road extraction method with boundary prior constraints according to claim 6, characterized in that, The specific steps for performing superpixel segmentation on the road segmentation prediction map, and then performing mean pooling based on the decoupled encoded feature map to output the feature vectors corresponding to the superpixel map nodes are as follows: Based on the road segmentation prediction map The pixel grid formed by the spatial dimensions is converted into CIELAB color space vectors, while retaining the boundary coordinate information, to construct a vector feature set. The nth vector feature Represented as: in, , , This represents the CIELAB color space value of the nth vector feature. , Represents the pixel coordinates of the nth vector feature; K cluster centers are uniformly initialized on the pixel grid, where K represents the number of superpixels; For each cluster center Search for neighboring pixels within the neighborhood and calculate the superpixel spacing S: Where R represents the dimension of the feature vector; For each cluster center Calculate the features of each vector in the neighborhood. Feature distance The feature distance includes color distance. and spatial distance ; The color distance and the spatial distance The definition is as follows: Total distance is defined as: in, Representing cluster centers CIELAB luminance component, Representing cluster centers CIELAB's a-channel component, Representing cluster centers CIELAB's b-channel component, and Representing cluster centers pixel coordinates, Indicates the compactness parameter; Each of the vector features Assign the data to the nearest cluster center to generate superpixel segmentation results; Update the cluster centers by replacing them with the average value of all pixels within the superpixel region; The process continues until the cluster centers converge or the maximum number of iterations is reached. Skip connections are made to the decoupled coded feature map to output the fifth coded feature map F5; Construct the graph structure G: Where V represents the set of nodes and E represents the set of edges; The node set V includes K superpixel nodes, each superpixel graph node The corresponding feature vector is the average of the feature vectors of all pixels within the superpixel; The feature vector corresponding to the superpixel map node The calculation formula is: in, Denotes the set of pixels within the k-th superpixel. This represents the number of pixels within the k-th superpixel. Representing the same coordinate The feature vector corresponding to the fifth encoded feature map F5.

8. The multi-task road extraction method with boundary prior constraints according to claim 7, characterized in that, The specific steps for performing topological relationship reasoning and attention weight calculation on the feature vectors corresponding to the superpixel graph nodes to output the connectivity prediction graph are as follows: The feature vectors corresponding to the superpixel nodes are input into the graph attention network, and feature projection is first performed through a linear transformation: in, Let represent the original attention score of node n to its neighbor node m in the z-th attention head of layer l. This represents the linear transformation matrix of the z-th attention head in the l-th layer. The transpose of the learning vector representing the attention weights. This indicates a splicing operation. This represents the activation function. This represents the input feature vector of node n in the l-th layer; The original attention score is then normalized using the Softmax function to obtain the normalized attention score. : Where N(n) represents the set of neighboring nodes of node n. This represents the original attention score of node n for a given node g within its neighborhood. The features of neighboring nodes are weighted and summed based on the normalized attention scores to obtain the aggregated node features: in, Indicates the first The aggregated features of node n in the z-th attention head of layer; For multiple attention heads, the outputs of each attention head are concatenated or averaged to obtain the final node features: Where Z represents the number of attention heads; After passing through the graph attention network, the final features of the superpixel graph nodes are obtained. The final features of the superpixel graph nodes are mapped to a single-channel connectivity prediction graph. .

Images