A Method and System for Enhancing Road Connectivity in Remote Sensing Images Based on Conditional Masking

A method to enhance road connectivity in remote sensing images using conditional masking, and an optimization of the semantic segmentation network using channel fusion layer and regional continuity loss, solves the problem of road fragmentation in remote sensing images, achieving higher segmentation accuracy and clearer road boundaries.

CN119068184BActive Publication Date: 2026-11-10CHENGDU UNIVERSITY OF TECHNOLOGY
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410825874.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-25
Publication Date
2026-11-10
Estimated Expiration
2044-06-25

AI Technical Summary

Technical Problem

Extracting roads from remote sensing images faces challenges such as complex backgrounds, lighting conditions, small road spacing, and network irregularities leading to road fragmentation, which affect the recognition and segmentation results.

Method used

A method for enhancing road connectivity in remote sensing images using conditional masking is proposed. This method involves designing a channel fusion layer and a regional continuity loss, combined with a focus loss, to optimize the semantic segmentation network and enhance road connectivity.

Benefits of technology

It improves the accuracy of semantic segmentation networks, mitigates road disconnection issues, provides clearer road boundaries and spatial consistency, and is suitable for convolutional semantic segmentation networks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068184B_ABST
    Figure CN119068184B_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for enhancing road connectivity in remote sensing images based on conditional masking, using the remote sensing road prediction images from the original semantic segmentation network as additional supervision. Through a precise road backbone extraction module and a conditional masking strategy for label images, semantic information with deep road attributes is provided to the network. This invention designs a channel fusion layer to efficiently fuse the semantic information of the original input image and the predicted label image, adaptively fitting the feature extraction method of the original network. Furthermore, a region continuity loss function is proposed. This loss function assigns higher weights to discontinuous parts, prompting the model to focus on pixels with road disconnections in the prediction, thereby optimizing road connectivity. This invention can enhance the performance of the network in remote sensing image road semantic segmentation tasks with only a small increase in parameters and computational cost.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for enhancing road connectivity in remote sensing images based on conditional masking. Background Technology

[0002] Semantic segmentation of remote sensing images is the process of pixel-level classification of high-resolution images acquired by remote sensing technology, aiming to understand the types and locations of ground features in the image. In this task, whether in urban or rural scenes, the extraction of road networks is indispensable; therefore, accurate road segmentation is crucial. Road segmentation helps traffic management departments monitor and analyze road network conditions, identify traffic congestion points, optimize traffic signal control, and improve road utilization efficiency. Furthermore, by monitoring road expansion and changes in remote sensing images, the impact of human activities on the natural environment can be assessed, supporting the formulation of ecological protection and sustainable development strategies. However, the extraction of roads from high-resolution remote sensing images often faces the following four challenges.

[0003] First, roads are usually embedded in extremely complex backgrounds, including various object elements such as buildings, trees, vehicles, and pedestrians. The presence of these elements greatly increases the difficulty of the segmentation task.

[0004] Second, the lighting conditions during satellite image acquisition have a significant impact on the visualization of roads in remote sensing images. Changes in natural lighting conditions (such as sunlight intensity, time of day, and cloud cover) affect the brightness and contrast of the image, thereby affecting road identification and segmentation.

[0005] Third, in high-density road environments, the spatial intervals between roads are small, and adjacent roads may blend together in remote sensing images, making them difficult to distinguish.

[0006] Most importantly, road wear, repairs, or the use of different materials can create visual breaks in images, while irregularities in street networks, such as dead ends, roundabouts, or multi-level overpasses, can also cause breaks in the continuous road network in images. Such breaks prevent the extracted roads from being used for remote sensing image map detection and road condition analysis. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a method for enhancing road connectivity in remote sensing images based on conditional masking. This method utilizes a semantic segmentation optimization framework based on conditional masking to enhance connectivity in remote sensing road prediction images. This method provides a semantic segmentation training optimization framework for the road disconnection problem.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] The method for enhancing road connectivity in remote sensing images based on conditional masking provided by this invention includes the following steps:

[0010] Step S1: Select a basic semantic segmentation network, train the network until it converges, and use the inference result of the network as the predicted image.

[0011] Step S2: Remove structural information from the predicted image obtained in step S1 by conditionally masking it according to different strategies.

[0012] Step S3: Design a channel fusion layer to fuse the conditional mask prediction image obtained in step S2 with the original input image into a three-channel image with clear road boundary semantic information;

[0013] Step S4: Use the three-channel image obtained in step S3 as the training set input of the same basic semantic segmentation network, calculate the loss using region continuity loss and focus loss, and backpropagate and update the parameters until the network converges to obtain the optimal semantic segmentation algorithm model.

[0014] Step S5: Use the convolutional semantic segmentation network trained in step S4 to make predictions. Use the predicted image as the input of step S2. Repeat steps S2-S5 to obtain the final semantic segmentation result image with excellent continuity.

[0015] Furthermore, the specific operations for training the basic semantic segmentation network in step S1 are as follows:

[0016] Step S11: Randomly select training and validation samples for the basic semantic segmentation network from the total sample; randomly select 80% of the total sample as training samples for the network and 20% as validation samples.

[0017] Step S12: Traverse all training samples, randomly select one of the three enhancement methods to process the image and add it to the total training samples;

[0018] Step S13: Iteratively train the basic semantic segmentation network using the loss function and update the gradient until the network converges relatively.

[0019] Furthermore, the specific operations of conditional masking according to different strategies in step S2 are as follows:

[0020] Step S21: Select two aspects, the spatial domain and the road skeleton domain, respectively, to perform conditional masking on the label image;

[0021] Step S22: Use three strategies—random masking, block-by-block masking, and chessboard masking—to revise and predict the image;

[0022] Step S23: Extract the road skeleton from the predicted image, randomly select pixels in the road skeleton and use matrix blocks to cover the corresponding road of the pixel;

[0023] Furthermore, the specific operations of conditional masking according to different strategies in step S2 are as follows:

[0024] Step S21 selects a spatial masking strategy to disrupt the structural information of the road, prompting the model to learn deeper representations from limited information. The selection of a road backbone mask aims to simulate various road damage scenarios in the image.

[0025] Step S22 selects three strategies: random masking, block-by-block masking, and checkerboard masking. Random masking divides the road label image into regular, non-overlapping blocks based on pixel size, then samples each block and randomly masks the image blocks within it according to different proportions. Block-by-block masking randomly generates rectangular blocks with different aspect ratios and pixel coordinates, and masks the label image based on these coordinates. Checkerboard masking generates a mask grid covering the entire image based on a determined grid size and region.

[0026] Step S23 creates a 7×7 cross convolution kernel to extract the skeleton of each road, then selects skeleton points in the image's internal region from the coordinate points, randomly generates rectangles, and finally creates a mask at the corresponding coordinates.

[0027] Furthermore, the specific fusion operation in step S3 is as follows:

[0028] The channel fusion layer consists of an adaptive multi-head attention module and a fully connected layer with dilated convolution. The predicted grayscale image after conditional masking and the original input RGB image are used as two inputs to the channel fusion layer and passed through the convolutional attention mechanism module, and then converted into a three-channel image through the self-attention mechanism. The multilayer perceptron with dilated convolution adaptively preserves the spatial information of the original image and provides the model with clear road boundary semantic information.

[0029] Furthermore, the specific model training operation in step S4 is as follows:

[0030] Step S41 uses the three-channel image after the channel fusion layer as input to the basic semantic segmentation network;

[0031] Step S42 initializes the basic semantic segmentation network and retrains it.

[0032] Step S43 uses the region continuity loss combined with the focus loss to update the model weights until the network converges.

[0033] Furthermore, the calculation process for regional continuity loss in step S43 is as follows:

[0034] The region continuity loss first simplifies the road structure of the predicted image and detects breakpoints using morphological opening and closing operations and binary image thinning. It then calculates the Euclidean distance between each breakpoint to update the weight map, enabling the model to focus on discontinuous road regions. Subsequently, a differentiable soft-IOU loss function is used for calculation, and this loss is multiplied by the weight map to obtain the final region continuity loss.

[0035] Furthermore, the iterative training operation in step S5 is as follows:

[0036] Step S51 uses the trained semantic segmentation model for reasoning;

[0037] Step S52 involves conditionally masking the predicted image obtained through inference;

[0038] Step S53 repeats steps S3-S5 of claim 1, iterating the training 5 times to obtain the final semantic segmentation result image.

[0039] Furthermore, in step S1, a total of 3216 original remote sensing images with a resolution of 512×512 are acquired, including two types of objects.

[0040] The system for enhancing road connectivity in remote sensing images based on conditional masking provided by the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described method.

[0041] The beneficial effects of this invention are as follows:

[0042] The present invention provides a method for enhancing road connectivity in remote sensing images based on conditional masking. This method can effectively improve the accuracy of semantic segmentation networks and improve the situation of road disconnection in remote sensing prediction maps. It is a method for enhancing road connectivity in remote sensing images based on conditional masking applicable to any convolutional semantic segmentation network.

[0043] This invention conditionally edits the predicted image, removing information that needs to be learned and optimized in road segmentation, thereby enabling the basic semantic segmentation model to learn deep representations from limited information. Masks based on remote sensing image space (random masks, block-by-block masks, grid masks) eliminate most of the redundancy in the image, providing the model with more complex image representations; masks based on the remote sensing road network skeleton can effectively break down structural information such as boundaries in the original label image, while largely preserving semantic context information.

[0044] This method fuses the conditionally masked predicted grayscale image with the original input image through a channel fusion layer, aiming to fit road occlusion scenarios and remove erroneous roads. The channel fusion layer includes a convolutional attention module and a multi-head self-attention module. The convolutional attention module preserves precise positional information in both the grayscale image and the original image, establishing long-term spatial dependencies. The multi-head self-attention module connects the feature information of both to enhance the clarity of road boundaries and achieve structural information interaction. To enhance the model's attention to road breakpoints in the predicted image, the model uses region continuity loss and focus loss to update parameters.

[0045] This method is a general framework that can be used during the training of any convolutional-based semantic segmentation network, improving the network's performance in high-resolution remote sensing semantic segmentation tasks and demonstrating universal applicability to the semantic segmentation problem of remote sensing images. Furthermore, based on convolutional neural networks, this method can process remote sensing images more efficiently, providing accurate location information for urban planning, land use, and environmental monitoring.

[0046] This method provides a conditional mask-based approach to enhance road connectivity in remote sensing images. It involves masking the predicted image obtained after training a basic semantic segmentation network, removing certain structural information from the feature map. This masked image is then used as an additional channel and incorporated into the original remote sensing image as input for training, thereby connecting fragmented roads. Simultaneously, this method combines a proposed region continuity loss with a focus loss to enhance the network's attention to the coordinates of disconnected road pixels. As a novel learning strategy, this approach restores damaged encodings during training, enabling the model to learn road connectivity patterns and remove erroneous roads.

[0047] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0048] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration:

[0049] Figure 1 This is a flowchart of a method for enhancing road connectivity in remote sensing images based on conditional masking.

[0050] Figure 2 This is an overview diagram of the structure of a method for enhancing road connectivity in remote sensing images based on conditional masking.

[0051] Figure 3 A visual example of a random masking strategy.

[0052] Figure 4 A visual example of a block-by-block masking strategy.

[0053] Figure 5 Example diagram for visualizing the grid masking strategy.

[0054] Figure 6 This is a sample visualization of the extracted road backbone.

[0055] Figure 7 A visual example of a road skeleton masking strategy.

[0056] Figure 8 This is an example diagram of the channel fusion layer structure.

[0057] Figure 9 This is an example diagram of the overall iterative training operation.

[0058] Figure 10 A colorized map of the road prediction results after inputting a high-resolution remote sensing image into the network. Detailed Implementation

[0059] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structure, features and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0060] This embodiment provides a method for enhancing road connectivity in remote sensing images based on conditional masking. This method is a complex computational process involving multiple stages from data preparation to iterative training of a deep learning model. In the data preparation stage, this embodiment trains a basic semantic segmentation network until convergence. Then, the network weights are used to predict road segmentation maps. Based on the road skeleton pixel positions in the segmentation maps, conditional masking operations are performed in different domains, including spatial domain random masking, block-by-block masking, grid masking, and skeleton domain road skeleton masking strategies. The masked images are used to create a training dataset, where the specific pixel positions of roads in the images are masked, thereby simulating real-world scenarios where roads are obscured by clouds or vehicles.

[0061] Semantic segmentation of high-resolution remote sensing images can sequentially determine the land cover type at each image pixel, providing rich semantic and locational information for terrain observation and land use. In recent years, with the successive proposals of convolutional neural networks and Transformers, more and more researchers have utilized deep neural networks to extract land cover information from high-resolution images. However, in the task of road segmentation of high-resolution remote sensing images, roads are often obscured by the shadows of trees or buildings, leading to discontinuous road extraction by semantic segmentation networks. Therefore, to overcome the many difficulties in semantic segmentation methods for high-resolution remote sensing road images, this embodiment provides a method based on conditional masking to enhance road connectivity in remote sensing images.

[0062] This embodiment employs a road connectivity optimization framework applicable to any convolutional network, aiming to improve the semantic segmentation performance of the network through additional supervision of masking operations. This framework conditionally masks the original segmentation results predicted by the semantic segmentation network to remove structural information. The masked grayscale image and the original remote sensing image are used as input to the channel fusion layer to enhance the network's semantic expressiveness. To improve the model's attention to disconnected regions, this implementation utilizes region continuity loss. The road connectivity optimization framework iteratively enhances the topology of the semantic segmentation network, resulting in more accurate predictions, clearer boundaries, and better spatial consistency.

[0063] like Figure 1 , Figure 2 As shown in the example, this example discloses a method for enhancing road connectivity in remote sensing images based on conditional masking. This method utilizes a semantic segmentation optimization framework based on conditional masking to enhance connectivity in remote sensing road prediction images. The steps are as follows:

[0064] Step S1: Select a basic semantic segmentation network, train the network until convergence, and use the inference result of the network as the predicted image; in this embodiment, the basic semantic segmentation network is a basic convolutional semantic segmentation network.

[0065] Step S2: Apply conditional masks to the predicted image obtained in step S1 using different strategies to remove structural information from the predicted image and obtain a conditional mask prediction image.

[0066] Step S3: Design a channel fusion layer to fuse the conditional mask prediction image obtained in step S2 with the original input image into a three-channel image with clear road boundary semantic information;

[0067] Step S4: Use the three-channel image obtained in step S3 as the training set input of the same basic semantic segmentation network, calculate the loss using region continuity loss and focus loss, and backpropagate and update the parameters until the network converges to obtain the optimal semantic segmentation algorithm model.

[0068] Step S5: Use the convolutional semantic segmentation network trained in step S4 to make predictions. Use the predicted image as the input of step S2. Repeat steps S2-S5 to obtain the final semantic segmentation result image with excellent continuity.

[0069] In this embodiment, the specific operations for training the basic semantic segmentation network in step S1 are as follows:

[0070] Step S11: Randomly select training and validation samples for the basic semantic segmentation network from the total sample; randomly select 80% of the total sample as training samples for the network and 20% as validation samples.

[0071] Step S12: Traverse all training samples, randomly select one of the three enhancement methods to process the image and add it to the total training samples;

[0072] Step S13: Iteratively train the basic semantic segmentation network using the loss function and update the gradient until the network converges relatively.

[0073] In this embodiment, the specific operations for the conditional mask according to different strategies in step S2 are as follows:

[0074] Step S21: Select two aspects, the spatial domain and the road skeleton domain, respectively, to perform conditional masking on the label image;

[0075] Step S22: Use three strategies—random masking, block-by-block masking, and chessboard masking—to revise and predict the image;

[0076] Step S23: Extract the road skeleton from the predicted image, randomly select pixels in the road skeleton and use matrix blocks to cover the corresponding road of the pixel;

[0077] Step S21 specifically involves:

[0078] The spatial masking strategy aims to disrupt the structural information of roads, prompting the model to learn deeper representations from limited information. The road backbone masking strategy aims to simulate various road damage scenarios in the image.

[0079] Step S22 specifically involves:

[0080] Three strategies were chosen: random masking, block-by-block masking, and checkerboard masking. Random masking divides the road label image into regular, non-overlapping blocks based on pixel size, then samples each block and randomly masks the image blocks within it according to different proportions. Block-by-block masking randomly generates rectangular blocks with different aspect ratios and pixel coordinates, and masks the label image based on these coordinates. Checkerboard masking generates a mask grid covering the entire image based on a determined grid size and region.

[0081] Step S23 specifically involves:

[0082] Create a 7×7 cross convolution kernel to extract the skeleton of each road, then select skeleton points in the image's internal region from the coordinate points and randomly generate rectangles, and finally create a mask at the corresponding coordinates.

[0083] like Figure 3 , Figure 4 , Figure 5 As shown, Figure 3 The predicted image obtained through a random masking strategy. Figure 4 The predicted image is obtained through a block-by-block masking strategy. Figure 5 Let be the image obtained after grid masking, where (a), (b), and (c) represent the original image, the label image, and the masked predicted image, respectively. Figure 6 The extracted road skeleton map is shown. Figure 7 This is the predicted image obtained by passing through the road skeleton mask. For example... Figure 8 As shown, Figure 8 This is a schematic diagram of the channel fusion layer structure.

[0084] In this embodiment, the channel fusion layer in step S3 specifically comprises:

[0085] The channel fusion layer includes an adaptive multi-head attention module and a fully connected layer;

[0086] The adaptive multi-head attention module includes a convolutional attention mechanism and a self-attention mechanism;

[0087] The convolutional attention mechanism enhances the convolutional neural network's attention to important features by combining spatial and channel attention, thereby improving the model's performance.

[0088] The self-attention mechanism dynamically adjusts the weights by allowing each element to interact with other elements in the sequence, thereby capturing long-distance dependencies in the model sequence.

[0089] The fully connected layer comprises a multilayer perceptron with dilated convolutions and a convolutional stream consisting of stacked lightweight convolutional operations. Multiple convolutional operations effectively enhance the network's feature extraction capabilities.

[0090] The multilayer perceptron uses multiple fully connected layers and nonlinear activation functions to enable the model to learn complex patterns and features of the input data;

[0091] The convolutional stream, by combining convolutional neural networks and flow models, performs probabilistic modeling and generation of data, thereby improving the accuracy of feature extraction by the model.

[0092] The convolutional attention mechanism in the adaptive multi-head attention module takes the conditionally masked predicted grayscale image and the original input RGB image as two inputs.

[0093] In this example, we first use average pooling and max pooling operations to aggregate the spatial information of the feature maps to obtain descriptors for the two feature maps. These two descriptors are then fed into a multilayer perceptron (using the same terminology as described above), and the output features are merged by element-wise summation. The calculation formula can be expressed as:

[0094]

[0095] In the formula, σ represents the Sigmoid function;

[0096] W1 and W0 indicate shared parameters;

[0097] This represents the spatial context descriptor after average pooling;

[0098] This represents the spatial context descriptor after max pooling;

[0099] MLP stands for Multilayer Perceptron.

[0100] The channel information of the aggregated features is then combined through a series of pooling operations and connected to a standard convolutional layer to finally generate a two-dimensional spatial attention image. In this embodiment, the series of pooling operations are performed in two consecutive pooling operations.

[0101] Its calculation formula can be expressed as:

[0102]

[0103] In the formula, σ represents the Sigmoid function;

[0104] This represents the spatial context descriptor after average pooling;

[0105] This represents the spatial context descriptor after max pooling;

[0106] f 7×7 This indicates a convolution operation with a kernel size of 7.

[0107] The self-attention mechanism in the adaptive multi-head attention module normalizes the obtained multiple feature maps, enabling the basic semantic convolutional segmentation network to capture precise location information and establish long-term spatial dependencies; the multiple feature maps provided in this embodiment use two feature maps.

[0108] These feature maps are then subjected to independent linear transformations in each multi-head attention to generate Query, Key, and Value. For each head, the attention score is calculated using the dot product between Query and Key, and the GELU activation function and scaling are applied to obtain the result. The result is added to the relative position bias and the Softmax function is applied to obtain the final attention weights.

[0109] The Query, Key, and Value can be represented as:

[0110]

[0111] In the formula, Indicates shared parameters;

[0112] F img This represents the result of multiplying the original input image with the normalized feature image.

[0113] F label This represents the result of multiplying the label map with the normalized feature prediction map;

[0114] I represents the number of self-attention heads;

[0115] D represents the number of channels;

[0116] N represents the window size.

[0117] The process of calculating the corresponding weights for multi-head attention can be represented as follows:

[0118] MultiHead(Q,K,V)=Concat(head1,head2,...,head h W O

[0119] Each head i The formula for calculating ) is:

[0120]

[0121] The formula for calculating single-head attention is:

[0122]

[0123] In the above formula, Q, K, and V represent Query, Key, and Value, respectively.

[0124] W O This represents the parameter matrix that can be learned during model training;

[0125] Concat means concatenating the calculation results of multiple heads;

[0126] h represents the number of heads;

[0127] d k Indicates the dimension of the Key vector;

[0128] The softmax function represents a standardized exponential function applied to each row.

[0129] The fully connected layer uses a 1×1 convolution and two 3×3 dilated convolutions. The padding and dilation parameters of the dilated convolutions are [6,6] and [12,12], respectively, to extract feature information from different receptive fields.

[0130] In this embodiment, the semantic segmentation algorithm model training operation in step S4 is as follows:

[0131] Step S41 uses the three-channel image after the channel fusion layer as input to the basic semantic segmentation network;

[0132] Step S42 initializes the basic semantic segmentation network and retrains it.

[0133] Step S43 uses the region continuity loss combined with the focus loss to update the semantic segmentation algorithm model weights until the network converges.

[0134] The loss calculation in step S43 includes a focal loss calculation function and a regional continuity loss calculation function, as detailed below:

[0135] The focus loss calculation function is based on binary cross-entropy loss. By introducing a balance parameter and a dynamically adjusted parameter, it reduces the model's focus on easily distinguishable objects, suppresses the imbalance between positive and negative samples, and thus improves the model's ability to segment fine roads and blurred boundaries. Its calculation formula is as follows:

[0136]

[0137] L f =―α t (1―p t ) γ log(p t )

[0138] In the formula, p t This represents the probability that the model correctly predicts the class.

[0139] α t This represents the balance parameter, used to balance the ratio between positive and negative sample losses;

[0140] γ is a dynamically adjustable parameter used to reduce the contribution of easily classified samples and increase the influence of difficult-to-classify samples;

[0141] α t Set to 0.25;

[0142] The γ value is set to 2.

[0143] The region continuity loss calculation function first simplifies the road structure of the predicted image and detects breakpoints using morphological opening and closing operations and binary image thinning. A road breakpoint *c* is defined as a pixel location connected to only one other pixel; this location is detected by examining the neighborhood of a skeleton pixel. For each skeleton pixel, if only one of its four neighborhood pixels belongs to the road category, then that location is considered a road breakpoint *c*. Subsequently, a weight map is initialized (initial weights are set to 1), and the Euclidean distance between each breakpoint is calculated sequentially. The corresponding weights are updated based on the distances between breakpoints, making the model focus more on broken road regions. The weight calculation and update formulas are as follows:

[0144] w c =exp(α·D min (c))

[0145] In the formula, D min (c) represents the Euclidean distance between the nearest breakpoint to the breakpoint;

[0146] α is a positive adjustment parameter used to control the rate at which the distance affects the weight decay.

[0147] α is set to 50.

[0148] The loss is then calculated using a differentiable soft-IOU loss function and multiplied by the weight map to obtain the region continuity loss, which is calculated as follows.

[0149]

[0150] In the formula, N is the total number of categories;

[0151] This represents the predicted score of class n at position k;

[0152] These are real sample labels.

[0153] The two loss results are added together as the total loss for model training, and backpropagated to update the gradient until the semantic segmentation network model converges. Figure 9 As shown, Figure 9 This is a training flowchart.

[0154] In this embodiment, the specific operation of iterative training in step S5 is as follows:

[0155] Step S51 uses the trained semantic segmentation algorithm model for reasoning;

[0156] Step S52 involves conditionally masking the predicted image obtained through inference;

[0157] Step S53 repeats steps S3-S5, iterating 5 times. The final semantic segmentation result is used as the prediction result. The matching relationship between the grayscale value and RGB value of each object category is found. The RGB value of the road category is assigned based on the grayscale value, and color processing is performed. In this embodiment, the grayscale and RGB values ​​assigned to the road and background categories are shown in Table 1.

[0158] Table 1

[0159]

[0160]

[0161] like Figure 10 As shown in Table 1, the colorized prediction results of a high-resolution remote sensing image with an input size of 512×512 after entering the network in this embodiment are as follows:

[0162] When the target object is a road, assign it RGB values ​​of 255, 255, and 255 respectively, and assign it a grayscale value of 0 and a color of white.

[0163] When the target object is the background, assign it RGB values ​​of 0, 0, 0, and a grayscale value of 1, with the color being black.

[0164] The Intersection Over Union (IoU) precision and mean IoU of the prediction results before and after using this invention are shown in Table 2. Precision is defined as the number of correctly predicted positive samples divided by the total number of predicted positive samples, and recall is defined as the number of correctly predicted positive samples divided by the total number of actual positive samples. The F1 score is obtained by averaging the scores of all classes. IoU represents the similarity between the predicted region and the ground truth region in a given set of images.

[0165] Table 2

[0166] method accuracy Recall rate F1 score IoU mIoU Before use 0.775 0.761 0.769 0.625 0.804 After use 0.787 0.772 0.780 0.636 0.813

[0167] As can be seen from the experimental results and Table 2, this method effectively improves the segmentation accuracy of the semantic segmentation network for road backbones, and can provide rich feature information for urban planning, road construction and road network extraction.

[0168] The system for enhancing road connectivity in remote sensing images based on conditional masking provided in this embodiment includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described method.

[0169] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for enhancing road connectivity in remote sensing images based on conditional masking, characterized in that: Includes the following steps: Step S1: Select a basic semantic segmentation network, train the network until it converges, and use the inference result of the network as the predicted image. Step S2: Remove structural information from the predicted image obtained in step S1 using conditional masks with different strategies; the specific operations for conditional masks with different strategies are as follows: Step S21: Select two aspects, the spatial domain and the road skeleton domain, respectively, to perform conditional masking on the label image; Step S22: Use three strategies—random masking, block-by-block masking, and chessboard masking—to revise and predict the image; Step S23: Extract the road skeleton from the predicted image, randomly select pixels in the road skeleton and use matrix blocks to cover the corresponding road of the pixel; Step S3: Design a channel fusion layer to fuse the conditional mask prediction image obtained in step S2 with the original input image into a three-channel image with clear road boundary semantic information; the specific fusion operation is as follows: The channel fusion layer includes an adaptive multi-head attention module and a fully connected layer with dilated convolutions; The predicted grayscale image after conditional masking and the original input RGB image will be used as two inputs to the channel fusion layer and passed through the convolutional attention mechanism module respectively. It is then converted into a three-channel image through a self-attention mechanism; Multilayer perceptrons with dilated convolutions adaptively preserve the spatial information of the original image and provide clear road boundary semantic information to the basic semantic segmentation network; Step S4: Use the three-channel image obtained in step S3 as the training set input of the same basic semantic segmentation network, calculate the loss using region continuity loss and focus loss, and backpropagate and update the parameters until the network converges to obtain the optimal semantic segmentation algorithm model. Step S5: Use the semantic segmentation algorithm model trained in step S4 to make predictions. Use the predicted image as the input for step S2. Repeat steps S2-S5 to obtain the final semantic segmentation result image with excellent continuity.

2. The method for enhancing road connectivity in remote sensing images based on conditional masking according to claim 1, characterized in that, The specific steps for training the basic semantic segmentation network in step S1 are as follows: Step S11: Randomly select training and validation samples for the basic semantic segmentation network from the total sample. Step S12: Traverse all training samples, randomly select one of the three enhancement methods to process the image and add it to the total training samples; Step S13: Iteratively train the basic semantic segmentation network using the loss function and update the gradient until the network converges relatively.

3. The method for enhancing road connectivity in remote sensing images based on conditional masking according to claim 1, characterized in that, The specific operations for conditional masking according to different strategies in step S2 are as follows: Step S21 selects a spatial masking strategy to destroy the structural information of the road, prompting the model to learn a deeper representation from limited information; selects a road backbone mask to simulate various road damage scenarios in the image. Step S22 selects three strategies: random masking, block masking, and checkerboard masking. The random masking divides the road label image into regular non-overlapping blocks according to the pixel size, and then samples each block and randomly masks the image blocks in it according to different proportions. The block-by-block mask randomly generates rectangular blocks with different aspect ratios and pixel coordinates, and masks the label image based on these coordinates; the checkerboard mask generates a mask grid that covers the entire image based on the determined grid size and region. Step S23 creates a cross convolution kernel to extract the skeleton of each road, then selects the skeleton points in the image's internal region from the coordinate points, randomly generates a rectangle of a certain size, and finally creates a mask at the corresponding coordinates.

4. The method for enhancing road connectivity in remote sensing images based on conditional masking according to claim 1, characterized in that, The semantic segmentation algorithm model training operation in step S4 is as follows: Step S41 uses the three-channel image after the channel fusion layer as input to the basic semantic segmentation network; Step S42 initializes the basic semantic segmentation network and retrains it. Step S43 uses the region continuity loss combined with the focus loss to update the semantic segmentation algorithm model weights until the network converges.

5. The method for enhancing road connectivity in remote sensing images based on conditional masking according to claim 4, characterized in that, The calculation process for regional continuity loss in step S43 is as follows: The region continuity loss first simplifies the road structure of the model prediction image and detects breakpoints by using morphological opening and closing operations and binary image thinning operations. It then calculates the Euclidean distance between each breakpoint to update the weight map, making the model focus on the broken road regions. Subsequently, it is calculated using a differentiable soft-IOU loss function and multiplied by the weight map to finally obtain the region continuity loss.

6. The method for enhancing road connectivity in remote sensing images based on conditional masking according to claim 1, characterized in that, The iterative training operation in step S5 is as follows: Step S51 uses the trained semantic segmentation algorithm model for reasoning; Step S52 involves conditionally masking the predicted image obtained through inference; Step S53 repeats steps S3-S5 of claim 1, iterating the training 5 times to obtain the final semantic segmentation result image.

7. The method for enhancing road connectivity in remote sensing images based on conditional masking according to any one of claims 1 to 6, characterized in that, In step S1, a total of 3216 original remote sensing images with a resolution of 512×512 are acquired, including two types of objects.

8. A system for enhancing road connectivity in remote sensing images based on conditional masking, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Remote sensing image semantic segmentation method based on channel-space attention and DeeplabV3plus

    CN114596500A

  • Remote sensing image road extraction method, device and equipment based on mask image modeling

    CN116740570A