A road network optimization method considering direction-connectivity

Abstract

The present invention relates to the field of remote sensing image technology, and in particular to a road network optimization processing method that takes into account direction-connectivity, wherein a direction-guided road information reasoning module utilizes the structural and directional characteristics of the road to perform spatial information reasoning on broken roads from different directions, enabling the model to effectively model the spatial relationship between different rows or columns, thereby repairing broken roads. The post-processing network model simultaneously outputs prediction results of the road surface and road direction, and utilizes road semantic label information for loss supervision, thereby enhancing the reasoning ability of the post-processing network model and improving the extraction accuracy of the road surface. In addition, the input of the post-processing network model in the present invention includes the initial segmentation result, and the use of an encoder-decoder structure and jump connections in the post-processing network model will not result in the loss of narrow roads and the introduction of irrelevant noise information, thereby obtaining a more refined road segmentation map and road direction map.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing images, and in particular to a road network optimization processing method taking into account direction-connectivity. Background Art

[0002] Road information extracted from remote sensing images has a wide range of applications, including urban planning, land management, traffic management, automatic navigation, route analysis, and emergency response. Deep learning technology provides a reliable way to extract roads from high-resolution images. Deep learning methods are mostly based on encoder-decoder structures, which can extract effective road semantic features from complex scenes and have strong processing capabilities for complex and highly diverse roads. However, the problem of road occlusion is still one of the main problems affecting the performance of high-resolution remote sensing image road extraction tasks. Factors that occlude roads include trees, vehicles, buildings, or shadows, which will affect the consistency of the road's spectrum, color, and texture to varying degrees, directly leading to incomplete and discontinuous road extraction results. Therefore, how to effectively maintain the integrity and continuity of the road during the road extraction process and effectively enhance the model's anti-occlusion ability is a research focus in the road extraction task.

[0003] In the prior art, road context information is often used to enhance the semantic features of the occluded road parts. Commonly used context algorithms include multi-scale features and attention mechanisms. Multi-scale features model the dependency between objects and background environments, while attention mechanisms model the correlation between similar objects. However, this method still has the problem of insufficient utilization of multi-scale feature modules and insufficient coupling with feature learning processes, as well as the large number of parameters and computational complexity of the self-attention mechanism, making it difficult to apply to high-resolution feature maps. In addition, the prior art also directly optimizes and processes the initial road extraction results through post-processing, aiming to repair incomplete and discontinuous road sections. However, these methods do not perform targeted processing at road discontinuities, resulting in the inability of existing post-processing models to effectively cope with more complex complete occlusion situations. In addition, these post-processing methods have large computational complexity and training difficulty, high spatial and temporal complexity, and are difficult to achieve efficient post-processing optimization.

[0004] In summary, there is an urgent need to provide a road network optimization processing method that takes direction-connectivity into consideration to solve the technical problems existing in the prior art. Summary of the invention

[0005] The present invention aims to provide a road network optimization processing method taking into account direction-connectivity to solve the technical problems existing in the prior art. The specific technical scheme is as follows: A road network optimization processing method taking into account direction-connectivity comprises the following steps:

[0006] S1, obtain remote sensing images and their road semantic label training sets to generate road direction maps;

[0007] S2, using the road semantic label training set to train the semantic segmentation network model and generate a road segmentation map;

[0008] S3, constructing a post-processing network model for the segmentation results, and optimizing the road segmentation map;

[0009] S4, post-processing network model parameter update;

[0010] S5, post-processing network model update: repeat steps S3 and S4 until the post-processing network model converges; save the trained post-processing network model;

[0011] S6. Road segmentation map optimization: The road segmentation map and the remote sensing image are input into the post-processing network model trained in step S5 to obtain a refined road segmentation map and a road direction map.

[0012] Furthermore, step S1 is specifically as follows:

[0013] The remote sensing images and the corresponding road semantic label maps are obtained by downloading from the Internet; the obtained remote sensing images and the corresponding road semantic label maps are cropped into image block pairs of the same size with the same step length to form a remote sensing image and its road semantic label training set; the road semantic labels are used to generate a road direction map.

[0014] Furthermore, step S2 is specifically as follows:

[0015] S2.1, input the remote sensing images and semantic label maps in the road semantic label training set into the semantic segmentation network model;

[0016] S2.2, semantic segmentation network model update: calculate the error through the loss function, and back-propagate the error, update the parameters of the semantic segmentation network until the semantic segmentation network model converges; save the semantic segmentation network model, that is, obtain the trained semantic segmentation network model;

[0017] S2.3, input the remote sensing image into the trained semantic segmentation network model to obtain the road segmentation map;

[0018] Further, in step S3, the post-processing network model includes an encoding part and a decoding part;

[0019] The encoding part includes a semantic feature extraction module and a road information reasoning module for direction guidance;

[0020] The semantic feature extraction module is used to extract feature images with different sizes and recognition capabilities. Specifically, the RGB three-channel image of the remote sensing image and the road segmentation map generated by the semantic segmentation network model are spliced ​​and input into the first residual learning unit of the encoder to obtain the feature map output by the first residual learning unit; the feature map output by the first residual learning unit is downsampled by two times, input into the next residual learning unit and output the feature map; the steps of downsampling the feature map by two times and inputting into the next residual learning unit are repeated to obtain feature images with different sizes and recognition capabilities;

[0021] The direction-guided road information reasoning module is used to obtain a feature image containing road semantic information. Specifically, the feature map output by the last residual learning unit of the encoding part is input into the direction-guided road information reasoning module, and the road semantic information is transferred through spatial relationship convolution reasoning from top to bottom, from bottom to top, from left to right, and from right to left, so as to obtain a feature image containing road semantic information;

[0022] The decoding part is used to obtain a fine road segmentation map and a road direction map with the same size as the remote sensing image.

[0023] Furthermore, the semantic feature extraction module is a UNet-like structure, and the UNet-like structure is a structure in which the original convolutional layer in the UNet structure is replaced by a residual learning unit.

[0024] Furthermore, the reasoning process of the road information reasoning module for direction guidance is:

[0025] a) Input the feature map output by the last residual learning unit of the encoding part into the direction-guided road information reasoning module, split the feature map by row, and obtain multiple rows of independent feature tensors; the split feature map is sequentially passed through two reasoning processes from top to bottom and from bottom to top;

[0026] The specific calculation formula of the top-down reasoning process is as follows:

[0027]

[0028] in, represents the feature tensor of the i-th row after splitting by row, represents the feature tensor of the i-th row after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer;

[0029] The specific calculation formula of the bottom-up reasoning process is as follows:

[0030]

[0031] in, represents the feature tensor of the i-th row after splitting by row, represents the feature tensor of the i-th row after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer;

[0032] b) Perform convolution operations on each row in turn in the two reasoning processes from top to bottom and from bottom to top, add the convolution result of the previous row to the next row data element by element, obtain the input data for processing the next row, and realize information transmission between different rows; after the convolution operations from top to bottom and from bottom to top, all row tensors are recombined to obtain the combined feature map;

[0033] c) Split the combined feature map obtained in step b) by columns to obtain multiple columns of independent feature tensors; output a feature map of the spatial relationship between rows and columns of the convolution modeling through two reasoning processes from left to right and from right to left, that is, obtain a feature image containing road semantic information;

[0034] Specifically, the specific calculation formula of the reasoning process from left to right is as follows:

[0035]

[0036] in, represents the feature tensor of the jth column after column segmentation, represents the feature tensor of the jth column after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer;

[0037] The specific calculation process of the reasoning stage from right to left is as follows:

[0038]

[0039] in, represents the feature tensor of the jth column after column segmentation, represents the feature tensor of the jth column after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer.

[0040] Furthermore, the decoding process of the decoding part is as follows: the feature map output by the direction-guided road information inference module is spliced ​​with the feature output by the third residual learning unit of the encoder through a jump connection to obtain a third spliced ​​feature, and the third spliced ​​feature is input into the first residual learning unit of the decoder to output a first residual feature; the first residual feature is upsampled twice, and is spliced ​​with the feature output by the second residual learning unit of the encoder through a jump connection to obtain a second spliced ​​feature, and then the second spliced ​​feature is input into the second residual learning unit of the decoder to output a second residual feature; the second residual feature is upsampled twice, and is spliced ​​with the feature output by the first residual learning unit of the encoder and the road segmentation map generated by the semantic segmentation network model through a jump connection to obtain a first spliced ​​feature, and then the first spliced ​​feature is input into the third residual learning unit of the decoder to output a third residual feature; a fine road segmentation map and a road direction map with the same size as the remote sensing image are obtained through a 1×1 convolution output.

[0041] Furthermore, step S4 is specifically as follows: using the cross entropy loss function to calculate the error between the fine road segmentation map obtained by the decoding part and the real semantic label map in the road semantic label training set; using the Dice function to calculate the error between the road direction map obtained by the decoding part and the road direction map generated by the real semantic label map in the road semantic label training set; back-propagating the two errors and updating the post-processing network model parameters.

[0042] Furthermore, the cross entropy loss function is:

[0043]

[0044] Among them, N is the set of image pixels, a is the a-th pixel, and y represents the true label value. Represents the predicted label value.

[0045] Furthermore, the Dice loss function is:

[0046]

[0047] Among them, X represents the pixel category of the model predicted image, Y represents the pixel label of the real image, |X∩Y| is the intersection between X and Y, which is approximately the dot product between the predicted image and the real image, and the element results of the dot product are added together. |X| represents the addition of the positive example elements in X, and |Y| represents the addition of the positive example elements in Y.

[0048] The application of the technical solution of the present invention has the following beneficial effects:

[0049] The present invention provides a road network optimization processing method that takes direction-connectivity into consideration, which can optimize and repair the prediction results of the road extraction model, improve the accuracy of road extraction and the connectivity of the road network; wherein, the direction-guided road information reasoning module uses the structural and directional characteristics of the road to perform spatial information reasoning on the broken roads from different directions, so that the model can effectively model the spatial relationship between different rows or columns, thereby repairing the broken roads. The post-processing network model simultaneously outputs the prediction results of the road surface and the road direction, and uses the road semantic label information for loss supervision. Because there is a strong correlation between the road direction and the road surface, the two can constrain and promote each other, enhance the reasoning ability of the post-processing network model, and improve the extraction accuracy of the road surface. In addition, the input of the post-processing network model in the present invention includes the initial segmentation result, and the use of the encoder-decoder structure and the jump connection in the post-processing network model will not lead to the loss of narrow roads and the introduction of irrelevant noise information, thereby obtaining a more refined road segmentation map and road direction map.

[0050] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0052] Figure 1 is a flow chart of a road network optimization processing method taking into account direction-connectivity in the present invention;

[0053] Figure 2 is the post-processing network model diagram;

[0054] Figure 3 It is a flow chart of feature image extraction of the road information reasoning module for direction guidance;

[0055] Figure 4 is the qualitative evaluation result of the present invention on the Massachusetts dataset;

[0056] Figure 5 It is the qualitative evaluation result of the present invention on the DeepGlobe dataset;

[0057] Figure 6 This is the qualitative evaluation result of the present invention on the CHN6-CUG dataset. DETAILED DESCRIPTION

[0058] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0059] Example:

[0060] See also Figure 1 This embodiment provides a road network optimization processing method taking into account direction-connectivity, comprising the following steps:

[0061] S1. Obtain remote sensing images and their road semantic label training sets to generate a road direction map. Specifically, obtain remote sensing images and corresponding road semantic label maps by downloading them online. Crop the acquired remote sensing images and corresponding road semantic label maps into image block pairs of the same size with the same step length to form a remote sensing image and its road semantic label training set. Generate a road direction map using the road semantic labels through a road direction map generation algorithm.

[0062] S2. Using the road semantic label training set to train the semantic segmentation network model to generate a road segmentation map; specifically comprising the following steps:

[0063] S2.1, input the remote sensing images and semantic label maps in the road semantic label training set into the semantic segmentation network model;

[0064] S2.2, semantic segmentation network model update: calculate the error through the loss function, and back-propagate the error, update the parameters of the semantic segmentation network until the semantic segmentation network model converges; save the semantic segmentation network model, that is, obtain the trained semantic segmentation network model;

[0065] S2.3, input the remote sensing image into the trained semantic segmentation network model to obtain the road segmentation map;

[0066] S3, constructing a segmentation result post-processing network model (RIRNet) to optimize the road segmentation map; the post-processing network model includes an encoding part and a decoding part;

[0067] The encoding part includes a semantic feature extraction module and a road information reasoning module for direction guidance;

[0068] The semantic feature extraction module is used to extract feature images with different sizes and recognition capabilities. The semantic feature extraction module is a UNet-like structure, and the UNet-like structure replaces the original convolutional layer in the UNet structure with a residual learning unit. Specifically: the RGB three-channel image of the remote sensing image and the road segmentation map generated by the semantic segmentation network model are spliced ​​and input into the first residual learning unit of the encoder to obtain the feature map output by the first residual learning unit; the feature map output by the first residual learning unit is downsampled twice, input into the next residual learning unit and outputs the feature map; repeat the steps of downsampling the feature map twice and inputting into the next residual learning unit (in this embodiment, the semantic feature extraction module needs to repeat this step three times in total) to obtain feature images with different sizes and recognition capabilities;

[0069] The direction-guided road information reasoning module is used to obtain a feature image containing road semantic information. Specifically, the feature map output by the last residual learning unit of the encoding part is input into the direction-guided road information reasoning module, and the road semantic information is transferred through spatial relationship convolution reasoning from top to bottom, from bottom to top, from left to right, and from right to left, so as to obtain a feature image containing road semantic information;

[0070] The reasoning process of the road information reasoning module for direction guidance is:

[0071] a) Input the feature map output by the last residual learning unit of the encoding part into the direction-guided road information reasoning module, split the feature map by row, and obtain multiple rows of independent feature tensors; the split feature map is sequentially passed through two reasoning processes from top to bottom and from bottom to top;

[0072] The specific calculation formula of the top-down reasoning process is as follows:

[0073]

[0074] in, represents the feature tensor of the i-th row after splitting by row, represents the feature tensor of the i-th row after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer;

[0075] The specific calculation formula of the bottom-up reasoning process is as follows:

[0076]

[0077] in, represents the feature tensor of the i-th row after splitting by row, represents the feature tensor of the i-th row after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer;

[0078] b) Perform convolution operations on each row in turn in the two reasoning processes from top to bottom and from bottom to top, add the convolution result of the previous row to the next row data element by element, obtain the input data for processing the next row, and realize information transmission between different rows; after the convolution operations from top to bottom and from bottom to top, all row tensors are recombined to obtain the combined feature map;

[0079] c) Split the combined feature map obtained in step b) by columns to obtain multiple columns of independent feature tensors; output a feature map of the spatial relationship between rows and columns of the convolution modeling through two reasoning processes from left to right and from right to left, that is, obtain a feature image containing road semantic information;

[0080] Specifically, the specific calculation formula of the reasoning process from left to right is as follows:

[0081]

[0082] in, represents the feature tensor of the jth column after column segmentation, represents the feature tensor of the jth column after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer;

[0083] The specific calculation process of the reasoning stage from right to left is as follows:

[0084]

[0085] in, represents the feature tensor of the jth column after column segmentation, represents the feature tensor of the jth column after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer.

[0086] The decoding part is used to obtain a fine road segmentation map and a road direction map with the same size as the remote sensing image. The decoding process of the decoding part is: the feature map output by the direction-guided road information reasoning module is spliced ​​with the feature output by the third residual learning unit of the encoder through a jump connection to obtain a third spliced ​​feature, and the third spliced ​​feature is input into the first residual learning unit of the decoder to output a first residual feature; the first residual feature is upsampled twice, and is spliced ​​with the feature output by the second residual learning unit of the encoder through a jump connection to obtain a second spliced ​​feature, and then the second spliced ​​feature is input into the second residual learning unit of the decoder to output the second residual feature; the second residual feature is upsampled twice, and is spliced ​​with the feature output by the first residual learning unit of the encoder and the road segmentation map generated by the semantic segmentation network model through a jump connection to obtain a first spliced ​​feature, and then the first spliced ​​feature is input into the third residual learning unit of the decoder to output a third residual feature; a fine road segmentation map and a road direction map with the same size as the remote sensing image are obtained through a 1×1 convolution output.

[0087] S4, update the parameters of the post-processing network model; use the cross entropy loss function to calculate the error between the refined road segmentation map obtained by the decoding part and the real semantic label map in the road semantic label training set; use the Dice function to calculate the error between the road direction map obtained by the decoding part and the road direction map generated by the real semantic label map in the road semantic label training set; back-propagate the two errors and update the parameters of the post-processing network model.

[0088] The cross entropy loss function is:

[0089]

[0090] Among them, N is the set of image pixels, a is the a-th pixel, and y represents the true label value. Represents the predicted label value.

[0091] The Dice loss function is:

[0092]

[0093] Among them, X represents the pixel category of the model predicted image, Y represents the pixel label of the real image, |X∩Y| is the intersection between X and Y, which is approximately the dot product between the predicted image and the real image, and the element results of the dot product are added together, |X| represents the addition of positive elements in X, and |Y| represents the addition of positive elements in Y. Specifically, for a certain semantic segmentation category, the pixels predicted by the model as that category are regarded as positive elements in X, and the pixels of that category in the pixel label are regarded as positive elements in Y.

[0094] S5, post-processing network model update: repeat steps S3 and S4 until the post-processing network model converges; save the trained post-processing network model;

[0095] S6. Road segmentation map optimization: The road segmentation map and the remote sensing image are input into the post-processing network model trained in step S5 to obtain a refined road segmentation map and a road direction map.

[0096] For the road network optimization processing method considering direction-connectivity of the present invention, experimental demonstrations on three data sets (Massachusetts data set, Deepglobe data set and CHN6-CUG data set) and an ablation experiment were carried out, as follows:

[0097] Massachusetts data set: The Massachusetts road data set is an aerial remote sensing image data set collected in Massachusetts, USA, with a total coverage area of ​​about 2,600 square kilometers. The Massachusetts data set covers multiple geographical scenes such as urban, suburban and rural scenes. The road scale in the remote sensing image is small and relatively simple, but there are a lot of occlusions, which can well test the performance of the road extraction model. The data set contains a total of 1,171 images, of which 1,108 images are used for model training, 14 images are used for model verification, and 49 images are used for model testing. The spatial resolution of the data set is 1.2m, and the size of each image is 1500x 1500 pixels. In this embodiment, the remote sensing images of 1500x 1500 in the training set and the verification set are randomly cropped into several image blocks of 256x 256 in size, and a total of 20,000 images are obtained for model training, and 500 images are used for model testing and verification. At the same time, during the model training process, this embodiment randomly performs data enhancement on the training set images to expand the data set and enhance the generalization ability of the model.

[0098] Table 1 Quantitative evaluation results of the RIRNet model in this embodiment on the Massachusetts dataset

[0099]

[0100] In Table 1, the values ​​in brackets are the accuracy of the prediction results of the road segmentation model after post-processing optimization minus the accuracy before optimization. It can be seen from Table 1 that the RIRNet model proposed in the present invention can achieve excellent post-processing optimization effects on the Massachusetts dataset, and can achieve good accuracy improvements on all road extraction models. Among them, the road extraction model with the best post-processing optimization effect is the DANet model, which improves the accuracy by 9.04% on F1 and 10.95% on the intersection-over-union ratio; the road extraction model with the poorest post-processing optimization effect is the CA-HR-RoadNet model, but it can also improve the accuracy by 0.25% on F1 and 0.34% on the intersection-over-union ratio; this fully proves the effectiveness of the RIRNet model proposed in the present invention in post-processing optimization.

[0101] Figure 4 This is the qualitative analysis result of the post-processing network model of the present invention on the Massachusetts data set. In this embodiment, the UNet and CA-HR-RoadNet models are selected as qualitative analysis models. It can be clearly seen in the visualization results that the RIRNet model proposed in the present invention has excellent post-processing optimization effects on the Massachusetts data set. Not only can the integrity and continuity of the road be greatly improved, but the road boundary is smoother and the noise information is significantly reduced. Before the optimization process, there are multiple areas marked with circles in the visualization results. These areas are mainly complex background areas and areas where the road is severely occluded, which leads to errors such as missing, misclassification, incompleteness and discontinuity in the initial extraction results of the road extraction model. The RIRNet model proposed in the present invention can effectively optimize these erroneous predictions. Specifically, in the results of the first and third rows, the initial extraction results of the UNet and CA-HR-RoadNet models both have road discontinuity problems caused by complete occlusion, and the RIRNet model proposed in the present invention can well perform information reasoning on these broken parts of the road, thereby effectively repairing the interrupted road sections. In the visualization results of the second row, there are incomplete roads in the initial extraction results of the UNet model, and there are road misclassifications in the initial extraction results of the CA-HR-RoadNet model. The RIRNet model of the present invention can effectively optimize and handle these erroneous predictions, which is mainly due to the effectiveness of the road information reasoning module and road direction reasoning tasks in the RIRNet model. In the results of the fourth row, the RIRNet model can not only effectively handle the road discontinuity problem in the densely populated urban road area, but also effectively handle the road omission problem in the initial extraction results.

[0102] Deepglobe data set: The Deepglobe road data set is a satellite remote sensing image data set that contains images collected from Thailand, Indonesia, and India, covering an area of ​​2,220 square kilometers. The data set includes geographical scenes such as cities and suburbs. The road types in the images are rich, the background environment is relatively complex, and there are a large number of road occlusions. It has a large difficulty in road extraction and can comprehensively detect the performance of the road extraction model. The original data set contains 8,570 three-channel satellite remote sensing images, of which only 6,226 images contain corresponding real label data. The size of each image is 1024x1024 pixels, and the image spatial resolution is 50 cm. In this embodiment, the remote sensing images containing real labels are divided into data in a ratio of 7:1:2. Among them, the training set contains 5,000 images, the verification set contains 226 images, and the test set contains 1,000 images. In this embodiment, the 1024x 1024 remote sensing images in the training set and the validation set are randomly cropped into several image blocks of 256x 256, and a total of 25,000 images are obtained for model training and 1,130 images are used for model validation. At the same time, during the model training process, data enhancement is randomly performed on the training set images to expand the data set and enhance the generalization ability of the model.

[0103] Table 2 Quantitative evaluation results of the RIRNet model in this example on the DeepGlobe dataset

[0104]

[0105] In Table 2, the values ​​in brackets are the accuracy of the prediction results of the road segmentation model after post-processing optimization minus the accuracy before optimization. Table 2 is a quantitative analysis of the post-processing network model of this embodiment on the DeepGlobe dataset. It can be seen from Table 2 that the RIRNet model proposed in this embodiment can achieve excellent post-processing optimization effects on the DeepGlobe dataset, and can obtain good accuracy improvements on all road extraction models. Among them, the road extraction model with the best post-processing optimization effect is the DenseASPP model, which improves the accuracy by 5.76% on the F1 index and 7.22% on the intersection-over-union ratio index; the road extraction model with the poor post-processing optimization effect is the SENet model, but it can also improve the accuracy by 0.28% on the F1 index, and can also improve the accuracy by 0.36% on the intersection-over-union ratio index; this fully proves the effectiveness of the RIRNet model proposed in the present invention in post-processing optimization.

[0106] Figure 5This is the qualitative analysis result of the post-processing model of the present invention on the DeepGlobe dataset. This embodiment selects the ResUNet model and the CA-HR-RoadNet model as the main analysis objects. It can be clearly seen from the visualization results that the RIRNet model proposed in the present invention has excellent post-processing optimization performance; after the post-processing optimization, the initial extraction results of the two models have greatly improved the integrity and continuity of the roads. At the same time, the optimized road boundaries are smoother, and irrelevant noise information is significantly reduced. Specifically, there are errors in the initial segmentation results, such as omissions, misclassifications, and breaks, etc., and the RIRNet model proposed in the present invention can effectively handle erroneous predictions in these areas, thereby improving the accuracy of road extraction. Among them, the results of the first row show that the RIRNet model can achieve excellent post-processing effects in areas with complex backgrounds, effectively repair broken roads, and extract missing road segments. The visualization results in the second to fourth rows show the post-processing optimization effect of the RIRNet model in suburban road areas. In the initial road extraction results of the ResUNet and CA-HR-RoadNet models, there are a large number of roads that are completely blocked by vegetation, resulting in road discontinuities. The RIRNet model proposed in the present invention can well perform information reasoning on these discontinuous road areas and repair broken road sections. The visualization results in the fifth row show the post-processing effect of the RIRNet model in densely populated road areas in the city. There is a problem of discontinuity caused by roads being blocked by buildings in the image, and the RIRNet model of the present invention can effectively reason about the breaks and repair broken road sections.

[0107] CHN6-CUG data set: CHN6-CUG road data set is a large satellite image data set containing representative cities in China. The road types in this data set include railways, highways, urban roads and rural roads, etc., and the scale of the roads is relatively rich, the background environment is relatively complex, and the differences between different roads are relatively large, which can better detect the effect of the road extraction model. The CHN6-CUG road data set contains a total of 4511 remote sensing images of size 512x 512 and their corresponding real label data. Among them, 3608 images are used for model training, 903 images are used for testing and accuracy assessment, and the spatial resolution of the image is 50cm. In this embodiment, the remote sensing images of size 512x 512 in the training set are randomly cut into several image blocks of size 256x 256, and a total of 23,000 images are obtained for model training. At the same time, in the model training process, the training set images are randomly enhanced to expand the data set and enhance the generalization ability of the model.

[0108] Table 3 Quantitative evaluation results of the RIRNet model in this paper on the CHN6-CUG dataset

[0109]

[0110] In Table 3, the values ​​in brackets are the results of the prediction results of the road segmentation model after post-processing optimization minus the accuracy before optimization. Table 3 is a quantitative analysis of the post-processing network model of this embodiment on the CHN6-CUG dataset. It can be seen from Table 3 that the RIRNet model proposed in the present invention can achieve excellent post-processing optimization effects on the Massachusetts dataset, and can achieve good accuracy improvements on all road extraction models. Among them, the road extraction model with the best post-processing optimization effect is the ResUNet model, which improves the accuracy by 6.53% on F1 and 8.33% on the intersection-over-union ratio; the road extraction model with the poor post-processing optimization effect is the CA-HR-RoadNet model, but it can also improve the accuracy by 0.30% on F1, and at the same time, it can also improve the accuracy by 0.40% on the intersection-over-union ratio. This fully proves the effectiveness of the RIRNet model proposed in the present invention in post-processing optimization.

[0111] Figure 6This is the qualitative analysis result of the post-processing network model of this embodiment on the CHN6-CUG data set. This embodiment selects the ResUNet model and the CA-HR-RoadNet model as the main qualitative analysis objects. It can be clearly seen from the visualization results that the RIRNet model proposed in the present invention can obtain excellent post-processing effects, can effectively improve the integrity and continuity of the road, maintain the smoothness of the road boundary, and accurately handle the error prediction situations such as missed classification and misclassification. Specifically, the initial road extraction effect of the ResUNet model in the visualization results of the first to fifth rows is relatively poor, most of the roads are not complete, the road boundaries are relatively rough, and there is a large amount of irrelevant noise information, which may be due to the structure and performance of the ResUNet model itself. However, the RIRNet model of the present invention can effectively optimize and process the extraction results of the ResUNet model, so that most of the roads are relatively complete and continuous, the road boundaries are smoother, and the noise information is significantly reduced. This is mainly due to the effectiveness of the road information reasoning module and the road direction reasoning task in the RIRNet model, which can effectively optimize and repair the error prediction. At the same time, in the initial road extraction results of the CA-HR-RoadNet model proposed in this embodiment, there are also some road discontinuities caused by occlusions such as trees and shadows, such as the areas before post-processing in the first to fourth rows of results. The RIRNet model proposed in the present invention can effectively connect broken roads through road information reasoning, achieving better post-processing effects. At the same time, there are road misclassifications in the area before post-processing in the fifth row of results, and the RIRNet model of the present invention can also eliminate erroneous predictions through effective information reasoning.

[0112] Table 4 Efficiency analysis results of some models

[0113]

[0114] Note: The input image size is 512×512×3 when calculating the computational efficiency.

[0115] From the ablation experiment in Table 4, it can be seen that the parameters and operation efficiency of the RIRNet model of the present invention are only 2.85Mb and 42.88Gbps, which are much smaller than other mainstream convolutional neural networks and are lightweight models. Therefore, the RIRNet model is easy to train and reason, and can achieve efficient post-processing optimization.

[0116] The present invention provides a road network optimization processing method that takes direction-connectivity into consideration, which can optimize and repair the prediction results of the road extraction model, improve the accuracy of road extraction and the connectivity of the road network; wherein, the direction-guided road information reasoning module uses the structural and directional characteristics of the road to perform spatial information reasoning on the broken roads from different directions, so that the model can effectively model the spatial relationship between different rows or columns, thereby repairing the broken roads. The post-processing network model simultaneously outputs the prediction results of the road surface and the road direction, and uses the road semantic label information for loss supervision. Because there is a strong correlation between the road direction and the road surface, the two can constrain and promote each other, enhance the reasoning ability of the post-processing network model, and improve the extraction accuracy of the road surface. In addition, the input of the post-processing network model in the present invention includes the initial segmentation result, and the use of the encoder-decoder structure and the jump connection in the post-processing network model will not lead to the loss of narrow roads and the introduction of irrelevant noise information, thereby obtaining a more refined road segmentation map and road direction map.

[0117] It should be noted that the semantic segmentation network model and the post-processing network model in the present invention are both operated based on existing computer platforms.

[0118] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A road network optimization processing method taking into account direction-connectivity, characterized in that: The following steps are involved: S1, obtain remote sensing images and their road semantic label training sets to generate road direction maps; S2, using the road semantic label training set to train the semantic segmentation network model and generate a road segmentation map; S3, constructing a post-processing network model for the segmentation results, and optimizing the road segmentation map; S4, post-processing network model parameter update; S5, post-processing network model update: repeat steps S3 and S4 until the post-processing network model converges; save the trained post-processing network model; S6, road segmentation map optimization: input the road segmentation map and remote sensing image into the post-processing network model trained in step S5 to obtain a refined road segmentation map and road direction map; Step S1 is specifically as follows: Obtain remote sensing images and corresponding road semantic label maps by downloading from the Internet; crop the acquired remote sensing images and corresponding road semantic label maps into image block pairs of the same size with the same step length to form a remote sensing image and its road semantic label training set; generate a road direction map using the road semantic labels; Step S2 is specifically as follows: S2.1, input the remote sensing images and semantic label maps in the road semantic label training set into the semantic segmentation network model; S2.2, semantic segmentation network model update: calculate the error through the loss function, and back-propagate the error, update the parameters of the semantic segmentation network until the semantic segmentation network model converges; save the semantic segmentation network model, that is, obtain the trained semantic segmentation network model; S2.3, input the remote sensing image into the trained semantic segmentation network model to obtain the road segmentation map; In step S3, the post-processing network model includes an encoding part and a decoding part; The encoding part includes a semantic feature extraction module and a road information reasoning module for direction guidance; The semantic feature extraction module is used to extract feature images with different sizes and recognition capabilities. Specifically, the RGB three-channel image of the remote sensing image and the road segmentation map generated by the semantic segmentation network model are spliced ​​and input into the first residual learning unit of the encoder to obtain the feature map output by the first residual learning unit; Downsampling the feature map output by the first residual learning unit by two times, inputting it into the next residual learning unit and outputting the feature map; repeating the steps of downsampling the feature map by two times and inputting it into the next residual learning unit, to obtain feature images with different sizes and recognition capabilities; The direction-guided road information reasoning module is used to obtain a feature image containing road semantic information. Specifically, the feature map output by the last residual learning unit of the encoding part is input into the direction-guided road information reasoning module, and the road semantic information is transferred through spatial relationship convolution reasoning from top to bottom, from bottom to top, from left to right, and from right to left, so as to obtain a feature image containing road semantic information; The decoding part is used to obtain a fine road segmentation map and a road direction map with the same size as the remote sensing image.

2. The road network optimization processing method taking into account direction-connectivity according to claim 1, characterized in that: The semantic feature extraction module is a UNet-like structure, and the UNet-like structure replaces the original convolutional layer in the UNet structure with a residual learning unit.

3. The road network optimization processing method taking into account direction-connectivity according to claim 2, characterized in that: The reasoning process of the road information reasoning module for direction guidance is: a) Input the feature map output by the last residual learning unit of the encoding part into the direction-guided road information reasoning module, split the feature map by row, and obtain multiple rows of independent feature tensors; the split feature map is sequentially passed through two reasoning processes from top to bottom and from bottom to top; The specific calculation formula of the top-down reasoning process is as follows: in, represents the feature tensor of the i-th row after splitting by row, represents the feature tensor of the i-th row after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer; The specific calculation formula of the bottom-up reasoning process is as follows: in, represents the feature tensor of the i-th row after splitting by row, represents the feature tensor of the i-th row after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer; b) Perform convolution operations on each row in turn in the two reasoning processes from top to bottom and from bottom to top, add the convolution result of the previous row to the next row data element by element, obtain the input data for processing the next row, and realize information transmission between different rows; after the convolution operations from top to bottom and from bottom to top, all row tensors are recombined to obtain the combined feature map; c) Split the combined feature map obtained in step b) by columns to obtain multiple columns of independent feature tensors; output a feature map of the spatial relationship between rows and columns of the convolution modeling through two reasoning processes from left to right and from right to left, that is, obtain a feature image containing road semantic information; Specifically, the specific calculation formula of the reasoning process from left to right is as follows: in, represents the feature tensor of the jth column after column segmentation, represents the feature tensor of the jth column after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer; The specific calculation process of the reasoning stage from right to left is as follows: in, represents the feature tensor of the jth column after column segmentation, represents the feature tensor of the jth column after information inference, Conv(.) represents the convolution operation, ReLU(.) represents the nonlinear activation operation, and H represents a positive integer.

4. The road network optimization processing method taking into account direction-connectivity according to claim 3, characterized in that: The decoding process of the decoding part is as follows: splicing the feature map output by the direction-guided road information inference module with the feature output by the third residual learning unit of the encoder through a jump connection to obtain a third splicing feature, and inputting the third splicing feature into the first residual learning unit of the decoder to output a first residual feature; upsampling the first residual feature by two times, and splicing it with the feature output by the second residual learning unit of the encoder through a jump connection to obtain a second splicing feature, and then inputting the second splicing feature into the second residual learning unit of the decoder to output a second residual feature; upsampling the second residual feature by two times, and splicing it with the feature output by the first residual learning unit of the encoder and the road segmentation map generated by the semantic segmentation network model through a jump connection to obtain a first splicing feature, and then inputting the first splicing feature into the third residual learning unit of the decoder to output a third residual feature; Through 1×1 convolution output, a fine road segmentation map and road direction map with the same size as the remote sensing image are obtained.

5. The road network optimization processing method taking into account direction-connectivity according to claim 4, characterized in that: Step S4 is specifically as follows: the error between the refined road segmentation map obtained by the decoding part and the real semantic label map in the road semantic label training set is calculated using the cross entropy loss function; the error between the road direction map obtained by the decoding part and the road direction map generated by the real semantic label map in the road semantic label training set is calculated using the Dice loss function; the two errors are back-propagated to update the post-processing network model parameters.

6. The road network optimization processing method taking into account direction-connectivity according to claim 5, characterized in that: The cross entropy loss function is: Among them, N is the set of image pixels, a is the a-th pixel, and y represents the true label value. Represents the predicted label value.

7. The road network optimization processing method taking into account direction-connectivity according to claim 5, characterized in that: The Dice loss function is: Among them, X represents the pixel category of the model predicted image, Y represents the pixel label of the real image, |X∩Y| is the intersection between X and Y, which is approximately the dot product between the predicted image and the real image, and the element results of the dot product are added together. |X| represents the addition of the positive example elements in X, and |Y| represents the addition of the positive example elements in Y.

Images