A connectivity enhanced road extraction method, system, terminal and medium

Abstract

The application discloses a connectivity-enhanced road extraction method and relates to the field of remote sensing image analysis, and comprises the following steps: acquiring original remote sensing images and corresponding road SHP annotation files; constructing a road extraction model, wherein a C2G module is used to replace a standard residual module in an hourglass model, an attention mechanism is added to the hourglass model through an intermediate processing module, information learning of features in horizontal and vertical directions is enhanced, road features are output, a road segmentation module is used to extract deep road features during model training, road segmentation training and connectivity prediction training are performed respectively, a multi-layer segmented road result and connectivity probability prediction are output, and a decoder outputs a segmentation mask of a road region and judges the connectivity of the road; the road extraction model is trained by using a training set, a test set is input into the road extraction model for road extraction, and a road segmentation result and a connectivity prediction result are output. The method can accurately perform road segmentation and road connectivity prediction.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image analysis technology, specifically to a method, system, terminal, and medium for enhancing road connectivity extraction. Background Technology

[0002] Road extraction is a crucial task in remote sensing image analysis, widely applied in urban planning, traffic monitoring, disaster assessment, and map updating. With the rapid development of remote sensing and computer vision technologies, road extraction using satellite remote sensing imagery, UAV imagery, and high-resolution remote sensing data has become a research hotspot. However, due to the complexity and diversity of roads in different geographical environments, especially complex urban roads, rural roads, and narrow roads, traditional road extraction methods face problems such as low accuracy, poor adaptability, and significant interference from complex backgrounds. Existing road extraction schemes can be mainly categorized as follows:

[0003] Road extraction schemes based on traditional image processing methods: Traditional road extraction methods often rely on techniques such as image edge detection, grayscale analysis, and texture feature extraction. For example, image segmentation and edge detection methods (such as Canny edge detection) are often used for road extraction.

[0004] Road extraction schemes based on deep learning semantic segmentation: Publication No. CN117173557A proposes a scheme that combines an improved U-Net structure with a global contextual attention mechanism to enhance road extraction in rural areas; Publication No. CN117351360A introduces recurrent cross-attention and convolutional attention modules into the road extraction network to enhance the extraction of local information and achieve feature enhancement for road extraction; Publication No. CN117636154A proposes a scheme that uses edge detection and semantic segmentation to complement each other to enrich road edge information and enhance road connectivity.

[0005] In the published patent literature, methods based on traditional image processing schemes are not suitable for large-scale processing and have low applicability. There will also be large errors in the transfer of schemes between images at the same time. At the same time, these methods are easily affected by complex backgrounds, noise, shadows and other factors, resulting in inaccurate extraction results. Summary of the Invention

[0006] The purpose of this invention is to provide a road extraction method, system, terminal, and medium with enhanced connectivity, which can accurately perform road segmentation and road connectivity prediction.

[0007] This invention is achieved through the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide a connectivity-enhanced road extraction method, comprising:

[0009] Acquire the original remote sensing images and the corresponding road SHP annotation files, convert the road SHP annotation files into image files, align them with the original remote sensing images, and create the training and testing sets for the model.

[0010] A road extraction model is constructed, comprising an encoder, an intermediate processing module, an auxiliary road segmentation module, a decoder, and a segmentation module. The encoder replaces the standard residual module in the hourglass model with a C2G module to extract shallow features and gradient flow information from the input remote sensing image. The intermediate processing module adds an attention mechanism to the hourglass model to enhance the learning of information in the horizontal and vertical directions of features and outputs road features. The auxiliary road segmentation module is used to extract deep road features during model training, and performs road segmentation training and connectivity prediction training respectively, outputting multi-level segmented road results and connectivity probability predictions. The decoder maps the abstract road features back to a spatial dimension that matches the resolution of the input remote sensing image, outputting a segmentation mask of the road region and a judgment on the connectivity of the road.

[0011] The road extraction model is trained using the training set, and the test set is input into the road extraction model to extract roads. The outputs are road segmentation results and connectivity prediction results.

[0012] Furthermore, after the steps of outputting road segmentation results and connectivity prediction results, a post-processing strategy is also included, the specific methods of which include:

[0013] Extract the centerline from the road segmentation results, and obtain the road's degree based on the centerline;

[0014] Roads are segmented based on points with a degree greater than 2, and roads with a length less than a preset value are eliminated;

[0015] The width of the road is calculated based on the road segmentation results output by the centerline and the road extraction model. The width of the same road is calculated as the average of the widths at different points, and then the road is filled to the specified width using the dilation method.

[0016] Furthermore, the specific methods for the C2G module to process data include:

[0017] The input data is split into multiple sub-data;

[0018] Feature extraction and processing are performed on multiple sub-data sets separately to obtain the processing results;

[0019] Multiple processing results are concatenated and multi-branch features are integrated;

[0020] Convolutional operations are performed on multi-branch features to extract shallow features and gradient flow information from remote sensing images.

[0021] Furthermore, specific methods for processing data using attention mechanisms include:

[0022] A linear layer is used for inter-channel information exchange, and a normalized activation function is used to discretize channel information to distinguish strong foreground information and weak background information in remote sensing images.

[0023] Spatial attention in both row and column directions is used to compress and extract features in the horizontal direction, resulting in road horizontally compressed features.

[0024] After performing max pooling and average pooling on the road horizontally compressed features, the maximum and average features in the horizontal direction of the road are extracted.

[0025] The features are compressed and extracted in the vertical direction to obtain the vertically compressed features of the road.

[0026] After performing max pooling and average pooling on the vertically compressed road features, the maximum and average features in the vertical direction of the road are extracted.

[0027] The maximum and average features of the road in the horizontal and vertical directions are spliced ​​together to integrate features from different directions and expand the road's connectivity.

[0028] Secondly, another embodiment of the present invention provides a connectivity-enhanced road extraction system, comprising: a data acquisition module, a model building module, and a model prediction module.

[0029] The data acquisition module is used to acquire the original remote sensing images and the corresponding road SHP annotation files, convert the road SHP annotation files into image files, align them with the original remote sensing images, and create training and testing sets for the model.

[0030] The model building module is used to construct a road extraction model, which includes an encoder, an intermediate processing module, an auxiliary road segmentation module, a decoder, and a segmentation module. The encoder replaces the standard residual module in the hourglass model with a C2G module to extract shallow features and gradient flow information from the input remote sensing image. The intermediate processing module adds an attention mechanism to the hourglass model to enhance the information learning of features in the horizontal and vertical directions and outputs road features. The auxiliary road segmentation module is used to extract deep road features during model training, and performs road segmentation and connectivity prediction training respectively, outputting multi-level segmentation road results and connectivity probability predictions. The decoder maps the abstract road features back to a spatial dimension that matches the resolution of the input remote sensing image, outputting a segmentation mask of the road region and a judgment on the connectivity of the road.

[0031] The model prediction module trains the road extraction model using the training set and inputs the test set into the road extraction model to extract roads, outputting road segmentation results and connectivity prediction results.

[0032] Furthermore, it also includes a post-processing module, which extracts the centerline from the road segmentation result, obtains the degree of the road based on the centerline, segments the road based on points with a degree greater than 2, eliminates roads with a length less than a preset value, calculates the width of the road based on the centerline and the road segmentation result output by the road extraction model, calculates the average width of the same road based on the sum of the widths of different points, and then uses an expansion method to fill the road to the specified width.

[0033] Furthermore, the C2G module includes: a segmentation unit, a basic unit, a stitching unit, and a convolutional unit. There are multiple basic units. The segmentation unit segments the input data to obtain multiple sub-data. The segmented sub-data is then input into the basic unit. The basic unit performs feature extraction and processing on the sub-data respectively to obtain processing results. The stitching unit stitches together the multiple processing results and fuses the multi-branch features. The convolutional unit performs convolution operations on the multi-branch features to extract shallow features and gradient flow information from the remote sensing image.

[0034] Furthermore, the model building module includes an attention mechanism unit, which uses a linear layer for inter-channel information interaction and a normalized activation function to discretize channel information, distinguishing strong foreground information and weak background information in remote sensing images;

[0035] Spatial attention in both row and column directions is used to compress and extract features in the horizontal direction, resulting in road horizontally compressed features.

[0036] After performing max pooling and average pooling on the road horizontally compressed features, the maximum and average features in the horizontal direction of the road are extracted.

[0037] The features are compressed and extracted in the vertical direction to obtain the vertically compressed features of the road.

[0038] After performing max pooling and average pooling on the vertically compressed road features, the maximum and average features in the vertical direction of the road are extracted.

[0039] The maximum and average features of the road in the horizontal and vertical directions are spliced ​​together to integrate features from different directions and expand the road's connectivity.

[0040] Thirdly, another embodiment of the present invention provides an electronic device comprising: a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to invoke the program instructions to execute the method described in the first embodiment.

[0041] Fourthly, another embodiment of the present invention provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in the first embodiment.

[0042] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0043] This invention provides a connectivity-enhanced road extraction method, system, terminal, and medium. In the encoder, a C2G module replaces the standard residual module in the hourglass model, enabling better extraction of rich shallow features and gradient flow information. Rich shallow features help the road extraction model better capture important information within them, while rich gradient flow information accelerates model convergence and improves training efficiency. In the intermediate processing module, a C2G module combined with an attention mechanism enhances feature learning in both horizontal and vertical dimensions, ensuring road connectivity. The auxiliary road segmentation module and decoder employ multi-task learning to improve deep feature extraction, ensuring that important information is amplified and background information is weakened during upsampling. This method can accurately segment roads and predict road connectivity. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0045] Figure 1 A flowchart illustrating a connectivity enhancement road extraction method provided in the first embodiment of the present invention;

[0046] Figure 2 This is a schematic diagram of the road extraction model.

[0047] Figure 3 This is a schematic diagram of the C2G module.

[0048] Figure 4A schematic diagram of the feature attention mechanism for connectivity learning;

[0049] Figure 5 The following diagram illustrates the effect of the method provided in this embodiment of the invention on the DeepGlobe dataset;

[0050] Figure 6 The method provided in this embodiment of the invention is illustrated in the image of private satellite data.

[0051] Figure 7 This is a structural block diagram of a road extraction system with enhanced connectivity provided in another embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0053] like Figure 1 As shown in the figure, an embodiment of the present invention provides a road extraction method for enhancing connectivity, comprising:

[0054] Acquire the original remote sensing images and the corresponding road SHP annotation files, convert the road SHP annotation files into image files, align them with the original remote sensing images, and create the training and testing sets for the model.

[0055] A road extraction model is constructed, comprising an encoder, an intermediate processing module, an auxiliary road segmentation module, a decoder, and a segmentation module. The encoder replaces the standard residual module in the hourglass model with a C2G module to extract shallow features and gradient flow information from the input remote sensing image. The intermediate processing module adds an attention mechanism to the hourglass model to enhance the learning of information in the horizontal and vertical directions of features and outputs road features. The auxiliary road segmentation module is used to extract deep road features during model training, and performs road segmentation training and connectivity prediction training respectively, outputting multi-level segmented road results and connectivity probability predictions. The decoder maps the abstract road features back to a spatial dimension that matches the resolution of the input remote sensing image, outputting a segmentation mask of the road region and a judgment on the connectivity of the road.

[0056] The road extraction model is trained using the training set, and the test set is input into the road extraction model to extract roads. The outputs are road segmentation results and connectivity prediction results.

[0057] In this embodiment, the dataset creation specifically includes: acquiring high-resolution remote sensing images (0.3-1m spatial resolution) and corresponding road SHP annotation files; converting the SHP annotation files into image files and aligning them with the original images; then using a sliding window to cut the data into 512×512 pixel sample blocks, retaining 20% ​​overlap to avoid loss of edge information. The dataset is divided into training and test sets in a 9:1 ratio.

[0058] The road extraction model includes an encoder, an intermediate processing module, an auxiliary road segmentation module, a decoder, and a segmentation module. The encoder replaces the standard residual module used in the hourglass model with a C2G module. Compared to the standard residual module, the C2G module can better extract richer feature extraction and gradient flow information, sparsely stacking shallow features, which helps the model better capture important information in shallow features. At the same time, the rich gradient flow information can accelerate model convergence speed and improve training efficiency. The calculation formula for the C2G module is as follows:

[0059] (1) (2) (3)

[0060] In formula (1), `chunk` means splitting the feature into two parts according to the channel dimension and storing them in a list in feature F. In formula (2), the `extend` function means inserting a new feature at the end of list F. Formula (3) means stacking all the features and then performing a convolutional layer to compress the channels, while simultaneously achieving information interaction between multiple shallow features. In the above formulas, `x` represents the feature learned in the previous layer, `y` represents the feature list, `F` represents the updated feature, and `BasicBlockList` represents a list of `BasicBlock`s with a quantity of M, where M is an integer greater than or equal to 1.

[0061] The specific methods by which the C2G module processes data include:

[0062] The input data is split into multiple sub-data by using the Split operation;

[0063] Multiple sub-data are input into several BasicBlocks for feature extraction and processing to obtain the processing results. The BasicBlock is composed of components such as convolutional layers, such as two convolutional (Conv) layers (some BasicBlocks also involve the summation of convolutional results, i.e. residual connections).

[0064] Multiple processing results are concatenated using the Concat operation to fuse multi-branch features;

[0065] Convolutional operations are performed on multi-branch features to extract shallow features and gradient flow information from remote sensing images.

[0066] The function of the C2G module:

[0067] Multi-scale feature extraction: Through a series of operations such as convolution, segmentation, and stitching, the C2G module can extract features from input data at different scales and dimensions. For example, in road segmentation tasks, it can capture local details of the road (such as road surface texture and road markings) as well as global features such as the overall direction and shape of the road, which is crucial for accurately identifying road regions.

[0068] Feature fusion and propagation: The Split and Concat operations enable the splitting and fusion of features across different paths, allowing features of different types and levels to complement and propagate each other. In road connectivity prediction tasks, this feature fusion mechanism helps the network better understand the connectivity relationships between roads and make accurate judgments by comprehensively considering the feature information of different road segments.

[0069] Enhanced model expressive power: The combination of multiple BasicBlocks and the orderly arrangement of different operations endow the C2G module with stronger feature transformation and expressive power, enabling it to process complex image information and adapt to various situations that may occur in road scenes, such as different lighting conditions and complex environments around the road.

[0070] Due to the shading of trees or buildings, road extraction suffers from poor connectivity. To enhance road connectivity, an attention mechanism is introduced into the hourglass network, such as... Figure 4 As shown, the intermediate processing module combines the C2G module and an attention mechanism to enhance feature learning in both the H and W dimensions, ensuring road connectivity. The C2G module is used for feature transfer and transformation between different scales and channels; the attention mechanism allows the network to focus on road-related regions or features (such as road shape and direction of extension), improving the ability to capture key road information and ultimately enhancing the network's performance in road segmentation and connectivity prediction tasks. Multiple such intermediate processing modules are cascaded to progressively refine the features output by the encoder, continuously optimizing feature representations and providing refined features for subsequent road segmentation and connectivity prediction tasks.

[0071] The overall attention mechanism is mainly divided into two parts. The first part is a simple channel attention, which realizes information interaction between channels through a linear layer and then uses a normalized activation function to discretize the channel information, thereby accelerating the differentiation between strong foreground information and weak background information. The second part adopts spatial attention in the row and column directions. By learning the features of rows and columns respectively, it enhances the information interaction between roads in different directions. Finally, the information matrices of the two directions are fused to expand the overall road connectivity.

[0072] Specifically, the specific methods by which attention mechanisms process data include:

[0073] A linear layer is used for inter-channel information exchange, and a normalized activation function is used to discretize channel information to distinguish strong foreground information and weak background information in remote sensing images.

[0074] Spatial attention in both row and column directions is used to compress and extract features in the horizontal direction, resulting in road horizontally compressed features.

[0075] After performing max pooling and average pooling on the road horizontally compressed features, the maximum and average features in the horizontal direction of the road are extracted.

[0076] The features are compressed and extracted in the vertical direction to obtain the vertically compressed features of the road.

[0077] After performing max pooling and average pooling on the vertically compressed road features, the maximum and average features in the vertical direction of the road are extracted.

[0078] The maximum and average features of the road in the horizontal and vertical directions are spliced ​​together to integrate features from different directions and expand the road's connectivity.

[0079] During model training, the auxiliary road segmentation module is used to extract deep road features. After the intermediate processing module extracts road features from different directions, road segmentation and connectivity prediction are performed. This is a multi-task assisted training, where the task is predicted in the middle stage of the network. On the one hand, it can use intermediate features to learn task-related patterns in advance, alleviating problems such as gradient vanishing in deep networks. On the other hand, it can leverage the advantages of multi-task learning to improve the overall task performance.

[0080] The road extraction model involves three tasks: road detection and segmentation, connectivity segmentation, and auxiliary segmentation at different layers. The main task is road detection and segmentation. Connectivity segmentation mainly guides the recognition performance of different road directions. Auxiliary segmentation helps improve the accuracy of deep features and ensures that important information is amplified and background information is weakened during the upsampling process.

[0081] When performing road segmentation, the road segmentation loss is adopted as the cross-entropy loss. and Dice loss It is composed of several components; among them, the Dice loss can effectively handle the problem of unbalanced foreground and background pixel ratios. The calculation formula for the road segmentation loss function is as follows:

[0082] (4)

[0083] (5)

[0084] In cross-entropy loss, N is the total number of pixels, and y i ∈{0,1} represents the true label of the i-th pixel (0 for background, 1 for road), p i To predict the probability that this pixel belongs to a road for the model; w yi and w 1−yi These are the weight coefficients for roads and backgrounds, respectively (e.g., w1≫w0), which significantly alleviate the class imbalance problem caused by sparse road pixels; in the Dice loss, x i,k and y i,k Let represent the one-hot encoding of the predicted probability and the true label (k∈{0,1}), respectively. In equation (5), ϵ in the denominator is a smoothing term to prevent division by zero.

[0085] During connectivity segmentation, cross-entropy loss is used for connectivity loss. Roads are decomposed into 37 categories based on their angle range. Overall connectivity is constructed by predicting the angle of each point. Simultaneously, feature enhancement methods in different directions are used to improve the globality of the feature search space and optimize road connectivity. Connectivity loss function. The calculation formula is as follows:

[0086] (6)

[0087] In the connectivity cross-entropy loss, N is the total number of pixels, D is the number of angle categories, and y d Represents the real label, with a value of y. d ∈{0, 36}, This represents the predicted probability.

[0088] Following the steps of outputting road segmentation results and connectivity prediction results, a post-processing strategy is also included, the specific methods of which include:

[0089] The centerline is extracted from the road segmentation results, and the degree of the road is obtained based on the centerline; where a degree of 1 represents an endpoint, a degree of 2 represents a connected road, and a degree greater than 2 represents an intersection.

[0090] Roads are segmented based on points with a degree greater than 2, and roads with a length less than a preset value are eliminated. Because there may be burrs when obtaining the road centerline, the preset value of the elimination length is determined based on the actual burr length.

[0091] The width of the road is calculated based on the road segmentation results output by the centerline and the road extraction model. The width of the same road is calculated as the average of the widths at different points, and then the road is filled to the specified width using the dilation method.

[0092] Table 1 shows the results of this method compared with several different experiments on the DeepGlobe dataset. Road IoU represents the union ratio of the detected road region and the label region. The higher the ratio, the better the segmentation effect. F1 represents the harmonic mean of precision and recall. The higher the score, the higher the recall and precision. APLS represents the road connectivity. The higher the value, the better the connectivity.

[0093] Table 1. Results of multiple experiments on the DeepGlobe dataset.

[0094]

[0095] The connectivity-enhanced road extraction method provided in this embodiment of the invention demonstrates its effectiveness on the DeepGlobe dataset as shown below. Figure 5 As shown. To verify the portability of this method, the effect on private satellite data is as follows. Figure 6 As shown.

[0096] This invention provides a connectivity-enhanced road extraction method. In the encoder, a C2G module replaces the standard residual module in the hourglass model, enabling richer extraction of shallow features and gradient flow information. Richer shallow features help the road extraction model better capture important information within them, while richer gradient flow information accelerates model convergence and improves training efficiency. In the intermediate processing module, a C2G module combined with an attention mechanism enhances feature learning in both horizontal and vertical dimensions, ensuring road connectivity. The auxiliary road segmentation module and decoder employ multi-task learning to improve deep feature extraction, ensuring that important information is amplified and background information is reduced during upsampling. The constructed road extraction model can accurately perform road segmentation and road connectivity prediction.

[0097] like Figure 7 As shown, another embodiment of the present invention provides a road extraction system for enhanced connectivity, comprising: a data acquisition module, a model building module, and a model prediction module.

[0098] The data acquisition module is used to acquire the original remote sensing images and the corresponding road SHP annotation files, convert the road SHP annotation files into image files, align them with the original remote sensing images, and create training and testing sets for the model.

[0099] The model building module is used to construct a road extraction model, which includes an encoder, an intermediate processing module, an auxiliary road segmentation module, a decoder, and a segmentation module. The encoder replaces the standard residual module in the hourglass model with a C2G module to extract shallow features and gradient flow information from the input remote sensing image. The intermediate processing module adds an attention mechanism to the hourglass model to enhance the information learning of features in the horizontal and vertical directions and outputs road features. The auxiliary road segmentation module is used to extract deep road features during model training, and performs road segmentation and connectivity prediction training respectively, outputting multi-level segmentation road results and connectivity probability predictions. The decoder maps the abstract road features back to a spatial dimension that matches the resolution of the input remote sensing image, outputting a segmentation mask of the road region and a judgment on the connectivity of the road.

[0100] The model prediction module trains the road extraction model using the training set and inputs the test set into the road extraction model to extract roads, outputting road segmentation results and connectivity prediction results.

[0101] The connectivity-enhancing road extraction system provided in this embodiment of the invention also includes a post-processing module. The post-processing module extracts the centerline from the road segmentation results, obtains the degree of the road based on the centerline, segments the road based on points with a degree greater than 2, eliminates roads with a length less than a preset value, calculates the width of the road based on the centerline and the road segmentation results output by the road extraction model, calculates the average width of the same road based on the sum of the widths of different points, and then uses an expansion method to fill the road to the specified width.

[0102] The C2G module includes a segmentation unit, a basic unit, a stitching unit, and a convolutional unit. There are multiple basic units. The segmentation unit divides the input data into multiple sub-data, and inputs the segmented sub-data into the basic units. The basic units perform feature extraction and processing on the sub-data respectively to obtain processing results. The stitching unit stitches the multiple processing results and fuses the multi-branch features. The convolutional unit performs convolution operations on the multi-branch features to extract shallow features and gradient flow information in the remote sensing image.

[0103] The model building module includes an attention mechanism unit, which uses a linear layer to interact with information between channels and uses a normalized activation function to discretize channel information, distinguishing strong foreground information and weak background information in remote sensing images.

[0104] Spatial attention in both row and column directions is used to compress and extract features in the horizontal direction, resulting in road horizontally compressed features.

[0105] After performing max pooling and average pooling on the road horizontally compressed features, the maximum and average features in the horizontal direction of the road are extracted.

[0106] The features are compressed and extracted in the vertical direction to obtain the vertically compressed features of the road.

[0107] After performing max pooling and average pooling on the vertically compressed road features, the maximum and average features in the vertical direction of the road are extracted.

[0108] The maximum and average features of the road in the horizontal and vertical directions are spliced ​​together to integrate features from different directions and expand the road's connectivity.

[0109] This invention provides a connectivity-enhanced road extraction system. In the encoder, a C2G module replaces the standard residual module in the hourglass model, enabling richer extraction of shallow features and gradient flow information. Richer shallow features help the road extraction model better capture important information within them, while richer gradient flow information accelerates model convergence and improves training efficiency. In the intermediate processing module, a C2G module combined with an attention mechanism enhances feature learning in both horizontal and vertical dimensions, ensuring road connectivity. The auxiliary road segmentation module and decoder employ multi-task learning to improve deep feature extraction, ensuring that important information is amplified and background information is reduced during upsampling. The constructed road extraction model can accurately perform road segmentation and road connectivity prediction.

[0110] Another embodiment of the present invention provides a smart terminal, which includes a processor, an input device, an output device, and a memory. The processor, the input device, the output device, and the memory are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to call the program instructions to execute the method described in the first embodiment above.

[0111] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0112] Input devices may include touchpads, microphones, etc., and output devices may include displays (LCDs, etc.), speakers, etc.

[0113] The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store information about the device type.

[0114] In specific implementations, the processor, input device, and output device described in the embodiments of the present invention can execute the implementation of the method embodiments described in the embodiments of the present invention, or they can execute the implementation of the system embodiments described in the embodiments of the present invention, which will not be repeated here.

[0115] The present invention also provides an embodiment of a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the method described in the first embodiment above.

[0116] The computer-readable storage medium can be an internal storage unit of the device described in the foregoing embodiments, such as the device's hard drive or memory. The computer-readable storage medium can also be an external storage device of the terminal, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the terminal. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0117] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0118] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the terminals and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A road extraction method with enhanced connectivity, characterized in that, include: Acquire the original remote sensing images and the corresponding road SHP annotation files, convert the road SHP annotation files into image files, align them with the original remote sensing images, and create the training and testing sets for the model. A road extraction model is constructed, comprising an encoder, an intermediate processing module, an auxiliary road segmentation module, a decoder, and a segmentation module. The encoder replaces the standard residual module in the hourglass model with a C2G module to extract shallow features and gradient flow information from the input remote sensing image. The intermediate processing module adds an attention mechanism to the hourglass model to enhance the learning of information in the horizontal and vertical directions of features and outputs road features. The auxiliary road segmentation module is used to extract deep road features during model training, and performs road segmentation training and connectivity prediction training respectively, outputting multi-level segmented road results and connectivity probability predictions. The decoder maps abstract road features back to a spatial dimension that matches the resolution of the input remote sensing image. The segmentation module outputs a segmentation mask of the road region and determines the connectivity of the road. The road extraction model is trained using the training set, and the test set is input into the road extraction model to extract roads. The outputs are road segmentation results and connectivity prediction results.

2. The connectivity-enhanced road extraction method according to claim 1, characterized in that, Following the steps of outputting road segmentation results and connectivity prediction results, a post-processing strategy is also included, the specific methods of which include: Extract the centerline from the road segmentation results, and obtain the road's degree based on the centerline; Roads are segmented based on points with a degree greater than 2, and roads with a length less than a preset value are eliminated; The width of the road is calculated based on the road segmentation results output by the centerline and the road extraction model. The width of the same road is calculated as the average of the widths at different points, and then the road is filled to the specified width using the dilation method.

3. The connectivity-enhanced road extraction method according to claim 1 or 2, characterized in that, The specific methods by which the C2G module processes data include: The input data is split into multiple sub-data; Feature extraction and processing are performed on multiple sub-data sets separately to obtain the processing results; Multiple processing results are concatenated and multi-branch features are integrated; Convolutional operations are performed on multi-branch features to extract shallow features and gradient flow information from remote sensing images.

4. The connectivity-enhanced road extraction method according to claim 1 or 2, characterized in that, Specific methods for processing data using attention mechanisms include: A linear layer is used for inter-channel information exchange, and a normalized activation function is used to discretize channel information to distinguish strong foreground information and weak background information in remote sensing images. Spatial attention in both row and column directions is used to compress and extract features in the horizontal direction, resulting in road horizontally compressed features. After performing max pooling and average pooling on the road horizontally compressed features, the maximum and average features in the horizontal direction of the road are extracted. The features are compressed and extracted in the vertical direction to obtain the vertically compressed features of the road. After performing max pooling and average pooling on the vertically compressed road features, the maximum and average features in the vertical direction of the road are extracted. The maximum and average features of the road in the horizontal and vertical directions are spliced ​​together to integrate features from different directions and expand the road's connectivity.

5. A road extraction system with enhanced connectivity, characterized in that, include: Data acquisition module, model building module, and model prediction module. The data acquisition module is used to acquire the original remote sensing images and the corresponding road SHP annotation files, convert the road SHP annotation files into image files, align them with the original remote sensing images, and create training and testing sets for the model. The model building module is used to construct a road extraction model, which includes an encoder, an intermediate processing module, an auxiliary road segmentation module, a decoder, and a segmentation module. The encoder replaces the standard residual module in the hourglass model with a C2G module to extract shallow features and gradient flow information from the input remote sensing image. The intermediate processing module adds an attention mechanism to the hourglass model to enhance the information learning of features in the horizontal and vertical directions and outputs road features. The auxiliary road segmentation module is used to extract deep road features during model training, and performs road segmentation and connectivity prediction training respectively, outputting multi-level segmentation road results and connectivity probability predictions. The decoder maps the abstract road features back to a spatial dimension that matches the resolution of the input remote sensing image, outputting a segmentation mask of the road region and a judgment on the connectivity of the road. The model prediction module trains the road extraction model using the training set and inputs the test set into the road extraction model to extract roads, outputting road segmentation results and connectivity prediction results.

6. The connectivity-enhanced road extraction system according to claim 5, characterized in that, It also includes a post-processing module, which extracts the centerline from the road segmentation results, obtains the degree of the road based on the centerline, segments the road based on points with a degree greater than 2, eliminates roads with a length less than a preset value, calculates the width of the road based on the centerline and the road segmentation results output by the road extraction model, calculates the average width of the same road based on the sum of the widths of different points, and then uses an expansion method to fill the road to the specified width.

7. The connectivity-enhanced road extraction system according to claim 5 or 6, characterized in that, The C2G module includes a segmentation unit, a basic unit, a stitching unit, and a convolution unit. There are multiple basic units. The segmentation unit divides the input data into multiple sub-data, and inputs the segmented sub-data into the basic units. The basic units perform feature extraction and processing on the sub-data respectively to obtain processing results. The stitching unit stitches the multiple processing results and fuses the multi-branch features. The convolution unit performs convolution operations on the multi-branch features to extract shallow features and gradient flow information in the remote sensing image.

8. The connectivity-enhanced road extraction system according to claim 5 or 6, characterized in that, The model building module includes an attention mechanism unit, which uses a linear layer to interact with information between channels and uses a normalized activation function to discretize channel information, distinguishing strong foreground information and weak background information in remote sensing images. Spatial attention in both row and column directions is used to compress and extract features in the horizontal direction, resulting in road horizontally compressed features. After performing max pooling and average pooling on the road horizontally compressed features, the maximum and average features in the horizontal direction of the road are extracted. The features are compressed and extracted in the vertical direction to obtain the vertically compressed features of the road. After performing max pooling and average pooling on the vertically compressed road features, the maximum and average features in the vertical direction of the road are extracted. The maximum and average features of the road in the horizontal and vertical directions are spliced ​​together to integrate features from different directions and expand the road's connectivity.

9. A smart terminal, comprising: The system includes a processor, an input device, an output device, and a memory, which are interconnected. The memory is used to store a computer program, which includes program instructions. The processor is configured to invoke the program instructions to execute the method as described in any one of claims 1-4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-4.

Images