A remote sensing image road extraction method, system, device and medium based on large kernel convolution and direction strip convolution

Abstract

A remote sensing image road extraction method, system, device and medium based on large kernel convolution and direction strip convolution, the method is: the Deepglobe data set is divided into training set and test set;Data enhancement is carried out on the training set;The remote sensing image road extraction network model based on large kernel convolution and direction strip convolution is constructed;BCE Loss and Dice Loss are used to construct the loss function for training the remote sensing image road extraction network model based on large kernel convolution and direction strip convolution;The remote sensing image road extraction network model based on large kernel convolution and direction strip convolution is trained using the training set after data enhancement and the loss function, and the optimal model is obtained;The test set is predicted using the optimal model, and the prediction result is obtained;The prediction result is evaluated;The system, device and medium are used to realize the method;The present application has the advantages of less missed detection and false detection, high connectivity and accuracy, strong updating and iteration ability and the like.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to a method, system, device, and medium for road extraction from remote sensing images based on large kernel convolution and directional strip convolution. Background Technology

[0002] Road networks are essential infrastructure for transportation and economic development, and a crucial component of geographic information systems (GIS). They play a vital role in urban planning, intelligent transportation, map updates, and many other fields, especially in navigation and disaster emergency response, where timely and accurate road information is paramount. In the era of information technology and big data, understanding the overall road network, fully utilizing road resources, quickly filling gaps in road information, and updating road information promptly are increasingly important. Using manual offline surveying methods would consume significant human, material, and financial resources, and timeliness could not be guaranteed. Remote sensing imagery contains rich information about geographical features and possesses characteristics such as macroscopic scope, real-time data, and massive volume, providing a new avenue for timely acquisition of road information and offering substantial data support for road extraction.

[0003] With continuous breakthroughs in artificial intelligence in computer vision, deep learning-based methods are increasingly being applied to road extraction and design. These methods can automatically learn basic road features from massive amounts of data, extracting road information more automatically and accurately. However, a significant amount of manual annotation is still required in the early stages to assist learning, and their efficiency and generalization performance when dealing with road data from different regions are insufficient to meet practical needs and applications. Further comprehensive research into how to extract road information more accurately and completely from high-resolution remote sensing imagery, thereby applying more accurate road information to fields such as emergency response and map updates, has profound research value and practical significance.

[0004] Traditional road extraction methods typically rely on the geometric, textural, and spectral features of roads, depending on human experience to design method features and parameters. However, road features in remote sensing imagery are complex and easily change with seasons and shooting angles. Human experience-based design struggles to fully consider the complexity of road features and background. Therefore, traditional road extraction methods are better suited for simple road scenes and fail to achieve high accuracy in more complex road scenarios, presenting new challenges.

[0005] With the continuous breakthroughs in artificial intelligence technology in the field of computer vision, deep learning-based methods are gradually being applied to road extraction design. They can automatically learn the basic features of roads from massive amounts of data, and extract road information more automatically and accurately. U-Net (onneberger O, Fischer P, Brox TU-net: Convolutional networks for biomedical image segmentation[C] / / Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015:18th International Conference,Munich,Germany,October 5-9,2015,Proceedings,Part III 18.Springer InternationalPublishing,2015:234-241.) network integrates pixel-level features with high-level semantic features in a skip connection manner. The model is simple and performs well in various segmentation tasks. Some scholars (Li J,Liu Y,Zhang Y,et al.Cascadedattention DenseUNet(CADUNet)for road extraction from very-high-resolutionimages[J].ISPRS International Journal of Geo-Information, 2021, 10(5): 329.) In the DenseUNet framework, a set of cascaded global attention modules are introduced to extract the contextual information of roads. Secondly, a set of cascaded core attention modules are embedded to ensure that road information is fully transmitted between dense blocks, further assisting the global attention modules in obtaining multi-scale features, thereby improving the connectivity of the road network.The DLinkNet network (Zhou L, Zhang C, Wu MD-LinkNet: LinkNet with pretrained encoder and dilated convolution for high-resolution satellite imagery road extraction[C] / / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.2018:182-186.) employs an encoder-decoder structure, using dilated convolution in the middle part to expand the receptive field and capture multi-scale features. The Connectivity Attention Network (CoANet) (Mei J, Li RJ, Gao W, et al. CoANet: Connectivityattention network for road extraction from satellite imagery[J].IEEE Transactions on Image Processing,2021,30:8540-8552.) can learn pairwise dependencies. This model uses strip convolution in the decoder to consider road directionality and calculates connectivity loss by generating connectivity cubes, thereby enhancing the network's learning of road connectivity. This method can effectively preserve road connectivity while maintaining high accuracy. DDU-Net (Wang Y, Peng Y, Liu X, et al. DDU-Net: Dual-Decoder-U-Net for Road Extraction Using High-Resolution Remote Sensing Images[J]. 2022. DOI:10.48550 / arXiv.2201.06750.) builds upon U-Net by designing a dual-decoder structure to obtain more road information. A dilated convolutional attention module is introduced between the encoder and decoder to increase the receptive field and extract multi-scale features, thereby improving perception capabilities. These methods have achieved state-of-the-art results on publicly available remote sensing road datasets. However, most semantic segmentation networks consider relatively little about the inherent extension and connectivity features of roads. Currently, the extracted roads still suffer from connectivity and integrity issues that urgently need to be addressed.

[0006] The existing technology has the following drawbacks:

[0007] 1) Existing technologies have poor perception of fine road targets;

[0008] 2) Existing technologies do not pay enough attention to the inherent characteristics of roads, such as their slenderness and meandering nature;

[0009] 3) Existing technologies suffer from issues such as missed detections and false detections, poor connectivity, inadequate road extraction performance, and insufficient accuracy.

[0010] Existing deep learning-based remote sensing image road extraction techniques typically employ conventionally sized convolutional kernels (3×3 kernels) for road feature extraction. However, roads in remote sensing images are often elongated, extending from one end of the image to the other, with road networks covering the entire image. Such small convolutional kernels have limited receptive fields, making it difficult to effectively learn the long-distance relationships between roads in a single convolutional operation. Furthermore, roads in images tend to meander in all directions rather than following regular horizontal and vertical lines, making it difficult for ordinary square convolutional kernels to accurately learn the directional characteristics of roads. Therefore, existing road extraction techniques still have shortcomings in terms of road feature extraction, and the quality of road feature extraction directly impacts the final road extraction result. Summary of the Invention

[0011] To overcome the shortcomings of the prior art, the present invention aims to provide a method, system, device, and medium for road extraction from remote sensing images based on large kernel convolution and directional strip convolution. It incorporates a series of improvements to the encoder-decoder-based architecture, including a large kernel convolution feature encoding module (LKSMP Encoder) with a large kernel depthwise separable convolution combined with SMP operations as its core. This module learns long-distance road relationships while accelerating the learning speed. A multi-layer dilated convolution pyramid structure combining stitching and fusion is used to further extract and fuse long-distance relationships. Directional strip convolution is added to extract directional road features. Skip connections at the same level are replaced with oblique connections, and directional strip convolution is added to compensate for road directional features in the previous level. This achieves effective extraction of long-distance and directional road features, thereby improving the accuracy of road extraction in remote sensing image backgrounds.

[0012] To achieve the above objectives, the present invention adopts the following technical solution:

[0013] A method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution includes the following steps:

[0014] S1. Divide the Deepglobe dataset into a training set and a test set;

[0015] S2. Perform data augmentation on the training set partitioned in step S1;

[0016] S3. Construct a remote sensing image road extraction network model based on large kernel convolution and directional strip convolution;

[0017] S4. Construct the loss function of the remote sensing image road extraction network model based on large kernel convolution and directional strip convolution in training step S3 using BCE Loss and Dice Loss.

[0018] S5. Using the training set augmented in step S2 and the loss function constructed in step S4, train the remote sensing image road extraction network model constructed in step S3 based on large kernel convolution and directional strip convolution to obtain the optimal model.

[0019] S6. Using the optimal model obtained in step S5, predict the test set divided in step S1 to obtain the prediction results.

[0020] S7. Evaluate the prediction results obtained in step S6.

[0021] Step S1 specifically includes:

[0022] S101. First, the Deepglobe dataset is divided into urban dataset and rural dataset. The training set and test set are randomly divided from the urban dataset and the rural dataset in proportion. Then, the training set of the urban dataset and the training set of the rural dataset are merged into an experimental training set, and the test set of the urban dataset and the test set of the rural dataset are merged into an experimental test set.

[0023] S102. The experimental training set and experimental test set obtained in step S101 are trimmed to obtain new experimental training set and experimental test set;

[0024] Step S2 specifically includes:

[0025] S201. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image of the new experimental training set obtained in step S102 as the output image Image1 of this step. If the random number is less than a threshold of 0.5, randomly adjust the hue, saturation, and brightness of the image of the new experimental training set obtained in step S102. Convert the image from the BGR color space to the HSV color space and separate it into 3 channels. Add an integer representing the color rotation angle to the H channel to adjust the color. Add a floating-point number representing the saturation change ratio to the S channel to adjust the saturation. Add a floating-point number representing the brightness change ratio to the V channel to adjust the brightness. Then merge the 3 channels into an HSV image and convert it back to the BGR color space as the processed image Image1.

[0026] S202. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image1 obtained in step S201 as the output image Image2 of this step. If the random number is less than the threshold of 0.5, perform random translation, scaling, and rotation operations on the image Image1 obtained in step S201. Randomly generate rotation angle, scaling ratio, aspect ratio, and translation offset within the limit range, and calculate the transformation matrix together. Define an original bounding box containing the coordinates of the four corner points, translate the center to the origin, and calculate the new bounding box according to the transformation matrix. Then restore this bounding box to the original coordinates of the image and add the translation offset. Convert the two borders to floating-point type and calculate the perspective transformation matrix, and apply it to the image Image1 obtained in step S201 as the processed image Image2.

[0027] S203. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image2 obtained in step S202 as the output image Image3 of this step. If the random number is less than a threshold of 0.5, perform a horizontal flip operation on the image Image2 obtained in step S202 to obtain the processed image Image3.

[0028] S204. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to the 0.5 threshold, use the image Image3 obtained in step S203 as the output image Image4 of this step. If the random number is less than the 0.5 threshold, perform a vertical flip operation on the image Image3 obtained in step S203 to obtain the processed image Image4.

[0029] S205. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image4 obtained in step S204 as the output image Image5 of this step. If the random number is less than a threshold of 0.5, rotate the image Image4 obtained in step S204 by 90 degrees to obtain the processed image Image5.

[0030] Step S3 specifically includes:

[0031] S301. The remote sensing image road extraction network model based on large kernel convolution and directional strip convolution includes a large kernel convolution feature encoding module LKSMP Encoder, a hollow pyramid module, a directional strip feature module StripConv Module, and a feature decoding module, forming an encoding-decoding architecture.

[0032] S302. Construct the Stem module, which consists of a convolutional layer with a 3×3 kernel and a stride of 2, a depthwise separable convolutional layer with a 3×3 kernel, a convolutional layer with a 1×1 kernel, and a depthwise separable convolutional layer with a 3×3 kernel and a stride of 2 in sequence; this module is used to extract shallow features from the input image.

[0033] S303. Construct the LKSMP Block module. The main branch of this module consists of a batch normalization (BN) layer, a 1×1 kernel convolutional layer, a K×K depthwise separable convolutional layer, and a 1×1 kernel convolutional layer in sequence. The K×K depthwise separable convolution introduces SMP operation, selects n points as the main focus points and records the corresponding parameter values, and the other points are calculated from these n points. The residual branch performs a 5-kernel depthwise separable convolution operation on the input feature map, and the residual is added to the main branch as compensation information as the output.

[0034] S304. Construct the ConvFFN module. The main branch of this module consists of a batch normalization layer (BN), a convolutional layer with a kernel size of 1×1, a GELU activation layer, and a convolutional layer with a kernel size of 1×1 in sequence. The residual branch is not operated on. Finally, the main branch and the residual branch are added together as the output.

[0035] S305. Based on the LKSMP Block module constructed in step S303 and the ConvFFN module constructed in step S304, the two are stacked sequentially and repeated N times to construct the Stage module. This module is the core module for feature extraction. It extracts the long-distance relationship of road targets by large kernel K×K convolution. SMP operation is introduced in the calculation process to record less parameter information while making the model gradually focus on important information and perform depthwise separable convolution.

[0036] S306. Construct the Transition module, which consists of a convolutional layer with a kernel size of 1×1 and a depthwise separable convolutional layer with a kernel size of 3×3 and a stride of 2. This module is used to perform downsampling operations on the intermediate feature maps.

[0037] S307. Construct a large kernel convolutional feature encoding module, LKSMP Encoder. This module is based on the Stem module constructed in step S302, the Stage module constructed in step S305, and the Transition module constructed in step S306. These three modules are stacked sequentially. Specifically, a Stem module is stacked first, transforming the input image (256×256×3) into (64×64×64). The combination of the LKSMP Block module and the ConvFFN module is repeated twice, i.e., N is set to 2, forming the Stage1 module, which is stacked sequentially after the Stem module. The large kernel convolution K is set to 31. After Stage1, the feature map size and number of channels remain unchanged, and shallow features are extracted. Then, a Transition module is stacked to downsample the feature map, transforming it from (64×64×64) into (32×32×128). Similar to Stage1, N is set to 2 and K is set to 29, forming the Stage2 module. The feature map size and number of channels remain unchanged, and shallow features are extracted. Then, a Transition module is stacked, transforming the feature map from (64×64×64) into (32×32×128). Similar to Stage1, N is set to 2 and K is set to 29, forming the Stage2 module. The feature map size and number of channels remain unchanged, and shallow features are extracted. The transition module further downsamples the feature map, changing it from (32×32×128) to (16×16×256). Similar to Stage 1, N is set to 18 and K to 27 to form Stage 3, where the feature map size and number of channels remain unchanged, and deep features are extracted. Then, the transition modules are stacked to further downsample the feature map, changing it from (16×16×256) to (8×8×512). Similar to Stage 1, N is set to 2 and K to 13 to form Stage 4, where the feature map size and number of channels remain unchanged, and deep features are extracted. After passing through the LKSMP Encoder module, the feature map changes from (256×256×3) to (8×8×512), gradually encoding the shallow features into deep feature maps. A total of 5 valid intermediate feature maps E0, E1, E2, E3, and E4 are retained in the output.

[0038] S308. Construct a dilated pyramid module, which consists of stacked, progressively larger dilated convolutions. The output of the large kernel convolutional feature encoding module LKSMP Encoder constructed in step S307 is denoted as the input of the dilated pyramid module. A 1×1 convolution operation is performed on the input, keeping the feature map size unchanged, but adjusting its channel number from 512 to 256. Subsequently, four dilated convolution operations are performed consecutively, with the dilation rate dr set to 1, 2, 4, and 8 respectively, resulting in four effective intermediate feature maps: Dp1, Dp2, Dp3, and Dp4, where the channel number and feature map size remain unchanged. These four effective intermediate feature maps are then added to fuse the feature information from different receptive fields, and then concatenated with the input, increasing the channel number to 768. Finally, a 1×1 convolution is performed to allow the information between the channels of the feature map to interact and fuse, resulting in the final output feature map Output.

[0039] S309. Construct directional strip feature extraction blocks, the core of which is strip convolution operation in four directions; First, perform a 1×1 convolution operation on the effective intermediate feature layer generated by the large kernel convolution feature encoding module LKSMP Encoder constructed in step S307, and adjust its channel number to 1 / 4 of the input. Then, perform strip convolution operation in four directions on it respectively, and adjust the channel number to 1 / 8 of the original. Concatenate the four feature maps and pool them, and then perform a 1×1 convolution operation to adjust the channel number to n, which is consistent with the feature map channel of the corresponding layer of the encoder.

[0040] S310. Construct a directional strip feature module. This module is formed by stacking the directional strip feature extraction blocks constructed in step S309 four times. The four blocks take E0, E1, E2 and E3 generated in step S307 as inputs and generate effective intermediate feature maps St1, St2, St3 and St4 respectively.

[0041] S311. Construct the Decoder Block module. The core of this module is the transposed convolution, which upsamples the input feature map. First, the number of channels is adjusted to 1 / 4 of the input through a 1×1 convolution. Then, upsampling is performed through a transposed convolution with a kernel size of 3×3 and a stride of 2. Finally, a 1×1 convolution is used to adjust the number of channels back to the initial size and enable information exchange between channels. Batch normalization (BN) and ReLU activation are performed after each convolution.

[0042] S312. Construct the Final Block module. The core of this module is transposed convolution, which upsamples the input feature map. The input feature map is subjected to transposed convolution with a kernel size of 4 and a stride of 2 to upsample the image and restore it to its initial size. At the same time, the number of channels is reduced. Then, after two convolution operations with a kernel size of 3, the number of channels is finally adjusted to 1, which is used as the output.

[0043] S313. Construct a feature decoding module, which consists of repeating the Decoder Block module constructed in step S311 four times and sequentially stacking the Final Block module constructed in step S312. This module takes the feature map Output obtained in step S308 as input and performs staged fusion and decoding on the effective intermediate feature maps St1, St2, St3, and St4 output by the directional strip feature module in step S310, and finally obtains a prediction result map that matches the input image.

[0044] S314. Construct a remote sensing image road extraction network model based on large kernel convolution and directional strip convolution. This model is built upon the LKSMP Encoder module constructed in step S307, the hollow pyramid module constructed in step S308, the directional strip feature module constructed in step S310, and the feature decoding module constructed in step S313. Specifically, the input image is converted into a feature matrix. The LKSMP Encoder module continuously downsamples and extracts features, generating five effective feature maps: E0, E1, E2, E3, and E4. The hollow pyramid module uses E4 as input to generate feature map Dp. The directional strip feature module uses LKSMP... The Encoder module generates E0, E1, E2, and E3 as inputs, which are used for directional feature extraction to generate four feature maps: St1, St2, St3, and St4. In the decoding stage, the feature map Dp generated by the Hollow Pyramid module and the feature map St4 generated by the directional strip feature module are first fused together for decoding and upsampling. Subsequently, the output De of the feature decoding module is continuously fused and upsampled with the feature map St obtained by the directional strip feature module. After passing through the feature decoding module, a binary image with the same size as the input image is obtained. This image is the prediction result and is used as the output of the network model.

[0045] Step S4 specifically includes:

[0046] S401. Construct the BCE Loss; its formula is defined as follows:

[0047] BCELoss=-(1-y)log(1-x)-ylog(x)

[0048] Where x is the predicted category of each pixel by the network model, and y is the label, i.e., the true category value of each pixel; then the following calculation is performed:

[0049]

[0050] Setting it to 0, we get x = y. The goal is to make the predicted value x continuously approach the true value y, thereby improving the prediction accuracy.

[0051] S402. Constructing Dice Loss; First, the formula for calculating the Dice coefficient is as follows:

[0052]

[0053] Similar to BCE Loss, x is the predicted value of the network model, y is the true value, |x∩y| represents the number of pixel intersections between x and y, and |x| and |y| represent the number of pixels in x and y, respectively. |x∩y| can be approximated as the dot product of the predicted pixel class value and the true pixel class value, and the results are added together. In turn, |x| and |y| can be approximated as the sum of their respective pixel class values ​​in the image.

[0054] The formula for Dice Loss is defined as follows:

[0055]

[0056] When the confidence level of the prediction result is high, the Dice coefficient will also be high and the Dice Loss will be low. When the confidence level is low, the Dice Loss will be correspondingly high, thus penalizing and constraining the training of the model.

[0057] S403. Based on the BCE Loss constructed in step S401 and the Dice Loss constructed in step S402, construct the loss function L of the remote sensing image road extraction network model based on large kernel convolution and oriented strip convolution in training step S3. The specific formula is as follows:

[0058]

[0059] In step S5: During training, the Adam optimizer is used for optimization. The initial learning rate is set to 0.0002. When the calculated loss value no longer decreases, the learning rate is multiplied by 0.2 until the learning rate decreases to below 0.0000005, at which point training stops. The model with the lowest loss value is saved as the optimal model.

[0060] In step S6: During prediction, the Time-of-Test (TTA) method is used, including horizontal image flipping, vertical image flipping, and diagonal image flipping, and then the output is restored to the position that matches the original image; for each prediction probability, a binary output is generated using 0.5 as the prediction threshold, which is the prediction result.

[0061] Step S7 specifically includes:

[0062] S701. The prediction results of step S6 are comprehensively and quantitatively evaluated using the evaluation metrics of accuracy, mean intersection-over-union ratio (mIoU), precision, recall, and F1 score.

[0063] S7011. Evaluate prediction results using accuracy:

[0064] Accuracy refers to the proportion of pixels whose predicted category matches the true label; it is expressed by the following formula:

[0065]

[0066] Wherein, TP represents the number of pixels correctly predicted (positive class as positive, i.e., the number of pixels predicted as roads); TN represents the number of pixels correctly predicted (negative class as negative, i.e., the number of pixels predicted as background); FN represents the number of pixels incorrectly predicted (positive class as negative, and the number of pixels predicted as roads); and FP represents the number of pixels incorrectly predicted (negative class as positive, and the number of pixels predicted as roads).

[0067] S7012. Use the mean intersection-over-union ratio (mIoU) to evaluate the prediction results:

[0068] The Intersection over Union (IoU) ratio for road categories is the ratio of the intersection to the union of the predicted and actual label values ​​for each road category; it is expressed by the following formula:

[0069]

[0070] The mean intersection-union ratio (mIoU) is the average intersection-union ratio between the road category and the background category, expressed by the following formula:

[0071]

[0072] Where k is 2;

[0073] S7013. Evaluate the prediction results using precision:

[0074] Precision refers to the proportion of pixels correctly predicted as roads; it is expressed by the following formula:

[0075]

[0076] S7014. Use recall to evaluate prediction results:

[0077] Recall is the ratio of the number of pixels correctly predicted as roads to the number of pixels in the true labels. In other words, it represents the proportion of correctly predicted road pixels out of all possible road pixels. It is expressed by the following formula:

[0078]

[0079] S7015. Use the F1 score (F-Score) to evaluate the prediction results:

[0080] It can be expressed by the following formula:

[0081]

[0082] The closer the F1 score is to 1, the better the model's predictive performance.

[0083] S702. Using the three-color image method, the predicted results are compared pixel by pixel with the true labels. The road category is taken as the positive class. According to the confusion matrix, the pixels of TP are marked as green, the pixels of FP are marked as red, and the pixels of FN are marked as blue to form a visual effect image. The merits of the optimal model are analyzed.

[0084] This invention also provides a remote sensing image road extraction system based on large kernel convolution and directional strip convolution, comprising:

[0085] Dataset partitioning module: Used to divide the Deepglobe dataset into training and testing sets;

[0086] Training set data augmentation module: used to augment the partitioned training set;

[0087] Network model building module: used to build a remote sensing image road extraction network model based on large kernel convolution and oriented strip convolution;

[0088] Loss function building module: Used to build loss functions for training remote sensing image road extraction network models based on large kernel convolution and oriented strip convolution using BCE Loss and Dice Loss;

[0089] Network model training module: used to train a remote sensing image road extraction network model based on large kernel convolution and directional strip convolution using the data augmented training set and the constructed loss function, and obtain the optimal model;

[0090] Prediction Result Acquisition Module: Used to make predictions on the test set using the optimal model and obtain the prediction results;

[0091] Prediction Result Evaluation Module: Used to evaluate the obtained prediction results.

[0092] This invention also provides a remote sensing image road extraction device based on large kernel convolution and directional strip convolution, comprising:

[0093] Memory: A computer program that stores the above-mentioned method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution, and is a computer-readable device;

[0094] Processor: Used to implement the aforementioned method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution when executing the computer program.

[0095] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the aforementioned method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution.

[0096] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0097] 1) The feature encoding stage of this invention focuses on the long-distance characteristics of roads, resulting in better road connectivity in the prediction results. Based on the inherent connectivity and long-distance characteristics of roads, a novel large-kernel convolution and SMP operation are introduced. This not only provides a large receptive field to connect pixel features at long distances, but also reduces the number of parameters and allows it to automatically focus on important information points, resulting in better prediction performance.

[0098] 2) The tilted skip connection of this invention focuses on the elongation and meandering of roads, and uses a directional strip feature extraction module to improve the model's ability to perceive objects on narrow roads. After deep convolution and downsampling operations in the feature encoding stage, the resolution of the image in the network is continuously reduced until the image is represented as a very small feature map. In this process, a large amount of spatial and detailed information is lost. Moreover, for narrow roads, their feature information exists in a very small proportion, or even no longer exists. This is not conducive to feature recovery in the feature decoding stage. By fusing the features from the encoding stage and further directional feature information as compensating information with the decoding features, the extraction effect of narrow roads is further improved. This independent module can also be used in target extraction tasks similar to road features, such as blood vessels, canals, and power lines.

[0099] 3) The remote sensing image road extraction network model based on large kernel convolution and directional strip convolution proposed in this invention achieves better prediction results. In terms of quantization metrics, IoU is improved by 4.3%, and F-Score is improved by 3%. In the three-color image visualization prediction results mentioned in this invention, the problems of missed detection and false detection can be clearly seen to be improved.

[0100] 4) The model proposed in this invention adopts a modular design approach. The network infrastructure, the SMP-optimized convolutional operation in the feature encoder, the lightweight dilated pyramid module in the middle part, the directional feature extraction module on the skip connections, and the feature decoding module can all be replaced or improved and updated. Modules can be added or modified according to the shortcomings of the network model in the prediction task. With the development of emerging technologies and the proposal of better network modules, this invention can be iteratively updated at any time to improve the model performance.

[0101] In summary, this invention, through a large kernel convolutional feature encoding module, a dilated pyramid module, and a tilted jump connection based on a directional strip feature module, can focus on the long-distance characteristics of roads, automatically focus on important information points, supplement the directional details of roads, and improve the model's ability to perceive weak road targets. It has the advantages of fewer missed detections and false detections, higher connectivity and accuracy, and strong update and iteration capabilities. Attached Figure Description

[0102] Figure 1 This is a flowchart of the present invention.

[0103] Figure 2 This is the overall structure diagram of the network model proposed in this invention.

[0104] Figure 3 This is a detailed structural diagram of the large kernel convolutional feature encoding module of the present invention.

[0105] Figure 4 This is a detailed structural diagram of the hollow pyramid module of the present invention.

[0106] Figure 5 This is a detailed structural diagram of the directional strip feature extraction block of the present invention.

[0107] Figure 6 This is a detailed structural diagram of the feature decoding module of the present invention.

[0108] Figure 7 This is a detailed structural diagram of the network model proposed in this invention.

[0109] Figure 8 This is a comparison of the three-color image method used in this invention with several other methods for predicting test set images. Detailed Implementation

[0110] The technical solution adopted by the present invention will be further described below with reference to the accompanying drawings.

[0111] See Figure 1 A method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution includes the following steps:

[0112] S1. Divide the Deepglobe dataset into a training set and a test set;

[0113] In remote sensing imagery, roads can be broadly categorized into urban and rural roads based on their background characteristics. In urban environments, roads are typically regular and wide, usually constructed of cement or asphalt, and surrounded by buildings and greenbelts. In rural environments, roads are irregular, narrow, mostly dirt roads with less cement, and are sparsely surrounded by buildings, easily blending into the surrounding fields and becoming difficult to distinguish. Because of these differences in road characteristics, a significant discrepancy between the feature distribution in the training and test sets can hinder accurate model performance evaluation. Therefore, consistency in feature distribution is crucial when partitioning the dataset.

[0114] The construction process in step S1 specifically includes the following steps:

[0115] S101. Based on the above road features, the Deepglobe dataset is first divided into an urban dataset and a rural dataset, resulting in 1699 and 3281 images respectively. The training set and test set are randomly divided from the urban dataset in an 8:2 ratio, and the training set and test set are randomly divided from the rural dataset in an 8:2 ratio. Then, the training sets of the urban dataset and the rural dataset are merged into an experimental training set, and the test sets of the urban dataset and the rural dataset are merged into an experimental test set. In the resulting experimental training set and experimental test set, the proportion of images with urban backgrounds and rural backgrounds is consistent, the feature distribution shows a consistent pattern, and the randomness of the division is guaranteed.

[0116] S102. The experimental training set and experimental test set obtained in step S101 are cropped respectively. The original 1024×1024 images are cropped to 256×256, with adjacent images overlapping by 64 pixels. New experimental training set and experimental test set are obtained, with 124,500 and 31,150 images respectively.

[0117] S2. Perform data augmentation on the training set partitioned in step S1;

[0118] The features, such as road distribution and extension direction, as well as background information that can be presented in the dataset are limited. In order to generate as many information-rich road data sample types as possible and improve the coverage of sample features, data augmentation is performed to expand the data samples and avoid overfitting.

[0119] The construction process in step S2 specifically includes the following steps:

[0120] S201. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image of the new experimental training set obtained in step S102 as the output image Image1 of this step. If the random number is less than a threshold of 0.5, randomly adjust the hue, saturation, and brightness of the image of the new experimental training set obtained in step S102. Convert the image from the BGR color space to the HSV color space and separate it into 3 channels. Add an integer representing the color rotation angle to the H channel to adjust the color. Add a floating-point number representing the saturation change ratio to the S channel to adjust the saturation. Add a floating-point number representing the brightness change ratio to the V channel to adjust the brightness. Then merge the 3 channels into an HSV image and convert it back to the BGR color space as the processed image Image1.

[0121] S202. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image1 obtained in step S201 as the output image Image2 of this step. If the random number is less than the threshold of 0.5, perform random translation, scaling, and rotation operations on the image Image1 obtained in step S201. Randomly generate rotation angle, scaling ratio, aspect ratio, and translation offset within the limit range, and calculate the transformation matrix together. Define an original bounding box containing the coordinates of the four corner points, translate the center to the origin, and calculate the new bounding box according to the transformation matrix. Then restore this bounding box to the original coordinates of the image and add the translation offset. Convert the two borders to floating-point type and calculate the perspective transformation matrix, and apply it to the image Image1 obtained in step S201 as the processed image Image2.

[0122] S203. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image2 obtained in step S202 as the output image Image3 of this step. If the random number is less than a threshold of 0.5, perform a horizontal flip operation on the image Image2 obtained in step S202 to obtain the processed image Image3.

[0123] S204. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to the 0.5 threshold, use the image Image3 obtained in step S203 as the output image Image4 of this step. If the random number is less than the 0.5 threshold, perform a vertical flip operation on the image Image3 obtained in step S203 to obtain the processed image Image4.

[0124] S205. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image4 obtained in step S204 as the output image Image5 of this step. If the random number is less than a threshold of 0.5, rotate the image Image4 obtained in step S204 by 90 degrees to obtain the processed image Image5.

[0125] S3. Construct a remote sensing image road extraction network model based on large kernel convolution and directional strip convolution;

[0126] In order to effectively extract features such as long-distance relationships and directional extension of roads and restore them to the original image size, a network model was designed and constructed based on the principle of semantic segmentation to classify each pixel in the image and finally obtain the road extraction result map.

[0127] The construction process in step S3 specifically includes the following steps:

[0128] S301. The remote sensing image road extraction network model based on large kernel convolution and oriented strip convolution mainly includes a large kernel convolution feature encoding module (LKSMP Encoder), a dilated pyramid module, an oriented strip feature module (StripConv Module), and a feature transpose decoder module, forming an encoding-decoding architecture to realize the road feature extraction and reconstruction process. For the overall overview structure of the network model, please refer to [link to network model overview]. Figure 2 ;

[0129] S302. Construct the Stem module, which consists of a convolutional layer with a 3×3 kernel and a stride of 2, a depthwise separable convolutional layer with a 3×3 kernel, a convolutional layer with a 1×1 kernel, and a depthwise separable convolutional layer with a 3×3 kernel and a stride of 2 in sequence. This module mainly performs shallow feature extraction on the input image and reduces the image resolution to reduce the complexity of subsequent convolution operations.

[0130] S303. Construct the LKSMP Block module. The main branch of this module consists of a batch normalization (BN) layer, a 1×1 kernel convolutional layer, a K×K depthwise separable convolutional layer, and a 1×1 kernel convolutional layer in sequence. The K×K depthwise separable convolution introduces SMP operation, selecting n points as the main focus and recording the corresponding parameter values. Other points are calculated from these n points. This allows the model to gradually focus on important information while recording less parameter information, reducing computational complexity and speeding up training. The residual branch performs a 5-kernel depthwise separable convolution operation on the input feature map, and its compensation information is added to the main branch as the output.

[0131] S304. Construct the ConvFFN module. The main branch of this module consists of a batch normalization layer (BN), a convolutional layer with a kernel size of 1×1, a GELU activation layer, and a convolutional layer with a kernel size of 1×1 in sequence. The residual branch is not operated on. Finally, the main branch and the residual branch are added together as the output.

[0132] S305. Based on the LKSMP Block module constructed in step S303 and the ConvFFN module constructed in step S304, the two are stacked sequentially and repeated N times to construct the Stage module. This module is the core module for feature extraction. It extracts the long-distance relationship of road targets by large kernel K×K convolution. SMP operation is introduced in the calculation process to record less parameter information while making the model gradually focus on important information. Depth-separable convolution is also performed to reduce parameters, reduce computational complexity, and speed up training.

[0133] S306. Construct the Transition module, which consists of a convolutional layer with a kernel size of 1×1 and a depthwise separable convolutional layer with a kernel size of 3×3 and a stride of 2. This module mainly performs downsampling operations on the intermediate feature maps to reduce the image resolution and learn deep features.

[0134] S307. Construct the large kernel convolutional feature encoding module LKSMP Encoder. This module is based on the Stem module constructed in step S302, the Stage module constructed in step S305, and the Transition module constructed in step S306. These three modules are stacked sequentially. For detailed structure, please refer to [link to relevant documentation]. Figure 3The specific stacking method is as follows: First, a Stem module is stacked, transforming the input image (256×256×3) (Note: the feature map is represented as height×width×number of channels, and this representation will be used throughout) into (64×64×64); the combination of the LKSMP Block module and the ConvFFN module is repeated twice, i.e., N is set to 2, forming the Stage1 module, which is stacked sequentially after the Stem module. The large kernel convolution K is set to 31. After Stage1, the feature map size and number of channels remain unchanged, and shallow features are extracted; then, a Transition module is stacked, downsampling the feature map from (64×64×64) to (32×32×128); similar to Stage1, N is set to 2, and K is set to 29, forming the Stage2 module. The feature map size and number of channels remain unchanged, and shallow features are extracted; then, a Transition module is stacked, downsampling the feature map from (64×64×64) into (32×32×128); similar to Stage1, N is set to 2, and K is set to 29, forming the Stage2 module. The feature map size and number of channels remain unchanged, and shallow features are extracted; finally, a Transition module is stacked... The transition module further downsamples the feature map, changing it from (32×32×128) to (16×16×256). Similar to Stage 1, N is set to 18 and K to 27 to form Stage 3, with the feature map size and number of channels remaining unchanged, extracting deep features. Then, the transition modules are stacked to further downsample the feature map, changing it from (16×16×256) to (8×8×512). Similar to Stage 1, N is set to 2 and K to 13 to form Stage 4, with the feature map size and number of channels remaining unchanged, extracting deep features. After passing through the LKSMP Encoder module, the feature map changes from (256×256×3) to (8×8×512), gradually encoding the shallow features into deep feature maps containing features such as long-distance road relationships, containing rich high-level semantic information about roads. A total of 5 valid intermediate feature maps E0, E1, E2, E3, and E4 are retained in the output.

[0135] S308. Construct the Dilated Pyramid Module. This module mainly consists of stacked convolutions with gradually increasing dilation. For its detailed structure, please refer to [link to module]. Figure 4The output of the large kernel convolutional feature encoding module LKSMP Encoder constructed in step S307 is denoted as the input of the dilated pyramid module. A 1×1 convolution operation is performed on the input, keeping the feature map size unchanged, but adjusting its number of channels from 512 to 256 to reduce the computational complexity of subsequent steps. Then, four consecutive dilated convolution operations are performed, with the dilation rate dr set to 1, 2, 4, and 8 respectively, resulting in four effective intermediate feature maps: Dp1, Dp2, Dp3, and Dp4, where the number of channels and the feature map size remain unchanged. These four effective intermediate feature maps are then added to fuse the feature information from different receptive fields, and then concatenated with the input, increasing the number of channels to 768. Next, a 1×1 convolution is performed to allow the information between the channels of the feature map to interact and fuse, resulting in the final output feature map Output. This module can expand the receptive field without reducing the image resolution and can capture and fuse multi-scale contextual relationships to obtain feature maps containing complex semantic information for subsequent accurate target localization.

[0136] S309. Construct the StripConv Block, which is based on four-directional strip convolution operations. For its detailed structure, please refer to [link to documentation]. Figure 5 First, a 1×1 convolution operation is performed on the effective intermediate feature layer generated by the large kernel convolutional feature encoding module LKSMPEncoder constructed in step S307, adjusting its channel number to 1 / 4 of the input to reduce computational complexity. Then, four strip convolution operations are performed on it respectively, adjusting the channel number to 1 / 8 of the original. The four feature maps are concatenated and pooled to reduce the resolution of the feature maps so as to fuse with the decoder features. Then, a 1×1 convolution operation is performed to adjust the channel number to n, which is consistent with the feature map channels of the corresponding layer of the encoder. This structure can extract the directional features of the road in the image and fuse them with the decoder features to compensate for the loss of resolution and the directional feature information of the road, making the decoding more accurate.

[0137] S310. Construct the StripConv Module, which is formed by stacking the StripConv feature extraction blocks constructed in step S309 four times. The four blocks take E0, E1, E2, and E3 generated in step S307 as inputs and generate effective intermediate feature maps St1, St2, St3, and St4 respectively. Each block serves as a treaty connection between the encoder and decoder, and is responsible for feature extraction and fusion at different levels. This allows for better fusion of multiple feature information and compensation for details lost by the encoder, thus assisting the decoder in effective decoding.

[0138] S311. Construct the Decoder Block module. The core of this module is the transposed convolution, which upsamples the input feature map. First, the number of channels is adjusted to 1 / 4 of the input through a 1×1 convolution to reduce the amount of computation. Then, upsampling is performed through a transposed convolution with a kernel size of 3×3 and a stride of 2. Finally, a 1×1 convolution is used to adjust the number of channels back to the initial size and enable information exchange between channels. Batch normalization (BN) and ReLU activation are performed after each convolution.

[0139] S312. Construct the Final Block module. The core of this module is transposed convolution, which upsamples the input feature map. The input feature map is subjected to transposed convolution with a kernel size of 4 and a stride of 2 to upsample the image and restore it to its initial size. At the same time, the number of channels is reduced. Then, after two convolution operations with a kernel size of 3, the number of channels is finally adjusted to 1, which is used as the output.

[0140] S313. Construct the feature decoding module, which consists of repeating the Decoder Block module constructed in step S311 four times and sequentially stacking the Final Block module constructed in step S312. For detailed structure, please refer to [link to relevant documentation]. Figure 6 This module takes the feature map Output obtained in step S308 as input, and the effective intermediate feature maps St1, St2, St3, and St4 output by the directional strip feature module in step S310, and performs staged fusion and decoding to finally obtain a prediction result map that matches the input image;

[0141] S314. Construct a remote sensing image road extraction network model based on large kernel convolution and directional strip convolution. Based on the LKSMP Encoder module constructed in step S307, the hollow pyramid module constructed in step S308, the directional strip feature module constructed in step S310, and the feature decoding module constructed in step S313, construct a complete network model. For its detailed structure, please refer to [link to detailed structure]. Figure 7The specific process is as follows: The input image is converted into a feature matrix. The LKSMP Encoder module continuously downsamples and extracts features, generating five effective feature maps: E0, E1, E2, E3, and E4. The hollow pyramid module uses E4 as input to generate feature map Dp. The directional strip feature module uses E0, E1, E2, and E3 generated by the LKSMP Encoder module as input, extracting directional features to generate four feature maps: St1, St2, St3, and St4, which are mainly used to assist the subsequent decoding process. In the decoding stage, the feature map Dp generated by the hollow pyramid module and the feature map St4 generated by the directional strip feature module are first fused together for decoding and upsampling. Subsequently, the output De of the feature decoding module is continuously fused and upsampled with the feature map St obtained from the directional strip feature module. After passing through the feature decoding module, a binary image of the same size and matching the input image is obtained. This image is the prediction result and serves as the output of the network model. This network model can extract basic image features of roads, automatically focus on and extract long-distance relationship information of roads, and use the directional features of roads in the feature decoding stage to effectively restore and decode features to obtain the final result.

[0142] S4. Construct the loss function of the remote sensing image road extraction network model based on large kernel convolution and directional strip convolution in training step S3 using BCE Loss and Dice Loss.

[0143] The network model constructed in step S3 and the process from encoder to decoder are the forward propagation process after the image enters the model. During the learning and iteration process of the network model, after each forward propagation, it is necessary to calculate the difference between the predicted result and the true result, which is denoted as loss, and then propagate it backward to continuously adjust the parameters in the network model in order to obtain the optimal model.

[0144] Step S4 specifically includes the following steps:

[0145] S401. Construct the BCE Loss; its formula is defined as follows:

[0146] BCELoss=-(1-y)log(1-x)-ylog(x)

[0147] Where x is the predicted category of each pixel by the network model, and y is the label, i.e., the true category value of each pixel; then the following calculation can be performed:

[0148]

[0149] Setting it to 0, we get x = y. The goal is to make the predicted value x continuously approach the true value y, thereby improving the prediction accuracy. This loss function is suitable for solving binary classification problems. For example, this method is used to solve the problem of separating roads and background.

[0150] S402. Constructing Dice Loss; First, the formula for calculating the Dice coefficient is as follows:

[0151]

[0152] Similar to BCE Loss, x is the predicted value of the network model, y is the true value, |x∩y| represents the number of pixel intersections between x and y, and |x| and |y| represent the number of pixels in x and y, respectively. |x∩y| can be approximated as the dot product of the predicted pixel class value and the true pixel class value, and the results are added together. In turn, |x| and |y| can be approximated as the sum of their respective pixel class values ​​in the image.

[0153] The formula for Dice Loss is defined as follows:

[0154]

[0155] When the confidence level of the prediction result is high, the Dice coefficient will also be high and the Dice Loss will be low. When the confidence level is low, the Dice Loss will be correspondingly high, thus penalizing and constraining the training of the model.

[0156] S403. Based on the BCE Loss constructed in step S401 and the Dice Loss constructed in step S402, construct the loss function L of the remote sensing image road extraction network model based on large kernel convolution and oriented strip convolution in training step S3. The specific formula is as follows:

[0157]

[0158] This loss function combines the advantages of BCE Loss and Dice Loss, and can more comprehensively measure the difference between the predicted results and the true labels, thereby continuously optimizing the training of the model.

[0159] S5. Using the training set augmented in step S2 and the loss function constructed in step S4, train the remote sensing image road extraction network model constructed in step S3 based on large kernel convolution and directional strip convolution to obtain the optimal model.

[0160] In step S5: During training, the Adam optimizer is used for optimization. The initial learning rate is set to 0.0002. When the calculated loss value no longer decreases, the learning rate is multiplied by 0.2 until the learning rate decreases to below 0.0000005, at which point training stops. The model with the lowest loss value is saved as the optimal model.

[0161] S6. Using the optimal model obtained in step S5, predict the test set divided in step S1 to obtain the prediction results.

[0162] In step S6: During prediction, the Time-of-Test (TTA) method is used, including horizontal image flipping, vertical image flipping, and diagonal image flipping (predicting each image 2×2×2=8 times), and then the output is restored to the position matching the original image; for each prediction probability, a binary output is generated using 0.5 as the prediction threshold, which is the prediction result.

[0163] S7. Evaluate the prediction results obtained in step S6.

[0164] The prediction results obtained in step S6 are quantitatively evaluated using evaluation metrics commonly used in semantic segmentation. The core of this evaluation is the confusion matrix.

[0165] Step S7 specifically includes:

[0166] S701. The prediction results of step S6 are comprehensively and quantitatively evaluated using the evaluation metrics of accuracy, mean intersection-over-union ratio (mIoU), precision, recall, and F1 score.

[0167] S7011. Evaluate prediction results using accuracy:

[0168] Accuracy refers to the proportion of pixels whose predicted category matches the true label; it is expressed by the following formula:

[0169]

[0170] Wherein, TP represents the number of pixels correctly predicted (positive class as positive, i.e., the number of pixels predicted as roads); TN represents the number of pixels correctly predicted (negative class as negative, i.e., the number of pixels predicted as background); FN represents the number of pixels incorrectly predicted (positive class as negative, and the number of pixels predicted as roads); and FP represents the number of pixels incorrectly predicted (negative class as positive, and the number of pixels predicted as roads).

[0171] S7012. Use the mean intersection-over-union ratio (mIoU) to evaluate the prediction results:

[0172] The Intersection over Union (IoU) ratio for road categories is the ratio of the intersection to the union of the predicted and actual label values ​​for each road category; it is expressed by the following formula:

[0173]

[0174] The mean intersection-union ratio (mIoU) is the average intersection-union ratio between the road category and the background category, expressed by the following formula:

[0175]

[0176] Where k is 2; because there are only two categories: road and background;

[0177] S7013. Evaluate the prediction results using precision:

[0178] Precision refers to the proportion of pixels correctly predicted as roads; it is expressed by the following formula:

[0179]

[0180] S7014. Use recall to evaluate prediction results:

[0181] Recall is the ratio of the number of pixels correctly predicted as roads to the number of pixels in the true labels. In other words, it represents the proportion of correctly predicted road pixels out of all possible road pixels. It is expressed by the following formula:

[0182]

[0183] S7015. Use the F1 score (F-Score) to evaluate the prediction results:

[0184] This evaluation metric combines precision and recall, providing a comprehensive assessment of model performance. It is expressed by the following formula:

[0185]

[0186] The closer the F1 score is to 1, the better the model's predictive performance.

[0187] The method of this invention was compared with four other semantic segmentation-based road extraction methods, including U-Net, D-LinkNet34, CoANet, and DDUnet. The quantitative comparison results are shown in Table 1 below. The method of this invention achieved the best performance across all metrics, with IoU and F1 scores reaching 0.732 and 0.845, respectively.

[0188] Table 1: Quantitative Comparison Results of the Prediction of Test Set Images by the Invention and Several Other Methods

[0189] U-Net 0.981 0.810 0.639 0.769 0.779 D-LinkNet34 0.984 0.824 0.665 0.769 0.799 CoANet 0.981 0.815 0.650 0.855 0.788 DDUnet 0.984 0.835 0.688 0.821 0.815 ours 0.986 0.859 0.732 0.858 0.845

[0190] S707. Secondly, the three-color image method is used, which compares the predicted results with the true labels pixel by pixel. Road category is taken as the positive class. Based on the confusion matrix, pixels of TP are marked green, pixels of FP are marked red, and pixels of FN are marked blue, thus forming a visual effect image for intuitive analysis of the optimal model's performance. The original image, label image, road extraction effect images of four other methods, and the method of this invention are shown below. Figure 8 As shown.

[0191] In the first three rows, representing scenarios with relatively simple road structures, the roads extracted by the method of this invention are almost entirely accurate. The fourth and fifth rows depict scenarios with densely packed and haphazardly distributed buildings, while the last two rows show scenarios with densely packed but relatively orderly buildings; in all these scenarios, the method of this invention extracts a relatively complete road network. Compared to existing technologies, the method of this invention has only a small number of FP and FN cases. The tree occlusion problem is more severe in the third and seventh rows; the method of this invention only misses a small number of roads, while the rest are relatively complete. Overall, the roads extracted by the method of this invention are basically consistent with the actual roads on the ground, with very few FP and FN cases. These visualization results also verify the superiority of the method of this invention in road extraction tasks.

[0192] As can be seen from the embodiments, compared with the prior art, the present invention has the following innovative points:

[0193] 1. This invention selects the most commonly used encoder-decoder and skip connections to compensate for feature loss information as the basic architecture, and makes a series of improvements on it to make it perform stably in remote sensing image road extraction tasks. When a better and more stable architecture is proposed, the better basic architecture can be used to update the technology.

[0194] 2. In order to improve the model's ability to perceive road features and make up for the deficiencies in road connectivity and accuracy in the prediction results, this invention selects large kernel convolution with a huge receptive field and strip convolution with sensitive directional features for feature encoding, which improves the road prediction effect to a certain extent. When more road feature perception methods are proposed, better methods can be replaced or added to achieve technological updates and iterations, and further improve the performance of the model.

[0195] 3. To improve model efficiency while maintaining accuracy, this invention employs depthwise separable convolution to reduce computational complexity and introduces SMP convolution to automatically learn and focus on important information while retaining fewer parameters, thereby improving training and prediction efficiency. When better convolutional operations with reduced computational complexity and more efficient automatic focus on important information are proposed, they can be replaced or updated to achieve technological iteration and further improve model efficiency.

[0196] 4. This invention proposes an efficient feature extraction method by analyzing the characteristics of roads in remote sensing images, such as long distance, connectivity, and curvature. This method is also applicable to other target extraction tasks with similar road features, such as the extraction of blood vessels, canals, and power lines.

[0197] 5. This invention employs a novel ultra-large kernel SMP convolution in the feature encoding stage. The feature extraction stage typically uses a 3×3 convolution kernel, which is stacked multiple times to increase the receptive field. This invention uses convolution kernels of sizes 31, 29, 27, and 13, employing an ultra-large receptive field to learn long-distance relationships in roads and extract long-distance features. By introducing SMP operations, during training iterations, based on the current kernel size, the values ​​of n key points in the kernel parameters are determined and recorded. In the next convolution, these n points are used to calculate and complete the parameter values ​​of the entire kernel, followed by depthwise separable convolution. During training iterations, the position and numerical information of the n points continuously change, automatically focusing on and learning important information points. Combined with depthwise separable convolution, this improves the accuracy of model predictions while recording a small amount of parameter information, reducing computational complexity, and accelerating training.

[0198] 6. This invention uses four types of directional strip convolutions to capture features in different directions. When extracting road feature information, square convolution kernels, such as 3×3 or 5×5, are typically used. This invention employs 1×9 strip convolutions, performing convolution operations on the input feature map in four directions: horizontal, vertical, diagonally upward, and diagonally downward, and then concatenating and fusing the feature information. This not only captures feature information in these four directions but also requires significantly fewer parameters than a 9×9 kernel, reducing computational complexity.

[0199] 7. This invention employs a skewed skip connection for directional feature extraction. Typically, there's a connection between the encoder and decoder that directly fuses feature encoding and decoding at the same level, for example, fusing feature encoding and decoding from layer 1 to perform the next decoding operation. In this invention, a skewed skip connection is used, for example, fusing feature encoding from layer 1 with feature decoding from layer 2. A directional feature extraction module is added to the skip connection to extract directional features. This is primarily to obtain directional feature information from the previous layer, fuse it with the feature decoding, and then perform the next decoding operation. This compensates for the spatial loss of feature information while incorporating road direction feature information, thus improving the performance of the decoding stage.

[0200] 8. This invention uses a lightweight dilated pyramid structure. This module takes the output of the feature encoding part as input, first halving its dimension to reduce computational complexity, and then performing four consecutive dilated convolution operations with progressively increasing dilation rates to continuously expand the receptive field, forming dilated pyramid structures with different receptive fields. The feature maps of different scales generated in the middle are added together for multi-scale information fusion, which increases the expressive power of information. The result is concatenated with the original input, increasing the number of features while retaining the original features, allowing effective information to continue to be passed on. This module only needs to compute one consecutive dilated convolution, the multi-scale feature maps generated in the middle can be reused, the computational complexity is low, and the information fusion, expression, and transmission capabilities are strong.

[0201] This invention also provides a remote sensing image road extraction system based on large kernel convolution and directional strip convolution, comprising:

[0202] Dataset partitioning module: used to partition the Deepglobe dataset into training and testing sets in step S1;

[0203] Training set data augmentation module: used to perform data augmentation on the partitioned training set in step S2;

[0204] Network model construction module: used to implement the construction of remote sensing image road extraction network model based on large kernel convolution and directional strip convolution in step S3;

[0205] Loss function construction module: used to implement the loss function for constructing the remote sensing image road extraction network model based on large kernel convolution and directional strip convolution in step S4 using BCE Loss and Dice Loss;

[0206] Network model training module: used to train the remote sensing image road extraction network model based on large kernel convolution and directional strip convolution using the data augmented training set and constructed loss function in step S5, and obtain the optimal model;

[0207] Prediction Result Acquisition Module: Used to implement step S6 by using the optimal model to predict the test set and obtain the prediction result;

[0208] Prediction result evaluation module: used to evaluate the prediction results obtained in step S7.

[0209] This invention also provides a remote sensing image road extraction device based on large kernel convolution and directional strip convolution, comprising:

[0210] Memory: A computer program that stores the above-mentioned method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution, and is a computer-readable device;

[0211] Processor: Used to implement the aforementioned remote sensing image road extraction method based on large kernel convolution and directional strip convolution when executing the computer program.

[0212] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the aforementioned method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution.

Claims

1. A method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution, characterized in that: Includes the following steps: S1. Divide the Deepglobe dataset into a training set and a test set; S2. Perform data augmentation on the training set partitioned in step S1; S3. Construct a remote sensing image road extraction network model based on large kernel convolution and directional strip convolution; Step S3 specifically includes: S301. The remote sensing image road extraction network model based on large kernel convolution and directional strip convolution includes a large kernel convolution feature encoding module LKSMP Encoder, a hollow pyramid module, a directional strip feature module StripConv Module, and a feature decoding module, forming an encoding-decoding architecture. S302. Construct the Stem module, which consists of a convolutional layer with a 3×3 kernel and a stride of 2, a depthwise separable convolutional layer with a 3×3 kernel, a convolutional layer with a 1×1 kernel, and a depthwise separable convolutional layer with a 3×3 kernel and a stride of 2 in sequence; this module is used to extract shallow features from the input image. S303. Construct the LKSMP Block module. The main branch of this module consists of a batch normalization (BN) layer, a 1×1 kernel convolutional layer, a K×K depthwise separable convolutional layer, and a 1×1 kernel convolutional layer in sequence. The K×K depthwise separable convolution introduces SMP operation, selects n points as the main focus points and records the corresponding parameter values, and the other points are calculated from these n points. The residual branch performs a 5-kernel depthwise separable convolution operation on the input feature map, and the residual is added to the main branch as compensation information as the output. S304. Construct the ConvFFN module. The main branch of this module consists of a batch normalization layer (BN), a convolutional layer with a kernel size of 1×1, a GELU activation layer, and a convolutional layer with a kernel size of 1×1 in sequence. The residual branch is not operated on. Finally, the main branch and the residual branch are added together as the output. S305. Based on the LKSMP Block module constructed in step S303 and the ConvFFN module constructed in step S304, the two are stacked sequentially and repeated N times to construct the Stage module. This module is the core module for feature extraction. It extracts the long-distance relationship of road targets by large kernel K×K convolution. SMP operation is introduced in the calculation process to record less parameter information while making the model gradually focus on important information and perform depthwise separable convolution. S306. Construct the Transition module, which consists of a convolutional layer with a kernel size of 1×1 and a depthwise separable convolutional layer with a kernel size of 3×3 and a stride of 2. This module is used to perform downsampling operations on the intermediate feature maps. S307. Construct a large-kernel convolutional feature encoding module, LKSMP Encoder. This module is based on the Stem module constructed in step S302, the Stage module constructed in step S305, and the Transition module constructed in step S306. These three modules are stacked sequentially. Specifically, a Stem module is stacked first, changing the input image size from (256×256×3) to (64×64×64). The combination of the LKSMP Block module and the ConvFFN module is repeated twice, i.e., N is set to 2, forming the Stage1 module, which is stacked sequentially after the Stem module. The large-kernel convolution K is set to 31. After Stage1, the feature map size and number of channels remain unchanged, and shallow features are extracted. Then, a Transition module is stacked to downsample the feature map from (64×64×64) to (32×32×128). Similar to Stage1, N is set to 2 and K is set to 29, forming the Stage2 module. The feature map size and number of channels remain unchanged, and shallow features are extracted. Then, the Transition module is stacked. The transition module further downsamples the feature map, changing it from (32×32×128) to (16×16×256). Similar to Stage 1, N is set to 18 and K to 27 to form Stage 3, where the feature map size and number of channels remain unchanged, and deep features are extracted. Then, the transition modules are stacked to further downsample the feature map, changing it from (16×16×256) to (8×8×512). Similar to Stage 1, N is set to 2 and K to 13 to form Stage 4, where the feature map size and number of channels remain unchanged, and deep features are extracted. After passing through the LKSMP Encoder module, the feature map changes from (256×256×3) to (8×8×512), gradually encoding shallow features into deep feature maps. A total of 5 valid intermediate feature maps E0, E1, E2, E3, and E4 are retained in the output. S308. Construct a dilated pyramid module, which consists of stacked, progressively larger dilated convolutions. The output of the large kernel convolutional feature encoding module LKSMP Encoder constructed in step S307 is denoted as the input of the dilated pyramid module. A 1×1 convolution operation is performed on the input, keeping the feature map size unchanged, but adjusting its channel number from 512 to 256. Subsequently, four consecutive dilated convolution operations are performed, with the dilation rate dr set to 1, 2, 4, and 8 respectively, resulting in four effective intermediate feature maps: Dp1, Dp2, Dp3, and Dp4, where the channel number and feature map size remain unchanged. These four effective intermediate feature maps are then added to fuse the feature information from different receptive fields, and then concatenated with the input, increasing the channel number to 768. Finally, a 1×1 convolution is performed to allow the information between the channels of the feature map to interact and fuse, resulting in the final output feature map Output. S309. Construct directional strip feature extraction blocks, the core of which is strip convolution operation in four directions; First, perform a 1×1 convolution operation on the effective intermediate feature map generated by the large kernel convolution feature encoding module LKSMP Encoder constructed in step S307, and adjust its channel number to 1 / 4 of the input. Then, perform strip convolution operation in four directions on it respectively, and adjust the channel number to 1 / 8 of the original. Concatenate the four feature maps and pool them, and then perform a 1×1 convolution operation to adjust the channel number to n, which is consistent with the feature map channel of the corresponding layer of the encoder. S310. Construct a directional strip feature module. This module is formed by stacking the directional strip feature extraction blocks constructed in step S309 four times. The four blocks take E0, E1, E2 and E3 generated in step S307 as inputs and generate effective intermediate feature maps St1, St2, St3 and St4 respectively. S311. Construct the Decoder Block module. The core of this module is the transposed convolution, which upsamples the input feature map. First, the number of channels is adjusted to 1 / 4 of the input through a 1×1 convolution. Then, upsampling is performed through a transposed convolution with a kernel size of 3×3 and a stride of 2. Finally, a 1×1 convolution is used to adjust the number of channels back to the initial size and enable information exchange between channels. Batch normalization (BN) and ReLU activation are performed after each convolution. S312. Construct the Final Block module. The core of this module is transposed convolution, which upsamples the input feature map. The input feature map is subjected to transposed convolution with a kernel size of 4 and a stride of 2 to upsample the image and restore it to its initial size. At the same time, the number of channels is reduced. Then, after two convolution operations with a kernel size of 3, the number of channels is finally adjusted to 1, which is used as the output. S313. Construct a feature decoding module, which consists of repeating the Decoder Block module constructed in step S311 four times and sequentially stacking the Final Block module constructed in step S312. This module takes the feature map Output obtained in step S308 as input and performs staged fusion and decoding on the effective intermediate feature maps St1, St2, St3, and St4 output by the directional strip feature module in step S310, and finally obtains a prediction result map that matches the input image. S314. Construct a remote sensing image road extraction network model based on large kernel convolution and directional strip convolution. This model is built upon the LKSMP Encoder module constructed in step S307, the hollow pyramid module constructed in step S308, the directional strip feature module constructed in step S310, and the feature decoding module constructed in step S313. Specifically, the input image is converted into a feature matrix. The LKSMP Encoder module continuously downsamples and extracts features, generating five effective intermediate feature maps: E0, E1, E2, E3, and E4. The hollow pyramid module uses E4 as input to generate feature map Dp. The directional strip feature module uses LKSMP... The E0, E1, E2, and E3 generated by the Encoder module are used as inputs to extract directional features, generating four effective intermediate feature maps: St1, St2, St3, and St4. In the decoding stage, the feature map Dp generated by the Hollow Pyramid module and the feature map St4 generated by the directional strip feature module are first fused together for decoding and upsampling. Subsequently, the output De of the feature decoding module is continuously fused and upsampled with the feature map St obtained by the directional strip feature module. After passing through the feature decoding module, a binary image of the same size and matching the input image is obtained. This image is the prediction result and serves as the output of the network model. S4. Construct the loss function of the remote sensing image road extraction network model based on large kernel convolution and directional strip convolution in training step S3 using BCE Loss and Dice Loss. S5. Using the training set augmented in step S2 and the loss function constructed in step S4, train the remote sensing image road extraction network model constructed in step S3 based on large kernel convolution and directional strip convolution to obtain the optimal model. S6. Using the optimal model obtained in step S5, predict the test set divided in step S1 to obtain the prediction results. S7. Evaluate the prediction results obtained in step S6.

2. The method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution according to claim 1, characterized in that: Step S1 specifically includes: S101. First, the Deepglobe dataset is divided into urban dataset and rural dataset. The training set and test set are randomly divided from the urban dataset and the rural dataset in proportion. Then, the training set of the urban dataset and the training set of the rural dataset are merged into an experimental training set, and the test set of the urban dataset and the test set of the rural dataset are merged into an experimental test set. S102. The experimental training set and experimental test set obtained in step S101 are trimmed to obtain new experimental training set and experimental test set; Step S2 specifically includes: S201. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image of the new experimental training set obtained in step S102 as the output image Image1 of this step. If the random number is less than a threshold of 0.5, randomly adjust the hue, saturation, and brightness of the image of the new experimental training set obtained in step S102. Convert the image from the BGR color space to the HSV color space and separate it into 3 channels. Add an integer representing the color rotation angle to the H channel to adjust the color. Add a floating-point number representing the saturation change ratio to the S channel to adjust the saturation. Add a floating-point number representing the brightness change ratio to the V channel to adjust the brightness. Then merge the 3 channels into an HSV image and convert it back to the BGR color space as the processed image Image1. S202. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image1 obtained in step S201 as the output image Image2 of this step. If the random number is less than the threshold of 0.5, perform random translation, scaling, and rotation operations on the image Image1 obtained in step S201. Randomly generate rotation angle, scaling ratio, aspect ratio, and translation offset within the limit range, and calculate the transformation matrix together. Define an original bounding box containing the coordinates of the four corner points, translate the center to the origin, and calculate the new bounding box according to the transformation matrix. Then restore this bounding box to the original coordinates of the image and add the translation offset. Convert the two borders to floating-point type and calculate the perspective transformation matrix, and apply it to the image Image1 obtained in step S201 as the processed image Image2. S203. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image2 obtained in step S202 as the output image Image3 of this step. If the random number is less than a threshold of 0.5, perform a horizontal flip operation on the image Image2 obtained in step S202 to obtain the processed image Image3. S204. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image3 obtained in step S203 as the output image Image4 of this step. If the random number is less than a threshold of 0.5, perform a vertical flip operation on the image Image3 obtained in step S203 to obtain the processed image Image4. S205. Randomly generate a random number between 0 and 1. If the random number is greater than or equal to a threshold of 0.5, use the image Image4 obtained in step S204 as the output image Image5 of this step. If the random number is less than a threshold of 0.5, rotate the image Image4 obtained in step S204 by 90 degrees to obtain the processed image Image5.

3. The method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution according to claim 1, characterized in that: Step S4 specifically includes: S401. Construct the BCE Loss; its formula is defined as follows: Where x is the predicted category of each pixel by the network model, and y is the label, i.e., the true category value of each pixel; then the following calculation is performed: Setting it to 0, we get x = y. The goal is to make the predicted value x continuously approach the true value y, thereby improving the prediction accuracy. S402. Constructing Dice Loss; First, the formula for calculating the Dice coefficient is as follows: Similar to BCE Loss, x is the predicted value of the network model, y is the true value, |x ∩ y| represents the number of pixel intersections between x and y, and |x| and |y| represent the number of pixels in x and y, respectively. |x ∩ y| can be approximated as the dot product of the predicted pixel class value and the true pixel class value, and the results are added together. In turn, |x| and |y| can be approximated as the sum of their respective pixel class values ​​in the image. The formula for Dice Loss is defined as follows: When the confidence level of the prediction result is high, the Dice coefficient will also be high and the Dice Loss will be low. When the confidence level is low, the Dice Loss will be correspondingly high, thus penalizing and constraining the training of the model. S403. Based on the BCE Loss constructed in step S401 and the Dice Loss constructed in step S402, construct the loss function L of the remote sensing image road extraction network model based on large kernel convolution and oriented strip convolution in training step S3. The specific formula is as follows: 。 4. The method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution according to claim 1, characterized in that: In step S5: During training, the Adam optimizer is used for optimization. The initial learning rate is set to 0.0002. When the calculated loss value no longer decreases, the learning rate is multiplied by 0.2 until the learning rate decreases to below 0.0000005, at which point training stops. The model with the lowest loss value is saved as the optimal model.

5. The method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution according to claim 1, characterized in that: In step S6: During prediction, the Time-of-Test (TTA) method is used, which includes horizontal image flipping, vertical image flipping, and diagonal image flipping. Then, the output is restored to the position that matches the original image. For each prediction probability, a binary output is generated using 0.5 as the prediction threshold, which serves as the prediction result.

6. The method for road extraction from remote sensing images based on large kernel convolution and directional strip convolution according to claim 1, characterized in that: Step S7 specifically includes: S701. The prediction results of step S6 are comprehensively and quantitatively evaluated using the evaluation metrics of accuracy, mean intersection-union ratio (mIoU), precision, recall, and F-score. S7011. Use accuracy to evaluate the prediction results: Accuracy refers to the proportion of pixels whose predicted category matches the true label; it is expressed by the following formula: Wherein, TP represents the number of pixels correctly predicted (positive class as positive, i.e., the number of pixels predicted as roads); TN represents the number of pixels correctly predicted (negative class as negative, i.e., the number of pixels predicted as background); FN represents the number of pixels incorrectly predicted (positive class as negative, and the number of pixels predicted as roads); and FP represents the number of pixels incorrectly predicted (negative class as positive, and the number of pixels predicted as roads). S7012. Use the average crossover ratio (mIoU) to evaluate the prediction results: The Intersection over Union (IoU) of road categories is the ratio of the intersection to the union of the predicted and actual label values ​​for each road category; it is expressed by the following formula: The average intersection-union ratio (mIoU) is the average intersection-union ratio between the road category and the background category; S7013. Use Precision to evaluate the prediction results: Precision refers to the proportion of pixels correctly predicted as road categories; it is expressed by the following formula: S7014. Use recall to evaluate prediction results: Recall is the ratio of the number of pixels correctly predicted as roads to the number of pixels in the true labels; in other words, it represents the proportion of correctly predicted road pixels out of all road pixels. It is expressed by the following formula: S7015. Use the F1 score (F-Score) to evaluate the prediction results: It can be expressed by the following formula: The closer the F1 score is to 1, the better the model's predictive performance. S702. Using the three-color image method, the predicted results are compared pixel by pixel with the true labels. The road category is taken as the positive class. According to the confusion matrix, the pixels of TP are marked as green, the pixels of FP are marked as red, and the pixels of FN are marked as blue to form a visual effect image. The merits of the optimal model are analyzed.

7. A road extraction system based on large kernel convolution and directional strip convolution, used to implement the method of claim 1, characterized in that: include: Dataset partitioning module: Used to divide the Deepglobe dataset into training and testing sets; Training set data augmentation module: used to augment the partitioned training set; Network model building module: used to build a remote sensing image road extraction network model based on large kernel convolution and oriented strip convolution; Loss function building module: Used to build loss functions for training remote sensing image road extraction network models based on large kernel convolution and oriented strip convolution using BCE Loss and Dice Loss; Network model training module: used to train a remote sensing image road extraction network model based on large kernel convolution and directional strip convolution using the data augmented training set and the constructed loss function, and obtain the optimal model; Prediction Result Acquisition Module: Used to make predictions on the test set using the optimal model and obtain the prediction results; Prediction Result Evaluation Module: Used to evaluate the obtained prediction results.

8. A road extraction device for remote sensing images based on large kernel convolution and directional strip convolution, characterized in that: include: Memory: storing a computer program for a remote sensing image road extraction method based on large kernel convolution and directional strip convolution as described in any one of claims 1-6; Processor: Used to implement the remote sensing image road extraction method based on large kernel convolution and directional strip convolution as described in any one of claims 1-6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of the remote sensing image road extraction method based on large kernel convolution and directional strip convolution as described in any one of claims 1-6.

Images