Method, system, equipment and medium for extracting road network from high-resolution remote sensing images based on asymmetric neural network

Abstract

A high-resolution remote sensing image road network extraction method, system, equipment and medium based on an asymmetric neural network, the method comprising: first obtaining high-resolution road remote sensing image data to be trained and preprocessing it; secondly, using a multi-layer pre-training model ResNet34 to gradually obtain two layers of low-level features and high-level features of the road from the preprocessed road image data using an encoder; inputting the high-level road features extracted by the encoder into a high-level feature optimization module AFOM to optimize and enhance them, and removing redundant features to obtain optimized high-level road features; finally, using a decoder to optimize and fuse the two layers of low-level features of the road and the optimized high-level road features to obtain an optimized overall model, using a comprehensive representation of the loss function softDice and BCE to represent the optimized overall model, and finally outputting the extracted road network diagram by restoring the image resolution; the system, equipment and medium, based on the high-resolution remote sensing image road network extraction method using an asymmetric neural network, identify and segment roads in high-resolution remote sensing images; the present invention has the advantages of high accuracy and good robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of road extraction based on high-resolution remote sensing images, and in particular to a method, system, equipment and medium for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network. Background Art

[0002] Road networks play a key role in urban and rural infrastructure, crucial for driving economic growth and improving residents' quality of life. With the development of remote sensing technology, particularly the application of high-resolution remote sensing imagery, more detailed image data has been generated, providing a sufficient source of data for automatic detection and in-depth analysis of road networks, making the automatic identification and extraction of road networks more feasible.

[0003] However, although high-resolution images can represent finer surface features, since each pixel covers a small area of ​​ground, the same surface area requires more pixels to represent, and the amount of data will increase significantly. This not only requires higher computing power, but also puts higher demands on algorithm design.

[0004] Road network extraction methods have evolved through three stages: morphological feature-based, handcrafted feature-based, and deep learning-based. The first two traditional methods are typically time-consuming and rely on manual interpretation, limiting their accuracy. With the continued development and refinement of deep learning technology, road extraction methods based on this technology are also being optimized and improved. However, the complex terrain information in high-resolution remote sensing imagery introduces a significant amount of noise, such as obstructions from trees and buildings along the road, and interference from vehicles. This, coupled with the existence of the same object with different spectra and the same spectrum with different objects, greatly increases the difficulty of road network extraction.

[0005] The patent application (CN118072155A) describes a method for extracting roads from high-resolution remote sensing images based on a deformable attention mechanism. The method first augments the training dataset using sliding windows, horizontal flips, and vertical flips. A road enhancement module is constructed using one-dimensional convolutions in four directions: horizontal, vertical, left diagonal, and right diagonal. A 1×1 convolution module constrains the learning of the deformable offset in the self-attention mechanism, forming a deformable attention module that automatically adapts to road geometric transformations. A ResNet50 encoder and a bilinear interpolation upsampling decoder are used as the base model, with the deformable attention module and road enhancement module embedded in between to extract multi-scale road information. Road centerlines are extracted, and disconnection detection and reconnection are performed on the extracted centerlines to improve their connectivity. Finally, the optimized road network centerlines are converted from raster to vector to generate a vector road network. This application utilizes a deformable attention mechanism for road centerline extraction. However, this method's high computational complexity requires significant computational resources. Furthermore, the extracted road centerlines cannot intuitively reflect the width of each section of the road network, making them incapable of adapting to road scenarios with varying widths, hindering the model's generalization capabilities. Summary of the Invention

[0006] In order to overcome the problems of the above-mentioned prior art, the purpose of the present invention is to propose a method, system, equipment and medium for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network. A new road network extraction model is developed by combining an asymmetric network structure, multiple attention mechanisms and multiple feature enhancement modules. It uses an encoder to gradually acquire the primary low-level features and high-level features of the road, uses a high-level feature optimization module to optimize the high-level feature representation of the road, uses a variable convolution module and an attention refinement module in the decoder to optimize the low-level feature representation of the road, and fuses the optimized low-level features and high-level features, which has the advantages of low complexity and high accuracy.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network is proposed. First, the high-resolution road remote sensing image data to be trained is obtained and preprocessed. Second, an encoder is used to gradually obtain two layers of low-level road features from the preprocessed road image data using a multi-layer pre-trained model ResNet34. The high-level road features extracted by the encoder are input into an advanced feature optimization module (AFOM) for optimization and enhancement, and redundant features are removed to obtain optimized high-level road features. Finally, a decoder is used to optimize and fuse the two layers of low-level road features and the optimized high-level road features to obtain an optimized overall model. The optimized overall model is represented by a combination of the loss functions softDice and BCE. Finally, the extracted road network diagram is output by restoring the image resolution.

[0009] A method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network specifically comprises the following steps:

[0010] Step 1: Obtain high-resolution road remote sensing image data to be trained: Use public network resources to obtain high-resolution remote sensing images containing roads and perform preprocessing operations, including image segmentation, edge filling, rotation, horizontal and vertical migration, to effectively expand the sample dataset;

[0011] Step 2: Using a multi-layer pre-trained ResNet34 model as an encoder, we gradually acquire two layers of low-level features and high-level features of the road. The low-level features include road edges and road surface texture features, while the high-level features include the overall shape of the road and the spatial relationship of the road network.

[0012] Step 3: Input the high-level road features extracted in step 2 into the advanced feature optimization module AFOM to optimize and enhance the high-level road features and remove redundant information to obtain the optimized high-level road features.

[0013] In step 4, the two-layer low-level features of the road obtained in step 2 and the optimized high-level features of the road obtained in step 3 are input into the decoder for optimization and fusion to obtain the optimized overall model. The optimized overall model is represented by the comprehensive loss function softDice and BCE. By restoring the image resolution, the extracted road network diagram is finally output.

[0014] The step 2 includes:

[0015] First, a multi-layer pre-trained model, ResNet34, is used as the encoder. This encoder progressively extracts road features from the input image through one layer of convolution, one layer of maximum pooling, and four layers of residual convolution blocks. The first two layers of residual convolution blocks extract low-level features such as road edges and road surface texture, while the last two layers of residual convolution blocks extract high-level features such as the overall road shape and spatial relationships of the road network.

[0016]

[0017] Among them, input represents the preprocessed training data set, conv_bn_relu(·) represents convolution, normalization and relu function activation of the input to extract preliminary feature representation, maxpool(·) represents the maximum pooling operation on the convolved features to reduce the spatial dimension of the feature map and reduce the amount of calculation, encoder1-encoder4 represent the residual convolution blocks layer1-layer4 of ResNet34 respectively, and the generated low l1 、low l2, high represent the two layers of low-level feature vectors and high-level feature vectors obtained by the encoder respectively.

[0018] The specific method of step 3 is:

[0019] Step 3.1 The advanced feature optimization module AFOM includes a convolutional attention module CBAM and multiple feature enhancement modules, wherein the multiple feature enhancement modules include a void spatial convolution pooling pyramid module ASPP, a strip dilation convolution module, and a strip pooling module; the high-level feature vector obtained by the encoder in step 2 is input into the convolutional attention module CBAM to remove redundant information, wherein the convolutional attention module CBAM includes a channel attention module CAM and a spatial attention module SAM. The channel attention module CAM calculates the importance weight of each channel through global average pooling and a fully connected layer. The spatial attention module SAM aggregates spatial dimension information through deep convolution and uses a convolution layer to learn the weight of each position;

[0020] high′=SAM(CAM(high)), (2)

[0021] Among them, CAM(·) represents the channel attention mechanism, SAM(·) represents the spatial attention mechanism, and high′ is the high-level feature vector after the convolutional attention module;

[0022] Step 3.2 uses the atrous spatial convolution pooling pyramid module (ASPP), the strip dilation convolution module, and the strip pooling module to enable the model to obtain a more accurate representation of road features:

[0023] In step 3.2.1, the high-level feature vector high′ obtained by the convolutional attention module in step 3.1 is further extracted and integrated through the atrous spatial convolutional pooling pyramid module (ASPP). This module extracts multi-scale road information to enrich the perception of road details and enhance the understanding of the global structure of the road network. The atrous spatial convolutional pooling pyramid module (ASPP) uses four convolution kernels with different apertures to capture road features at different scales:

[0024]

[0025] Here, aspp1(·)-aspp4(·) represent convolution operations with different expansion coefficients, where the expansion coefficients are 1, 6, 12, and 18 respectively. Indicates concat operation, high′ ASPP Represents the high-level feature vector after ASPP operation;

[0026] Step 3.2.2: The high-level feature vector high′ obtained from the convolutional attention module is further extracted and integrated through the strip dilation convolution module and the strip pooling module:

[0027]

[0028] Here, AvgPoolH(·) represents horizontal average pooling, AvgPoolV(·) represents vertical average pooling, stripH(·) represents horizontal dilated convolution, stripV(·) represents vertical dilated convolution, stripDL(·) represents diagonal dilated convolution, and stripDR(·) represents anti-diagonal dilated convolution, where the dilation coefficient is 6. Indicates concat operation, high′ strip Represents the high-level feature vector after strip-dilated convolution and strip-pooling operations;

[0029] Step 3.2.3: Concatenate the high-level feature vectors from the dilated spatial convolution pooling pyramid module ASPP, the strip dilation convolution module, and the strip pooling module to obtain the optimized high-level features:

[0030]

[0031] here, represents the concat operation, and "high" represents the high-level feature vector optimized by the Advanced Feature Optimization Module (AFOM).

[0032] The specific method of step 4 is:

[0033] Step 4.1: The first layer of low-level feature vectors obtained in step 2 is passed through a variable convolution module to obtain low-level feature vectors adapted to different scales:

[0034] low' l1 =bn_relu(deformConv(low l1 )), (6)

[0035] Here, deformConv(·) represents a variable convolution operation, bn_relu(·) represents a batch normalization operation on the first layer of low-level feature vectors after the variable convolution operation and then a relu activation function, low′ l1 Represents the first layer of low-level feature vector after variable convolution operation;

[0036] Step 4.2: The second low-level feature vector of the two low-level feature vectors obtained in step 2 is passed through the attention refinement module ARM to obtain a more accurate low-level feature vector that ignores redundant features:

[0037] low' l2 =low l2×sigmoid(conv(GAP(low l2 ))), (7)

[0038] Here, GAP(·) represents the global average pooling operation, conv(·) represents the 1×1 convolution operation, sigmoid(·) represents the activation function, and low′ l2 Represents the second-layer low-level feature vector after the attention refinement module ARM;

[0039] Step 4.3: The high-level features obtained after step 3 are optimized by the high-level feature optimization module AFOM are subjected to convolution and upsampling operations to obtain a more accurate high-level feature vector:

[0040] high”′=upsample(conv_bn_relu(high”)), (8)

[0041] Here, conv_bn_relu(·) represents convolution, normalization, and relu function activation of high' to extract more accurate feature representation, upsample(·) represents bilinear interpolation operation on the feature, and high'' represents the optimized high-level feature vector finally obtained;

[0042] Step 4.4 concatenates the feature vectors obtained from steps 4.1 to 4.3, and uses the final convolutional layer to enhance the feature representation and improve the generalization ability of the model, obtaining an optimized overall model:

[0043]

[0044] Here, conv_bn_relu(·) indicates that the convolution, normalization and relu function activation of the concatenated features are performed for the final feature fusion, upsample(·) indicates that the feature vector is subjected to bilinear interpolation, and sigmoid(·) indicates the activation function. Represents the concat operation, and output represents the final output road network extraction graph;

[0045] Step 4.5 uses the combined loss function softDice and BCE to represent the optimized overall model, where softDice focuses on predicting the overlap between the road network graph and the label; BCE focuses on distinguishing between road and non-road areas:

[0046]

[0047] Here, y true represents the annotation label of the real image, Represents the road label predicted by the model, smooth represents a small smoothing constant to prevent the loss function from dividing by zero and causing errors. By restoring the image resolution, the extracted road network diagram is finally output.

[0048] A method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network, comprising:

[0049] The encoder module is used in step 2 to gradually obtain two layers of low-level features and high-level features of the road using a multi-layer pre-trained model ResNet34 as an encoder. The low-level features include road edges and road surface texture features, while the high-level features include the overall shape of the road and the spatial relationship of the road network.

[0050] The Advanced Feature Optimization Module (AFOM) is used in step 3 to input the high-level road features extracted in step 2 into the Advanced Feature Optimization Module (AFOM) to optimize and enhance the high-level road features and remove redundant information to obtain optimized high-level road features.

[0051] The decoder module is used in step 4. It inputs the two-layer low-level features of the road obtained in step 2 and the optimized high-level features of the road obtained in step 3 into the decoder for optimization and fusion to obtain the optimized overall model. The optimized overall model is represented by the comprehensive loss function softDice and BCE. By restoring the image resolution, the extracted road network diagram is finally output.

[0052] A high-resolution remote sensing image road network extraction device based on an asymmetric neural network includes: a memory and a processor, wherein the memory stores a computer program so that the processor executes the high-resolution remote sensing image road network extraction method based on an asymmetric neural network.

[0053] A storage medium for receiving user input programs, wherein when the stored computer program is executed by a processor, an accurate road network diagram can be obtained based on the high-resolution remote sensing image road network extraction method based on an asymmetric neural network.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. Low-level and high-level feature extraction: The present invention uses the pre-trained model ResNet34 as the encoder and selects 5 layers for feature extraction. The feature vectors extracted by the middle two layers are used as low-level features, and the output of the last layer is used as high-level features. The two features are optimized separately to enhance feature representation.

[0056] 2. High-level feature optimization module: The present invention refines and removes redundancy from the initial high-level features passed through the encoder through the convolutional attention module (CBAM), the atrous spatial convolutional pooling pyramid module (ASPP), the strip dilation convolution module, and the strip pooling module, so that the optimized high-level features can take into account the multi-scale information and strip characteristics of the road.

[0057] 3. Decoder Module: Variable convolution, attention refinement, and convolution operations are used in the decoder module to further optimize both low-level and high-level road information. The optimized low-level features focus more on accurate road context. The high-level features further optimized by convolution are concatenated with the optimized low-level features and then subjected to three layers of convolution to more effectively extract road features and improve the model's accurate reconstruction of the road.

[0058] 4. Road Network Extraction Using an Asymmetric Network Architecture: We developed a new asymmetric network framework for road network extraction from high-resolution remote sensing imagery. Combining techniques such as attention mechanisms, strip-dilated convolutions, strip-pooling, and variable convolutions, we optimized and integrated low-level and high-level road feature representations, enhancing the accuracy of road network extraction without significantly increasing time complexity.

[0059] 5. Verification Experiments and Performance Advantages: Extensive ablation experiments demonstrate the effectiveness of the proposed innovative submodules, reinforcing the feasibility and performance of the invention. Comparative experiments with existing UNet-based methods, DeepLabv3+-based methods, and Mamba-based methods demonstrate performance advantages on the DeepGlobe dataset, indicating that this model achieves superior results in road network extraction.

[0060] The present invention introduces an innovative road network extraction method, which improves the accuracy and robustness of road network extraction through a road network segmentation method based on low-level and high-level semantic feature extraction and feature fusion.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] 1. Effective solution to the problem of road discontinuity caused by obstruction of surrounding objects: Compared with the existing high-resolution remote sensing image road extraction method based on deformable attention mechanism (CN118072155A), which uses one-dimensional convolution in four directions, the present invention uses ASPP, four-directional strip dilation convolution and two-directional strip pooling in the high-level feature optimization module to optimize high-level features. Through these modules, the striping characteristics of the road can be better captured and the dilated convolution can be used to obtain multi-scale information of the road. This feature information is crucial for road connectivity.

[0063] 2. Lower time complexity: Compared with the existing high-resolution remote sensing image road extraction method based on deformable attention mechanism (CN118072155A), which adopts deformable attention mechanism, the present invention only adopts variable convolution after the first layer of low-level features without adding attention mechanism, which reduces the time complexity to a certain extent.

[0064] 3. Efficiency of an asymmetric network structure: This method utilizes an asymmetric network structure design, which reduces overall model complexity while maintaining high-precision extraction performance. This design reduces the demand for computing resources and improves the model's operational efficiency, making it more suitable for processing large-scale remote sensing image datasets.

[0065] 4. Optimized Feature Processing: The Advanced Feature Optimization Module (AFOM) further optimizes and enhances the high-level features extracted by the encoder, removing unnecessary redundant features while retaining the information most critical for road recognition. This step significantly improves the expressiveness of features and the generalization of the model, especially in changing road environments. The decoder fuses the optimized high-level features with low-level features, achieving multi-dimensional information integration. This fusion strategy not only enhances the model's ability to capture road details but also improves its understanding of complex road network structures, thereby improving model accuracy.

[0066] 5. Potential for Broad Applications: Utilizing the larger DeepGlobe dataset for training, the model is able to learn from more complex terrain environments. The pre-trained ResNet34 model effectively captures low-level to high-level road features, including edges, textures, shapes, and spatial relationships. Multiple attention mechanisms and feature optimization modules are used to extract multi-scale features and global contextual features, providing rich information for accurate road identification and segmentation. This is particularly suitable for extracting roads in complex backgrounds. Given the robustness and generalization capabilities of this invention, it has broad potential for application in a wide range of fields, including urban planning, traffic monitoring, disaster response, environmental monitoring, and national security, with broad application prospects.

[0067] The present invention focuses on the precise extraction and optimization of road features. First, ResNet34 is used to capture road features from low-level to high-level. For low-level features, variable convolution and attention refinement modules are used for optimization to include more road context information and remove redundant information, so that the remaining features are highly correlated with the final road features; for high-level features, attention mechanism, ASPP module, strip dilation convolution module and strip pooling are used for optimization to enrich the road feature representation and retain highly correlated features; finally, the optimized low-level features and high-level features are spliced ​​together and the spliced ​​features are further optimized through three-layer convolution. Finally, the road resolution is restored through bilinear upsampling to reconstruct the road network.

[0068] Overall, this method, combining an encoder, an advanced feature optimization module, and a decoder, can extract rich road features, demonstrating increased accuracy and efficiency, particularly when dealing with complex terrain obstructions and shadows. This approach is not only theoretically universal but also demonstrates significant performance advantages in practical applications.

[0069] In summary, the present invention utilizes an asymmetric neural network combined with multiple feature optimization modules. This design demonstrates universal applicability across diverse environments, eliminating the need for customized adjustments for individual datasets. By simultaneously optimizing both low-level and high-level road features, it achieves better representation of road characteristics, improving road segmentation capabilities in complex terrain scenarios. This results in highly accurate road segmentation, providing an efficient and reliable solution for applications across multiple fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 Extracting road flow graphs for asymmetric neural networks.

[0071] Figure 2 This is the overall architecture diagram of the high-resolution remote sensing image road network extraction model based on an asymmetric neural network.

[0072] Figure 3 This is the framework diagram of the Advanced Feature Optimization Module (AFOM).

[0073] Figure 4 This is the decoder attention refinement module (ARM) framework diagram. DETAILED DESCRIPTION

[0074] This example presents a method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network. Figure 1 The specific overall framework of the network structure is as follows: Figure 2 As shown:

[0075] Step 1: Obtain high-resolution road remote sensing image data to be trained: Use public network resources to obtain high-resolution remote sensing images containing roads and perform preprocessing operations, including image segmentation, edge filling, rotation, horizontal and vertical migration, to effectively expand the sample dataset;

[0076] Step 2: Use the multi-layer pre-trained model ResNet34 as an encoder to gradually obtain two layers of low-level features and high-level features of the road. The low-level features include road edges and road surface texture features, and the high-level features include the overall shape of the road and the spatial relationship of the road network. The specific method is:

[0077] First, a multi-layer pre-trained model, ResNet34, is used as the encoder. This encoder progressively extracts road features from the input image through one layer of convolution, one layer of maximum pooling, and four layers of residual convolution blocks. The first two layers of residual convolution blocks extract low-level features such as road edges and road surface texture, while the last two layers of residual convolution blocks extract high-level features such as the overall road shape and spatial relationships of the road network.

[0078]

[0079] Among them, input represents the preprocessed training data set, conv_bn_relu(·) represents convolution, normalization and relu function activation of the input to extract preliminary feature representation, maxpool(·) represents the maximum pooling operation on the features after the first layer of convolution to reduce the spatial dimension of the feature map and reduce the amount of calculation, encoder1-encoder4 respectively represent the residual convolution blocks layer1-layer4 of ResNet34, and the generated low l1 、low l2 , high represent the two layers of low-level feature vectors and high-level feature vectors obtained by the encoder respectively.

[0080] Step 3: Input the high-level road features extracted in step 2 into the high-level feature optimization module AFOM to optimize and enhance the high-level road features and remove redundant information to obtain the optimized high-level road features. The high-level feature optimization module AFOM includes a convolutional attention module CBAM and multiple feature enhancement modules including a dilated spatial convolutional pooling pyramid module (ASPP), a strip dilation convolution module, and a strip pooling module. The specific method is as follows:

[0081] Step 3.1 The advanced feature optimization module (AFOM) includes a convolutional attention module (CBAM) and multiple feature enhancement modules, wherein the multiple feature enhancement modules include a dilated spatial convolutional pooling pyramid module (ASPP), a strip dilation convolution module, and a strip pooling module; the high-level feature vector obtained by the encoder is input into the convolutional attention module CBAM to remove redundant information, wherein the convolutional attention module CBAM includes a channel attention module CAM and a spatial attention module SAM. The channel attention module CAM calculates the importance weight of each channel through global average pooling and a fully connected layer. The spatial attention module SAM aggregates spatial dimension information through deep convolution and uses a convolution layer to learn the weight of each position;

[0082] high′=SAM(CAM(high)), (2)

[0083] Among them, CAM(·) represents the channel attention mechanism, SAM(·) represents the spatial attention mechanism, and high′ is the high-level feature vector after the convolutional attention module;

[0084] Step 3.2 uses the atrous spatial convolution pooling pyramid module (ASPP), the strip dilation convolution module, and the strip pooling module to enable the model to obtain a more accurate representation of road features:

[0085] In step 3.2.1, the high-level feature vector high′ obtained by the convolutional attention module in step 3.1 is further extracted and integrated through the atrous spatial convolutional pooling pyramid module (ASPP). This module extracts multi-scale road information to enrich the perception of road details and enhance the understanding of the global structure of the road network. The atrous spatial convolutional pooling pyramid module (ASPP) uses four convolution kernels with different apertures to capture road features at different scales:

[0086]

[0087] Here, aspp1(·)-aspp4(·) represent convolution operations with different expansion coefficients, where the expansion coefficients are 1, 6, 12, and 18 respectively. Indicates concat operation, high′ ASPP Represents the high-level feature vector after ASPP operation;

[0088] Step 3.2.2: The high-level feature vector high′ obtained from the convolutional attention module is further extracted and integrated through the strip dilation convolution module and the strip pooling module:

[0089]

[0090] Here, AvgPoolH(·) represents horizontal average pooling, AvgPoolV(·) represents vertical average pooling, stripH(·) represents horizontal dilated convolution, stripV(·) represents vertical dilated convolution, stripDL(·) represents diagonal dilated convolution, and stripDR(·) represents anti-diagonal dilated convolution, where the dilation coefficient is 6. Indicates concat operation, high′ strip Represents the high-level feature vector after strip-dilated convolution and strip-pooling operations;

[0091] Step 3.2.3: Concatenate the high-level feature vectors from the dilated spatial convolution pooling pyramid module ASPP, the strip dilation convolution module, and the strip pooling module to obtain the optimized high-level features:

[0092]

[0093] here, represents the concat operation, and "high" represents the high-level feature vector optimized by the Advanced Feature Optimization Module (AFOM).

[0094] In step 4, the two layers of low-level features of the road obtained in step 2 (the first layer and the second layer of low-level features) and the optimized high-level features of the road obtained in step 3 are input into the decoder for optimization and fusion, the image resolution is restored, and the extracted road network map is finally output.

[0095] Specifically, the first-level low-level features of the road obtained by the second-level encoder are respectively passed through the variable convolution module, and the second-level low-level features of the road obtained by the third-level encoder are passed through the attention refinement module ARM to enhance the expressiveness of the low-level features; the optimized first and second-level low-level features are fused with the optimized high-level features, and the similarity between the extracted road network graph and the label is compared by setting the loss function, and the overall model is trained by backpropagation; finally, a fully connected layer with a softmax activation function is added after the asymmetric road extraction network model trained by backpropagation, and a fully supervised method is used to train the network to achieve road network extraction. The specific method is as follows:

[0096] Step 4.1: The first layer of low-level feature vectors obtained in step 2 are passed through a variable convolution module to obtain low-level feature vectors adapted to different scales:

[0097] low' l1 =bn_relu(deformConv(low l1 )), (6)

[0098] Here, deformConv(·) represents a variable convolution operation, bn_relu(·) represents a batch normalization operation on the first layer feature vector after the variable convolution operation and then a relu activation function, low′ l1 Represents the first layer of low-level feature vector after variable convolution operation;

[0099] Step 4.2: The second-layer low-level feature vector obtained in step 2 is passed through the attention refinement module ARM to obtain a more accurate low-level feature vector that ignores redundant features:

[0100] low' l2 =low l2 ×sigmoid(conv(GAP(low l2 ))), (7)

[0101] Here, GAP(·) represents the global average pooling operation, conv(·) represents the 1×1 convolution operation, sigmoid(·) represents the activation function, and low′ l2 Represents the second-layer low-level feature vector after the attention refinement module (ARM);

[0102] Step 4.3: The high-level features obtained after step 3 are optimized by the high-level feature optimization module AFOM are subjected to convolution and upsampling operations to obtain a more accurate high-level feature vector:

[0103] high”′=upsample(conv_bn_relu(high”)), (8)

[0104] Here, conv_bn_relu(·) represents convolution, normalization, and relu function activation of high' to extract more accurate feature representation, upsample(·) represents bilinear interpolation operation on the feature, and high'' represents the optimized high-level feature vector finally obtained;

[0105] Step 4.4 concatenates the feature vectors obtained from steps 4.1 to 4.3, and uses the final convolutional layer to enhance the feature representation and improve the generalization ability of the model:

[0106]

[0107] Here, conv_bn_relu(·) indicates that the convolution, normalization and relu function activation of the concatenated features are performed for the final feature fusion, upsample(·) indicates that the feature vector is subjected to bilinear interpolation, and sigmoid(·) indicates the activation function. Represents the concat operation, and output represents the final output road network extraction graph;

[0108] Step 4.5 optimizes the overall model using the combined loss function softDice and BCE, where softDice focuses on predicting the overlap between the road network graph and the label; BCE focuses on distinguishing between road and non-road areas:

[0109]

[0110] Here, y true represents the annotation label of the real image, Represents the road label predicted by the model. Smooth represents a small smoothing constant to prevent the loss function from dividing by zero and causing errors. Here, it is set to 1e-6. By restoring the image resolution, the extracted road network diagram is finally output.

[0111] Through experimental analysis, we fine-tune model hyperparameters. We use the publicly available DeepGlobe dataset. We perform preprocessing on high-resolution remote sensing imagery, including image segmentation, edge filling, rotation, and horizontal and vertical migration, effectively expanding the sample dataset.

[0112] The processed dataset is fed into a high-resolution remote sensing imagery road network extraction model based on an asymmetric neural network. To extract richer road features, after the encoder extracts low-level and high-level road features, the low-level features are optimized using a variable convolution module and an attention refinement module (ARM). High-level features are refined and de-redundant using a convolutional attention module (CBAM), a dilated spatial convolutional pooling pyramid module (ASPP), a striped dilated convolution module, and a striped pooling module. This allows the optimized high-level features to take into account both the multi-scale information and striped characteristics of the road. The optimized low-level and high-level features are then concatenated and finally optimized using a three-layer convolution operation. This approach helps the model learn better road feature representations for road network extraction.

[0113] The operating system used in the experiment is Ubuntu 22.04.1, and the deep learning framework used is PyTorch. The specific configurations involved in the experiment are shown in Table 1.

[0114] Table 1 Experimental configuration table

[0115]

[0116] Through a series of exhaustive ablation experiments, the effectiveness of each innovative submodule proposed in this paper has been fully verified, significantly enhancing the feasibility and performance of this invention. The following is an overview of these key experimental results:

[0117] Table 2 demonstrates the performance advantages of our base network on the DeepGlobe dataset. Specifically, in comparative experiments with existing UNet models, DeepLabv3+ models, and RS-Mamba models, our asymmetric base network model performs particularly well on the DeepGlobe dataset. Comparing four key evaluation metrics—recall, precision, intersection over union (IoU), and F1 score—our model achieves the best results in IoU and F1, two indicators that reflect the model's overall performance. This highlights the superiority of our model for segmenting road networks in remote sensing imagery.

[0118] Table 2. Comparative experimental results of base networks

[0119]

[0120] Table 3 lists the specific impact of each module added in steps 3 and 4 on model performance. Comprehensive ablation analysis shows that the model combining striped convolution, striped pooling, variable convolution, attention mechanism, and ARM modules achieves optimal performance on the three key metrics of Precision, IoU, and F1, confirming the effectiveness and necessity of these modules in improving model performance.

[0121] Table 3 Ablation experiment results in the high-level feature extraction module

[0122]

[0123] Table 4 lists the impact of different attention mechanisms on model performance. SAM means using global average pooling and convolution operations to generate spatial attention maps, which are applied to feature maps to highlight important areas. CAM means using global average pooling and fully connected layers to generate channel attention weights, which are then element-wise multiplied by the original feature maps to enhance the features of important channels. FSFF means integrating feature maps from different stages of the encoder to utilize semantic and structural information at different levels to enhance the model's ability to express global and local features. CBAM means combining SAM and CAM, first highlighting important feature channels through the channel attention module, and then further focusing on the key spatial positions in these channels through the spatial attention module to achieve fine-grained weighting of features. Experimental results show that the CBAM attention mechanism performs better in the two comprehensive indicators of IoU and F1, indicating that this attention mechanism can more effectively identify and strengthen key features in the model and more finely adjust the model's attention to features, thereby improving the accuracy and robustness of the model when processing complex data.

[0124] Table 4 Comparative experimental results of attention mechanism algorithms

[0125]

[0126] The experimental data in Table 5 clearly demonstrates the specific impact of different encoders on model performance. In these experiments, the model using ResNet34 as the encoder demonstrated excellent performance on four key evaluation metrics: Recall, Precision, Intersection over Union (IoU), and F1 score. This demonstrates the effectiveness of the ResNet34 encoder in this model and highlights its potential for improving model performance.

[0127] Table 5 Encoder comparison experimental results

[0128]

[0129] The ablation experiment results in Table 6 show the specific impact of the number of decoder layers on model performance. In existing models, the commonly used UNet architecture usually has symmetrical encoder and decoder layers. However, the findings of this experiment show that the model with a 2-layer decoder can achieve better performance than the symmetrical encoder-decoder structure. This result suggests that in this model, an asymmetric decoder layer configuration may be more helpful in improving the efficiency and accuracy of the model;

[0130] Table 6 Decoder number comparison experimental results

[0131]

[0132]

[0133] The data in Table 7 demonstrates that our model demonstrates superior performance across multiple key evaluation metrics when compared to state-of-the-art models from the past three years. Specifically, our model achieves a significant 8.27% improvement in Recall compared to the recently published DFC-UNet model. Despite a slight decrease in Precision, our model outperforms the DFC-UNet model in both IoU and F1 scores, two comprehensive performance metrics. This result fully demonstrates the effectiveness and superiority of our model in processing complex remote sensing image data.

[0134] Table 7 Performance comparison of different models

[0135]

[0136] It can be seen from the above simulation experiments and tables that compared with the existing technology, the present invention demonstrates significant advantages and innovations in the field of remote sensing image segmentation through the innovative application of an asymmetric base network model, an advanced feature extraction module, an attention mechanism, optimized selection of encoder and decoder layers, and performance comparison with the latest models.

Claims

1. A method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network, characterized in that: The specific steps include: Step 1: Obtain high-resolution road remote sensing image data to be trained: Use public network resources to obtain high-resolution remote sensing images containing roads and perform preprocessing operations, including image segmentation, edge filling, rotation, horizontal and vertical migration, to effectively expand the sample dataset; Step 2: Using a multi-layer pre-trained ResNet34 model as an encoder, we gradually acquire two layers of low-level features and high-level features of the road. The low-level features include road edges and road surface texture features, while the high-level features include the overall shape of the road and the spatial relationship of the road network. Step 3: Input the high-level road features extracted in step 2 into the advanced feature optimization module AFOM to optimize and enhance the high-level road features and remove redundant information to obtain the optimized high-level road features. Step 3.1 The advanced feature optimization module AFOM includes a convolutional attention module CBAM and multiple feature enhancement modules, wherein the multiple feature enhancement modules include a void spatial convolution pooling pyramid module ASPP, a strip dilation convolution module, and a strip pooling module; the high-level feature vector obtained by the encoder in step 2 is input into the convolutional attention module CBAM to remove redundant information, wherein the convolutional attention module CBAM includes a channel attention module CAM and a spatial attention module SAM. The channel attention module CAM calculates the importance weight of each channel through global average pooling and a fully connected layer. The spatial attention module SAM aggregates spatial dimension information through deep convolution and uses a convolution layer to learn the weight of each position; high'=SAM(CAM(high)), (1) Among them, CAM(·) represents the channel attention mechanism, SAM(·) represents the spatial attention mechanism, and high' is the high-level feature vector after the convolutional attention module; Step 3.2 uses the atrous spatial convolution pooling pyramid module (ASPP), the strip dilation convolution module, and the strip pooling module to enable the model to obtain a more accurate representation of road features: In step 3.2.1, the high-level feature vector high' obtained by the convolutional attention module in step 3.1 is further extracted and integrated through the atrous spatial convolutional pooling pyramid module ASPP. This module extracts multi-scale road information to enrich the perception of road details and enhance the understanding of the global structure of the road network. The atrous spatial convolutional pooling pyramid module ASPP uses four convolution kernels with different apertures to capture road features at different scales: Here, aspp1(·)-aspp4(·) represent convolution operations with different expansion coefficients, where the expansion coefficients are 1, 6, 12, and 18 respectively. Indicates concat operation, high' ASPP Represents the high-level feature vector after ASPP operation; Step 3.2.2: The high-level feature vector high' obtained from the convolutional attention module is further extracted and integrated through the strip dilation convolution module and the strip pooling module: Here, AvgPoolH(·) represents horizontal average pooling, AvgPoolV(·) represents vertical average pooling, stripH(·) represents horizontal dilated convolution, stripV(·) represents vertical dilated convolution, stripDL(·) represents diagonal dilated convolution, and stripDR(·) represents anti-diagonal dilated convolution, where the dilation coefficient is 6. Indicates concat operation, high' strip Represents the high-level feature vector after strip-dilated convolution and strip-pooling operations; Step 3.2.3: Concatenate the high-level feature vectors from the dilated spatial convolution pooling pyramid module ASPP, the strip dilation convolution module, and the strip pooling module to obtain the optimized high-level features: here, represents the concat operation, and "high" represents the high-level feature vector optimized by the Advanced Feature Optimization Module (AFOM); In step 4, the two-layer low-level features of the road obtained in step 2 and the optimized high-level features of the road obtained in step 3 are input into the decoder for optimization and fusion to obtain the optimized overall model. The optimized overall model is represented by the comprehensive loss function softDice and BCE. By restoring the image resolution, the extracted road network diagram is finally output.

2. The method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network according to claim 1, wherein: The step 2 includes: First, a multi-layer pre-trained model, ResNet34, is used as the encoder. This encoder progressively extracts road features from the input image through one layer of convolution, one layer of maximum pooling, and four layers of residual convolution blocks. The first two layers of residual convolution blocks extract low-level features such as road edges and road surface texture, while the last two layers of residual convolution blocks extract high-level features such as the overall road shape and spatial relationships of the road network. Among them, input represents the preprocessed training data set, conv_bn_relu(·) represents convolution, normalization and relu function activation of the input to extract preliminary feature representation, maxpool(·) represents the maximum pooling operation on the convolved features to reduce the spatial dimension of the feature map and reduce the amount of calculation, encoder1-encoder4 represent the residual convolution blocks layer1-layer4 of ResNet34 respectively, and the generated low l1 、low l2 , high represent the two layers of low-level feature vectors and high-level feature vectors obtained by the encoder respectively.

3. The method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network according to claim 1, wherein: The specific method of step 4 is: Step 4.1: The first layer of low-level feature vectors obtained in step 2 is passed through a variable convolution module to obtain low-level feature vectors adapted to different scales: low′ l1 =bn_relu(deformConv(low l1 )), (6) Here, deformConv(·) represents a variable convolution operation, bn_relu(·) represents a batch normalization operation on the first layer of low-level feature vectors after the variable convolution operation and then a relu activation function, low' l1 Represents the first layer of low-level feature vector after variable convolution operation; Step 4.2: The second low-level feature vector of the two low-level feature vectors obtained in step 2 is passed through the attention refinement module ARM to obtain a more accurate low-level feature vector that ignores redundant features: low’ l2 =low l2 ×sigmoid(conv(GAP(low l2 ))), (7) Here, GAP(·) represents the global average pooling operation, conv(·) represents the 1×1 convolution operation, sigmoid·) represents the activation function, low' l2 Represents the second-layer low-level feature vector after the attention refinement module ARM; Step 4.3: The high-level features obtained after step 3 are optimized by the high-level feature optimization module AFOM are subjected to convolution and upsampling operations to obtain a more accurate high-level feature vector: high”'=upsample(conv_bn_relu(high”)), (8) Here, conv_bn_relu(·) represents convolution, normalization, and relu function activation of high' to extract more accurate feature representation, upsample(·) represents bilinear interpolation operation on the feature, and high'' represents the optimized high-level feature vector finally obtained; Step 4.4 concatenates the feature vectors obtained from steps 4.1 to 4.3, and uses the final convolutional layer to enhance the feature representation and improve the generalization ability of the model, obtaining an optimized overall model: Here, conv_bn_relu(·) indicates that the convolution, normalization and relu function activation of the concatenated features are performed for the final feature fusion, upsample(·) indicates that the feature vector is subjected to bilinear interpolation, and sigmoid(·) indicates the activation function. Represents the concat operation, and output represents the final output road network extraction graph; Step 4.5 uses the combined loss function softDice and BCE to represent the optimized overall model, where softDice focuses on predicting the overlap between the road network graph and the label; BCE focuses on distinguishing between road and non-road areas: Here, y true represents the annotation label of the real image, Represents the road label predicted by the model, smooth represents a small smoothing constant to prevent the loss function from dividing by zero and causing errors. By restoring the image resolution, the extracted road network diagram is finally output.

4. A high-resolution remote sensing image road network extraction device based on an asymmetric neural network, characterized in that: include: A memory and a processor, wherein the memory stores a computer program so that the processor executes the high-resolution remote sensing image road network extraction method based on an asymmetric neural network as described in any one of claims 1 to 3.

5. A program storage medium for receiving user input, characterized in that: When the stored computer program is executed by a processor, it can obtain an accurate road network map based on the high-resolution remote sensing image road network extraction method based on an asymmetric neural network as described in any one of claims 1 to 3.

Images