High-resolution remote sensing image road network extraction method, system and device based on asymmetric neural network and medium

Abstract

The invention discloses a high-resolution remote sensing image road network extraction method, system and device based on an asymmetric neural network, and a medium. The method comprises the following steps: firstly, obtaining to-be-trained high-resolution road remote sensing image data and carrying out preprocessing; secondly, through a multi-layer pre-training model ResNet34, gradually obtaining two-layer low-level features and high-level features of the road from the preprocessed road image data by using an encoder; inputting the road advanced features extracted by the encoder into an advanced feature optimization module AFOM to optimize and enhance the road advanced features, and removing redundant features to obtain optimized road advanced features; and finally, optimizing and fusing the two-layer low-level features of the road and the optimized high-level features of the road by using a decoder to obtain an optimized overall model, comprehensively representing the optimized overall model by using loss functions softDice and BCE, and finally outputting an extracted road network diagram by recovering the image resolution. According to the system, the device and the medium, a road in a high-resolution remote sensing image is identified and segmented based on a high-resolution remote sensing image road network extraction method of an asymmetric neural network; the method has the advantages of high precision and good robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of road extraction based on high-resolution remote sensing images, and in particular to a method, system, equipment and medium for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network. Background Art

[0002] Road networks play a key role in urban and rural infrastructure, and are essential for driving economic growth and improving the quality of life of residents. With the development of remote sensing technology, especially the application of high-resolution remote sensing image technology, image data has become more refined, providing sufficient data sources for automatic detection and in-depth analysis of road networks, making automatic identification and extraction of road networks more feasible.

[0003] However, although high-resolution images can represent more detailed surface features, since each pixel covers a small area of ​​the ground, more pixels are needed to represent the same surface area, and the amount of data will increase significantly, which not only requires higher computing power, but also puts higher demands on algorithm design.

[0004] Road network extraction methods have gone through three stages of development, including methods based on morphological features, manual features, and deep learning. The first two traditional methods usually take a lot of time and rely on manual interpretation, so the accuracy of these methods is limited. With the continuous development and improvement of deep learning technology, road extraction methods based on this technology are also constantly being optimized and improved. However, the complex ground object information of high-resolution remote sensing images introduces a lot of noise, such as trees and building occlusions beside the road, and interference from vehicles, etc., coupled with the existence of the same object with different spectra and the same spectrum with different objects, which greatly increases the difficulty of road network extraction.

[0005] The patent application discloses a method for extracting roads from high-resolution remote sensing images based on a deformable attention mechanism (CN118072155A). First, the training data set is enhanced using sliding windows, horizontal flipping and vertical flipping operations. Then, a road enhancement module is formed using one-dimensional convolutions in four directions: horizontal, vertical, left diagonal and right diagonal. A 1×1 convolution module is used to constrain the learning of the deformable offset in the self-attention mechanism to form a deformable attention module that automatically adapts to the geometric transformation of the road. ResNet50 encoder and bilinear interpolation upsampling decoder are selected as the basic model, and a deformable attention module and a road enhancement module are embedded in the middle to extract multi-scale road information. The road centerline is extracted, and the extracted centerline is broken and reconnected to improve the connectivity of the road centerline. Finally, the optimized road network centerline is converted from raster to vector to generate a vector road network. This application uses a deformable attention mechanism to extract the road centerline. The high computational complexity of this method requires high computing resources, and the extracted road centerline cannot intuitively reflect the width of each part of the road network, and cannot adapt to road scenes with variable widths, which is not conducive to the generalization ability of the model. Summary of the invention

[0006] In order to overcome the problems of the above-mentioned prior art, the purpose of the present invention is to propose a method, system, device and medium for extracting road network from high-resolution remote sensing images based on an asymmetric neural network, and develop a new road network extraction model by combining an asymmetric network structure, multiple attention mechanisms and multiple feature enhancement modules. It uses an encoder to gradually acquire the primary low-level features and high-level features of the road, uses a high-level feature optimization module to optimize the high-level feature representation of the road, uses a variable convolution module and an attention refinement module in the decoder to optimize the low-level feature representation of the road, and fuses the optimized low-level features and high-level features, which has the advantages of low complexity and high accuracy.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network is disclosed. First, the high-resolution road remote sensing image data to be trained is obtained and preprocessed. Secondly, the encoder is used to gradually obtain two layers of low-level features of the road from the preprocessed road image data through a multi-layer pre-training model ResNet34. The high-level features of the road extracted by the encoder are input into the high-level feature optimization module AFOM to optimize and enhance them, and redundant features are removed to obtain the optimized high-level features of the road. Finally, the decoder is used to optimize and fuse the two layers of low-level features of the road and the optimized high-level features of the road to obtain an optimized overall model. The optimized overall model is represented by a comprehensive loss function of softDice and BCE. By restoring the image resolution, the extracted road network diagram is finally output.

[0009] A method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network specifically comprises the following steps:

[0010] Step 1: Obtain high-resolution road remote sensing image data to be trained: Use public network resources to obtain high-resolution remote sensing images containing roads, and perform preprocessing operations, including image segmentation, edge filling, rotation, horizontal and vertical migration, to effectively expand the sample data set;

[0011] Step 2, using the multi-layer pre-trained model ResNet34 as an encoder to gradually obtain two layers of low-level features and high-level features of the road, the low-level features include road edges and road surface texture features, and the high-level features include the overall shape of the road and the spatial relationship of the road network;

[0012] Step 3, inputting the high-level road features extracted in step 2 into the high-level feature optimization module AFOM to optimize and enhance the high-level road features, remove redundant information, and obtain optimized high-level road features;

[0013] In step 4, the two-layer low-level features of the road obtained in step 2 and the optimized high-level features of the road obtained in step 3 are input into the decoder for optimization and fusion to obtain the optimized overall model. The optimized overall model is represented by the comprehensive loss function softDice and BCE. By restoring the image resolution, the extracted road network diagram is finally output.

[0014] The step 2 comprises:

[0015] First, a multi-layer pre-trained model ResNet34 is used as an encoder. The encoder gradually extracts road features in the input image through 1 layer of convolution, 1 layer of maximum pooling, and 4 layers of residual convolution blocks. The first two layers of residual convolution blocks extract low-level features of road edges and road surface texture features, and the last two layers of residual convolution blocks extract high-level features of the overall shape of the road and the spatial relationship of the road network:

[0016]

[0017] Among them, input represents the preprocessed training data set, conv_bn_relu(·) represents convolution, normalization and relu function activation of the input to extract preliminary feature representation, maxpool(·) represents the maximum pooling operation on the convolutional features to reduce the spatial dimension of the feature map and reduce the amount of calculation, encoder1-encoder4 respectively represent the residual convolution blocks layer1-layer4 of ResNet34, and the generated low l1 、low l2, high represent the two layers of low-level feature vectors and high-level feature vectors obtained by the encoder respectively.

[0018] The specific method of step 3 is:

[0019] Step 3.1 The high-level feature optimization module AFOM includes a convolutional attention module CBAM and multiple feature enhancement modules, wherein the multiple feature enhancement modules include a dilated spatial convolutional pooling pyramid module ASPP, a strip dilation convolution module, and a strip pooling module; the high-level feature vector obtained by the encoder in step 2 is input into the convolutional attention module CBAM to remove redundant information, wherein the convolutional attention module CBAM includes a channel attention module CAM and a spatial attention module SAM, the channel attention module CAM calculates the importance weight of each channel through global average pooling and a fully connected layer, and the spatial attention module SAM aggregates spatial dimension information through deep convolution and then uses a convolutional layer to learn the weight of each position;

[0020] high′=SAM(CAM(high)), (2)

[0021] Among them, CAM(·) represents the channel attention mechanism, SAM(·) represents the spatial attention mechanism, and high′ is the high-level feature vector after the convolutional attention module;

[0022] Step 3.2: Through the atrous spatial convolution pooling pyramid module ASPP, the strip dilation convolution module and the strip pooling module, the model obtains a more accurate road feature representation:

[0023] Step 3.2.1: The high-level feature vector high′ passed through the convolutional attention module in step 3.1 is further extracted and integrated through the atrous spatial convolutional pooling pyramid module ASPP to extract multi-scale information of the road to enrich the perception of road details and enhance the understanding of the global structure of the road network; the atrous spatial convolutional pooling pyramid module ASPP captures road features of different scales through 4 convolution kernels with different apertures:

[0024]

[0025] Here, aspp1(·)-aspp4(·) represent convolution operations with different expansion coefficients, where the expansion coefficients are 1, 6, 12, and 18 respectively. Indicates concat operation, high′ ASPP Represents the high-level feature vector after ASPP operation;

[0026] Step 3.2.2: The high-level feature vector high′ after the convolutional attention module is further extracted and integrated through the strip dilation convolution module and the strip pooling module:

[0027]

[0028] Here, AvgPoolH(·) represents horizontal average pooling, AvgPoolV(·) represents vertical average pooling, stripH(·) represents horizontal dilated convolution, stripV(·) represents vertical dilated convolution, stripDL(·) represents diagonal dilated convolution, and stripDR(·) represents anti-diagonal dilated convolution, where the dilation coefficient is 6. Indicates concat operation, high′ strip Represents the high-level feature vector after strip-dilated convolution and strip-pooling operations;

[0029] Step 3.2.3 concatenates the high-level feature vectors of the dilated spatial convolutional pooling pyramid module ASPP, the strip dilation convolution module, and the strip pooling module to obtain the optimized high-level features:

[0030]

[0031] here, represents the concat operation, and "high" represents the high-level feature vector optimized by the Advanced Feature Optimization Module (AFOM).

[0032] The specific method of step 4 is:

[0033] Step 4.1 The first layer of low-level feature vectors in the two layers of low-level feature vectors obtained in step 2 is passed through a variable convolution module to obtain low-level feature vectors adapted to different scales:

[0034] low' l1 =bn_relu(deformConv(low l1 )), (6)

[0035] Here, deformConv(·) represents a variable convolution operation, bn_relu(·) represents a batch normalization operation on the first layer of low-level feature vectors after the variable convolution operation and then a relu activation function, low′ l1 Represents the first layer of low-level feature vector after variable convolution operation;

[0036] Step 4.2: The second layer of low-level feature vectors in the two layers of low-level feature vectors obtained in step 2 is passed through the attention refinement module ARM to obtain a more accurate low-level feature vector that ignores redundant features:

[0037] low' l2 =low l2×sigmoid(conv(GAP(low l2 ))), (7)

[0038] Here, GAP(·) represents the global average pooling operation, conv(·) represents the 1×1 convolution operation, sigmoid(·) represents the activation function, and low′ l2 Represents the second-layer low-level feature vector after the attention refinement module ARM;

[0039] Step 4.3: The advanced features obtained by the advanced feature optimization module AFOM in step 3 are subjected to convolution and upsampling operations to obtain a more accurate advanced feature vector:

[0040] high”′=upsample(conv_bn_relu(high”)), (8)

[0041] Here, conv_bn_relu(·) means convolution, normalization and relu function activation of high” to extract more accurate feature representation, upsample(·) means bilinear interpolation operation on the feature, and high”' means the optimized high-level feature vector finally obtained;

[0042] Step 4.4 concatenates the feature vectors obtained from step 4.1 to step 4.3, and uses the final convolutional layer to enhance the feature representation and improve the generalization ability of the model to obtain an optimized overall model:

[0043]

[0044] Here, conv_bn_relu(·) indicates that the concatenated features are convolved, normalized, and relu function activated for final feature fusion, upsample(·) indicates that the feature vector is bilinearly interpolated, and sigmoid(·) indicates the activation function. represents the concat operation, and output represents the final output road network extraction graph;

[0045] Step 4.5 uses the comprehensive loss function softDice and BCE to represent the optimized overall model, where softDice focuses on predicting the overlap between the road network graph and the label; BCE focuses on distinguishing between road and non-road areas:

[0046]

[0047] Here, y true represents the annotation label of the real image, Represents the road label predicted by the model, smooth represents a small smoothing constant to prevent the loss function from dividing by zero and causing errors. By restoring the image resolution, the extracted road network diagram is finally output.

[0048] A method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network, comprising:

[0049] The encoder module is used in step 2 to gradually obtain two layers of low-level features and high-level features of the road by using the multi-layer pre-trained model ResNet34 as an encoder, wherein the low-level features include road edges and road surface texture features, and the high-level features include the overall shape of the road and the spatial relationship of the road network;

[0050] The advanced feature optimization module AFOM is used in step 3 to input the advanced road features extracted in step 2 into the advanced feature optimization module AFOM to optimize and enhance the advanced road features and remove redundant information to obtain optimized advanced road features;

[0051] The decoder module is used in step 4. The two-layer low-level features of the road obtained in step 2 and the optimized high-level features of the road obtained in step 3 are input into the decoder for optimization and fusion to obtain the optimized overall model. The optimized overall model is represented by the comprehensive loss function softDice and BCE. By restoring the image resolution, the extracted road network diagram is finally output.

[0052] A high-resolution remote sensing image road network extraction device based on an asymmetric neural network comprises: a memory and a processor, wherein the memory stores a computer program so that the processor executes the high-resolution remote sensing image road network extraction method based on an asymmetric neural network.

[0053] A storage medium for receiving user input program, wherein when the stored computer program is executed by a processor, an accurate road network diagram can be obtained based on the high-resolution remote sensing image road network extraction method based on an asymmetric neural network.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. Low-level feature and high-level feature extraction: The present invention uses the pre-trained model ResNet34 as the encoder and selects 5 layers for feature extraction, in which the feature vectors extracted by the middle two layers are used as low-level features, and the output of the last layer is used as high-level features. The two features are optimized separately to enhance the feature representation.

[0056] 2. High-level feature optimization module: The present invention refines and removes redundancy of the initial high-level features passed through the encoder through the convolutional attention module CBAM, the atrous spatial convolution pooling pyramid module ASPP, the strip dilation convolution module and the strip pooling module, so that the optimized high-level features can take into account the multi-scale information and strip characteristics of the road.

[0057] 3. Decoder module: In the decoder module, variable convolution, attention refinement and convolution operations are used to further optimize the low-level and high-level information of the road. The optimized low-level features focus more on the precise contextual information of the road. The high-level features further optimized by convolution are concatenated with the optimized low-level features and then subjected to three layers of convolution operations again to more effectively extract the features of the road and improve the model's accurate reconstruction of the road.

[0058] 4. Road network extraction with asymmetric network architecture: A new asymmetric network framework has been developed for road network extraction based on high-resolution remote sensing images. Combining technologies such as attention mechanism, strip dilation convolution, strip pooling, and variable convolution, the low-level and high-level feature representations of roads are optimized and integrated, which enhances the accuracy of the model for road network extraction without increasing the time complexity too much.

[0059] 5. Verification experiments and performance advantages: Through sufficient ablation experiments, the effectiveness of the proposed innovative submodules is proved, which strengthens the feasibility and performance of the invention. Through comparative experiments with existing UNet-based methods, DeepLabv3+-based methods and Mamba-based methods, the present invention demonstrates performance advantages on the DeepGlobe dataset, which means that the present model achieves better results in road network extraction.

[0060] The present invention introduces an innovative road network extraction method, which improves the accuracy and robustness of road network extraction through a road network segmentation method of low-level-high-level semantic feature extraction and feature fusion.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] 1. Effective solution to the problem of road discontinuity caused by occlusion of objects around the road: Compared with the existing high-resolution remote sensing image road extraction method based on deformable attention mechanism (CN118072155A) that uses one-dimensional convolution in four directions, the present invention uses ASPP, strip dilation convolution in four directions and strip pooling in two directions in the high-level feature optimization module to optimize the high-level features. Through these modules, the striping features of the road can be better captured and the multi-scale information of the road can be obtained through dilation convolution. These feature information is crucial to road connectivity.

[0063] 2. Lower time complexity: Compared with the existing high-resolution remote sensing image road extraction method based on deformable attention mechanism (CN118072155A), which adopts deformable attention mechanism, the present invention only adopts variable convolution after the first layer of low-level features without adding attention mechanism, which reduces the time complexity to a certain extent.

[0064] 3. High efficiency of asymmetric network structure: The present invention adopts an asymmetric network structure design, which reduces the overall complexity of the model while maintaining high-precision extraction performance. This design reduces the demand for computing resources and improves the operating efficiency of the model, making it more suitable for processing large-scale remote sensing image data sets.

[0065] 4. Optimized feature processing: The Advanced Feature Optimization Module (AFOM) further optimizes and enhances the advanced features extracted by the encoder, removes unnecessary redundant features, and retains the most critical information for road recognition. This step significantly improves the expressiveness of features and the generalization ability of the model, especially in a changing road environment. The decoder fuses the optimized high-level features with the low-level features, and this method achieves multi-dimensional integration of information. This fusion strategy not only enhances the model's ability to capture road details, but also improves the understanding of complex road network structures, thereby improving model accuracy.

[0066] 5. Potential for wide application: Using a large-scale DeepGlobe dataset for training, the model can learn more complex terrain environments. The pre-trained ResNet34 model effectively captures low-level to high-level road features, including edges, textures, shapes, and spatial relationships, and extracts multi-scale features and global context features through a variety of attention mechanisms and feature optimization modules, which provides rich information for accurate road recognition and segmentation, and is particularly suitable for road extraction in complex backgrounds. In view of the good robustness and generalization ability of the present invention, it has a wide range of application potential, including urban planning, traffic monitoring, disaster response, environmental monitoring, homeland security and other fields, and has broad application prospects.

[0067] The present invention focuses on the accurate extraction and optimization of road features. First, ResNet34 is used to capture road features from low-level to high-level. For low-level features, variable convolution and attention refinement modules are used for optimization to include more road context information and remove redundant information, so that the remaining features are highly correlated with the final road features; for high-level features, attention mechanism, ASPP module, strip dilation convolution module and strip pooling are used for optimization to enrich the road feature representation and retain highly correlated features; finally, the optimized low-level features and high-level features are spliced ​​together and the spliced ​​features are further optimized through three-layer convolution, and finally the road resolution is restored through bilinear upsampling to reconstruct the road network.

[0068] In general, the present invention can extract rich road features by combining the encoder, advanced feature optimization module and decoder, especially when dealing with complex terrain occlusion and shadow effects around the road, showing higher accuracy and efficiency. This method is not only universal in theory, but also shows significant performance advantages in practical applications.

[0069] In summary, the present invention adopts an asymmetric neural network combined with multiple feature optimization modules. This design is universal in different environments and does not require personalized adjustments for each data set. By optimizing both low-level and high-level features of the road, it can better represent the road features, improve the road segmentation capability in complex terrain scenes, achieve high-precision road segmentation results, and provide efficient and reliable solutions for applications in multiple fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 Extracting road flow graphs for asymmetric neural networks.

[0071] Figure 2 This is the overall architecture diagram of the high-resolution remote sensing image road network extraction model based on an asymmetric neural network.

[0072] Figure 3 This is the framework diagram of the Advanced Feature Optimization Module (AFOM).

[0073] Figure 4 This is the decoder attention refinement module (ARM) framework diagram. DETAILED DESCRIPTION

[0074] This example presents a method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network. Figure 1 The specific network structure framework adopted is as follows Figure 2 As shown:

[0075] Step 1: Obtain high-resolution road remote sensing image data to be trained: Use public network resources to obtain high-resolution remote sensing images containing roads, and perform preprocessing operations, including image segmentation, edge filling, rotation, horizontal and vertical migration, to effectively expand the sample data set;

[0076] Step 2: Use the multi-layer pre-trained model ResNet34 as an encoder to gradually obtain two layers of low-level features and high-level features of the road. The low-level features include road edges and road surface texture features, and the high-level features include the overall shape of the road and the spatial relationship of the road network. The specific method is:

[0077] First, a multi-layer pre-trained model ResNet34 is used as an encoder. The encoder gradually extracts road features in the input image through 1 layer of convolution, 1 layer of maximum pooling, and 4 layers of residual convolution blocks. The first two layers of residual convolution blocks extract low-level features of road edges and road surface texture features, and the last two layers of residual convolution blocks extract high-level features of the overall shape of the road and the spatial relationship of the road network:

[0078]

[0079] Among them, input represents the preprocessed training data set, conv_bn_relu(·) represents convolution, normalization and relu function activation of the input to extract preliminary feature representation, maxpool(·) represents the maximum pooling operation on the features after the first layer of convolution to reduce the spatial dimension of the feature map and reduce the amount of calculation, encoder1-encoder4 respectively represent the residual convolution blocks layer1-layer4 of ResNet34, and the generated low l1 、low l2 , high represent the two layers of low-level feature vectors and high-level feature vectors obtained by the encoder respectively.

[0080] Step 3: Input the high-level road features extracted in step 2 into the high-level feature optimization module AFOM to optimize and enhance the high-level road features and remove redundant information to obtain optimized high-level road features. The high-level feature optimization module AFOM includes a convolutional attention module CBAM and multiple feature enhancement modules including a dilated spatial convolutional pooling pyramid module (ASPP), a strip dilation convolution module, and a strip pooling module. The specific method is:

[0081] Step 3.1 The advanced feature optimization module (AFOM) includes a convolutional attention module (CBAM) and multiple feature enhancement modules, wherein the multiple feature enhancement modules include a dilated spatial convolutional pooling pyramid module ASPP, a strip dilation convolution module, and a strip pooling module; the advanced feature vector obtained by the encoder is input into the convolutional attention module CBAM to remove redundant information, wherein the convolutional attention module CBAM includes a channel attention module CAM and a spatial attention module SAM, the channel attention module CAM calculates the importance weight of each channel through global average pooling and a fully connected layer, and the spatial attention module SAM aggregates spatial dimension information through deep convolution and then uses a convolutional layer to learn the weight of each position;

[0082] high′=SAM(CAM(high)), (2)

[0083] Among them, CAM(·) represents the channel attention mechanism, SAM(·) represents the spatial attention mechanism, and high′ is the high-level feature vector after the convolutional attention module;

[0084] Step 3.2: Through the atrous spatial convolution pooling pyramid module ASPP, the strip dilation convolution module and the strip pooling module, the model obtains a more accurate road feature representation:

[0085] Step 3.2.1: The high-level feature vector high′ passed through the convolutional attention module in step 3.1 is further extracted and integrated through the atrous spatial convolutional pooling pyramid module ASPP to extract multi-scale information of the road to enrich the perception of road details and enhance the understanding of the global structure of the road network; the atrous spatial convolutional pooling pyramid module ASPP captures road features of different scales through 4 convolution kernels with different apertures:

[0086]

[0087] Here, aspp1(·)-aspp4(·) represent convolution operations with different expansion coefficients, where the expansion coefficients are 1, 6, 12, and 18 respectively. Indicates concat operation, high′ ASPP Represents the high-level feature vector after ASPP operation;

[0088] Step 3.2.2: The high-level feature vector high′ after the convolutional attention module is further extracted and integrated through the strip dilation convolution module and the strip pooling module:

[0089]

[0090] Here, AvgPoolH(·) represents horizontal average pooling, AvgPoolV(·) represents vertical average pooling, stripH(·) represents horizontal dilated convolution, stripV(·) represents vertical dilated convolution, stripDL(·) represents diagonal dilated convolution, and stripDR(·) represents anti-diagonal dilated convolution, where the dilation coefficient is 6. Indicates concat operation, high′ strip Represents the high-level feature vector after strip-dilated convolution and strip-pooling operations;

[0091] Step 3.2.3 concatenates the high-level feature vectors of the dilated spatial convolutional pooling pyramid module ASPP, the strip dilation convolution module, and the strip pooling module to obtain the optimized high-level features:

[0092]

[0093] here, represents the concat operation, and "high" represents the high-level feature vector optimized by the Advanced Feature Optimization Module (AFOM).

[0094] In step 4, the two layers of low-level features of the road obtained in step 2 (the first layer and the second layer of low-level features) and the optimized high-level features of the road obtained in step 3 are input into the decoder for optimization and fusion, the image resolution is restored, and finally the extracted road network diagram is output.

[0095] Specifically, the first-layer low-level features of the road obtained by the second-layer encoder are respectively passed through the variable convolution module, and the second-layer low-level features of the road obtained by the third-layer encoder are passed through the attention refinement module ARM to enhance the expressiveness of the low-level features; the optimized first and second-layer low-level features and the optimized high-level features are fused, and the similarity between the extracted road network graph and the label is compared through the setting of the loss function, and the overall model is trained by back propagation; finally, a fully connected layer with a Softmax activation function is added after the asymmetric road extraction network model trained by back propagation, and a fully supervised method is used to train the network to achieve road network extraction. The specific method is:

[0096] Step 4.1 The first layer of low-level feature vectors obtained in step 2 are passed through a variable convolution module to obtain low-level feature vectors adapted to different scales:

[0097] low' l1 =bn_relu(deformConv(low l1 )), (6)

[0098] Here, deformConv(·) represents a variable convolution operation, bn_relu(·) represents a batch normalization operation on the first layer feature vector after the variable convolution operation and then a relu activation function, low′ l1 Represents the first layer of low-level feature vector after variable convolution operation;

[0099] Step 4.2: The second-layer low-level feature vector obtained in step 2 is passed through the attention refinement module ARM to obtain a more accurate low-level feature vector that ignores redundant features:

[0100] low' l2 =low l2 ×sigmoid(conv(GAP(low l2 ))), (7)

[0101] Here, GAP(·) represents the global average pooling operation, conv(·) represents the 1×1 convolution operation, sigmoid(·) represents the activation function, and low′ l2 Represents the second-layer low-level feature vector after the attention refinement module (ARM);

[0102] Step 4.3: The advanced features obtained by the advanced feature optimization module AFOM in step 3 are subjected to convolution and upsampling operations to obtain a more accurate advanced feature vector:

[0103] high”′=upsample(conv_bn_relu(high”)), (8)

[0104] Here, conv_bn_relu(·) means convolution, normalization and relu function activation of high” to extract more accurate feature representation, upsample(·) means bilinear interpolation operation on the feature, and high”' means the optimized high-level feature vector finally obtained;

[0105] Step 4.4 concatenates the feature vectors obtained from step 4.1 to step 4.3, and enhances the feature representation and improves the generalization ability of the model through the final convolutional layer:

[0106]

[0107] Here, conv_bn_relu(·) indicates that the concatenated features are convolved, normalized, and relu function activated for final feature fusion, upsample(·) indicates that the feature vector is bilinearly interpolated, and sigmoid(·) indicates the activation function. represents the concat operation, and output represents the final output road network extraction graph;

[0108] Step 4.5 optimizes the overall model using the comprehensive representation of the loss function softDice and BCE, where softDice focuses on predicting the overlap between the road network graph and the label; BCE focuses on distinguishing between road and non-road areas:

[0109]

[0110] Here, y true represents the annotation label of the real image, Represents the road label predicted by the model, and smooth represents a small smoothing constant to prevent the loss function from dividing by zero and causing errors. Here, 1e-6 is taken. By restoring the image resolution, the extracted road network diagram is finally output.

[0111] Through experimental analysis, the model hyperparameters are optimized. The public dataset DeepGlobe is used. Preprocessing operations are performed on high-resolution remote sensing image data, including image segmentation, edge filling, rotation, horizontal and vertical migration, effectively expanding the sample dataset.

[0112] The processed data set is passed into the high-resolution remote sensing image road network extraction model based on an asymmetric neural network. In order to extract richer road features, after the encoder extracts the low-level and high-level features of the road, the low-level features are optimized using the variable convolution module and the attention refinement module ARM respectively, and the high-level features are refined and de-redundant using the convolution attention module CBAM, the atrous spatial convolution pooling pyramid module ASPP, the strip dilation convolution module, and the strip pooling module, so that the optimized high-level features can take into account the multi-scale information and strip characteristics of the road. After the optimized low-level and high-level features are optimized and spliced, they are finally optimized through a three-layer convolution operation. This method can help the model learn better road feature representation for road network extraction.

[0113] The operating system used in the experiment is Ubuntu 22.04.1, and the deep learning framework used is PyTorch. The specific configurations involved in the experiment are shown in Table 1.

[0114] Table 1 Experimental configuration table

[0115]

[0116] Through a series of detailed ablation experiments, the effectiveness of each innovative submodule proposed in the present invention has been fully verified, thereby significantly enhancing the feasibility and performance of the present invention. The following is an overview of these key experimental results:

[0117] Table 2 shows the performance advantages of the basic network of the present invention on the DeepGlobe dataset. Specifically, in the comparative experiments with the existing UNet model, DeepLabv3+ model and RS-Mamba model, the asymmetric base network model of the present invention performs particularly well on the DeepGlobe dataset. By comparing the four key evaluation indicators of recall, precision, intersection over union (IoU) and F1 score, the present model achieved the best results in IoU and F1, two indicators that reflect the comprehensive performance of the model, highlighting the superiority of the present model in segmenting road networks in remote sensing images;

[0118] Table 2. Base network comparison experimental results

[0119]

[0120] Table 3 lists the specific impact of each module added in step 3 and step 4 on the model performance. Through comprehensive ablation experimental analysis, the model integrating strip convolution, strip pooling, variable convolution, attention mechanism and ARM modules achieved the best performance in terms of three key indicators: Precision, IoU and F1, which proves the effectiveness and necessity of these modules in improving model performance.

[0121] Table 3 Ablation experiment results in high-level feature extraction module

[0122]

[0123] Table 4 lists the impact of different attention mechanisms on model performance. SAM means using global average pooling and convolution operations to generate spatial attention maps, which are applied to feature maps to highlight important areas; CAM means using global average pooling and fully connected layers to generate channel attention weights, which are then element-wise multiplied by the original feature maps to enhance the features of important channels; FSFF means integrating feature maps at different stages of the encoder to utilize semantic and structural information at different levels to enhance the model's ability to express global and local features; CBAM means combining SAM and CAM, first highlighting important feature channels through the channel attention module, and then further focusing on the key spatial positions in these channels through the spatial attention module to achieve fine-grained weighting of features; experimental results show that the use of the CBAM attention mechanism can perform better in the two comprehensive indicators of IoU and F1, indicating that the attention mechanism can more effectively identify and strengthen the key features in the model and more finely adjust the model's attention to features, thereby improving the accuracy and robustness of the model when processing complex data;

[0124] Table 4 Comparative experimental results of attention mechanism algorithms

[0125]

[0126] The experimental data in Table 5 clearly show the specific impact of different encoders on model performance. In these experiments, the model using ResNet34 as the encoder showed excellent performance in the four key evaluation indicators of Recall, Precision, IoU and F1 score, which proved the effectiveness of the ResNet34 encoder in this model and highlighted its potential in improving model performance;

[0127] Table 5 Encoder comparison experimental results

[0128]

[0129] The ablation experiment results in Table 6 show the specific impact of the number of decoder layers on model performance. In existing models, the commonly used UNet architecture usually has symmetrical encoder and decoder layers. However, the findings of this experiment show that the model with a 2-layer decoder can achieve better performance than the symmetrical encoder-decoder structure. This result shows that in this model, an asymmetric decoder layer configuration may be more helpful in improving the efficiency and accuracy of the model;

[0130] Table 6 Decoder number comparison experimental results

[0131]

[0132]

[0133] The data in Table 7 show that the model of the present invention has demonstrated excellent performance in multiple key evaluation indicators when compared with the advanced models in the past three years. Specifically, compared with the newly published DFC-UNet model, the present model has achieved a significant improvement of 8.27% in the Recall value. Although there is a slight decrease in the Precision value, the present model is superior to the DFC-UNet model in terms of the two comprehensive performance indicators of IoU and F1 score. This result fully demonstrates the effectiveness and superiority of the model of the present invention in processing complex remote sensing image data.

[0134] Table 7 Performance comparison of different models

[0135]

[0136] It can be seen from the above simulation experiments and tables that compared with the prior art, the present invention demonstrates significant advantages and innovations in the field of remote sensing image segmentation through an asymmetric base network model, an advanced feature extraction module, innovative application of the attention mechanism, optimized selection of encoder and decoder layers, and performance comparison with the latest models.

Claims

1. A method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network, characterized in that: First, the high-resolution road remote sensing image data to be trained is obtained and preprocessed; secondly, the encoder is used to gradually obtain two layers of low-level features of the road including road edges and road surface texture features, and high-level features including the overall shape of the road and the spatial relationship of the road network from the preprocessed road image data through the multi-layer pre-training model ResNet34; The high-level road features extracted by the encoder are input into the high-level feature optimization module AFOM to optimize and enhance them, and remove redundant features to obtain optimized high-level road features; Finally, the decoder is used to optimize and fuse the two-layer low-level features of the road and the optimized high-level features of the road to obtain the optimized overall model. The optimized overall model is represented by the comprehensive loss function softDice and BCE. By restoring the image resolution, the extracted road network graph is finally output.

2. The method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network according to claim 1, characterized in that: The specific steps include: Step 1: Obtain high-resolution road remote sensing image data to be trained: Use public network resources to obtain high-resolution remote sensing images containing roads, and perform preprocessing operations, including image segmentation, edge filling, rotation, horizontal and vertical migration, to effectively expand the sample data set; Step 2, using the multi-layer pre-trained model ResNet34 as an encoder to gradually obtain two layers of low-level features and high-level features of the road, the low-level features include road edges and road surface texture features, and the high-level features include the overall shape of the road and the spatial relationship of the road network; Step 3, inputting the high-level road features extracted in step 2 into the high-level feature optimization module AFOM to optimize and enhance the high-level road features, remove redundant information, and obtain optimized high-level road features; In step 4, the two-layer low-level features of the road obtained in step 2 and the optimized high-level features of the road obtained in step 3 are input into the decoder for optimization and fusion to obtain the optimized overall model. The optimized overall model is represented by the comprehensive loss function softDice and BCE. By restoring the image resolution, the extracted road network diagram is finally output.

3. The method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network according to claim 2, characterized in that: The step 2 comprises: First, a multi-layer pre-trained model ResNet34 is used as an encoder. The encoder gradually extracts road features in the input image through 1 layer of convolution, 1 layer of maximum pooling, and 4 layers of residual convolution blocks. The first two layers of residual convolution blocks extract low-level features of road edges and road surface texture features, and the last two layers of residual convolution blocks extract high-level features of the overall shape of the road and the spatial relationship of the road network: Among them, input represents the preprocessed training data set, conv_bn_relu(·) represents convolution, normalization and relu function activation of the input to extract preliminary feature representation, maxpool(·) represents the maximum pooling operation on the convolutional features to reduce the spatial dimension of the feature map and reduce the amount of calculation, encoder1-encoder4 respectively represent the residual convolution blocks layer1-layer4 of ResNet34, and the generated low l1 、low l2 , high represent the two layers of low-level feature vectors and high-level feature vectors obtained by the encoder respectively.

4. The method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network according to claim 2, characterized in that: The specific method of step 3 is: Step 3.1 The high-level feature optimization module AFOM includes a convolutional attention module CBAM and multiple feature enhancement modules, wherein the multiple feature enhancement modules include a dilated spatial convolutional pooling pyramid module ASPP, a strip dilation convolution module, and a strip pooling module; the high-level feature vector obtained by the encoder in step 2 is input into the convolutional attention module CBAM to remove redundant information, wherein the convolutional attention module CBAM includes a channel attention module CAM and a spatial attention module SAM, the channel attention module CAM calculates the importance weight of each channel through global average pooling and a fully connected layer, and the spatial attention module SAM aggregates spatial dimension information through deep convolution and then uses a convolutional layer to learn the weight of each position; high′=SAM(CAM(high)), (2) Among them, CAM(·) represents the channel attention mechanism, SAM(·) represents the spatial attention mechanism, and high′ is the high-level feature vector after the convolutional attention module; Step 3.2: Through the atrous spatial convolution pooling pyramid module ASPP, the strip dilation convolution module and the strip pooling module, the model obtains a more accurate road feature representation: Step 3.2.1: The high-level feature vector high′ passed through the convolutional attention module in step 3.1 is further extracted and integrated through the atrous spatial convolutional pooling pyramid module ASPP to extract multi-scale information of the road to enrich the perception of road details and enhance the understanding of the global structure of the road network; the atrous spatial convolutional pooling pyramid module ASPP captures road features of different scales through 4 convolution kernels with different apertures: Here, aspp1(·)-aspp4(·) represent convolution operations with different expansion coefficients, where the expansion coefficients are 1, 6, 12, and 18 respectively. Indicates concat operation, high′ ASPP Represents the high-level feature vector after ASPP operation; Step 3.2.2: The high-level feature vector high′ after the convolutional attention module is further extracted and integrated through the strip dilation convolution module and the strip pooling module: Here, AvgPoolH(·) represents horizontal average pooling, AvgPoolV(·) represents vertical average pooling, stripH(·) represents horizontal dilated convolution, stripV(·) represents vertical dilated convolution, stripDL(·) represents diagonal dilated convolution, and stripDR(·) represents anti-diagonal dilated convolution, where the dilation coefficient is 6. Indicates concat operation, high′ strip Represents the high-level feature vector after strip-dilated convolution and strip-pooling operations; Step 3.2.3 concatenates the high-level feature vectors of the dilated spatial convolutional pooling pyramid module ASPP, the strip dilation convolution module, and the strip pooling module to obtain the optimized high-level features: here, represents the concat operation, and "high" represents the high-level feature vector optimized by the Advanced Feature Optimization Module (AFOM).

5. The method for extracting road network from high-resolution remote sensing images based on an asymmetric neural network according to claim 2, characterized in that: The specific method of step 4 is: Step 4.1 The first layer of low-level feature vectors in the two layers of low-level feature vectors obtained in step 2 is passed through a variable convolution module to obtain low-level feature vectors adapted to different scales: low′ l1 =bn_relu(deformConv(low l1 )), (6) Here, deformConv(·) represents a variable convolution operation, bn_relu(·) represents a batch normalization operation on the first layer of low-level feature vectors after the variable convolution operation and then a relu activation function, low′ l1 Represents the first layer of low-level feature vector after variable convolution operation; Step 4.2: The second layer of low-level feature vectors in the two layers of low-level feature vectors obtained in step 2 is passed through the attention refinement module ARM to obtain a more accurate low-level feature vector that ignores redundant features: low′ l2 =low l2 ×sigmoid(conv(GAP(low l2 ))), (7) Here, GAP(·) represents the global average pooling operation, conv(·) represents the 1×1 convolution operation, sigmoid(·) represents the activation function, and low′ l2 Represents the second-layer low-level feature vector after the attention refinement module ARM; Step 4.3: The advanced features obtained by the advanced feature optimization module AFOM in step 3 are subjected to convolution and upsampling operations to obtain a more accurate advanced feature vector: high”′=upsample(conv_bn_relu(high”)), (8) Here, conv_bn_relu(·) indicates that high” is convolved, normalized and relu activated to extract more accurate feature representation, upsample(·) indicates that bilinear interpolation is performed on the feature, and high”′ indicates the optimized high-level feature vector finally obtained; Step 4.4 concatenates the feature vectors obtained from step 4.1 to step 4.3, and uses the final convolutional layer to enhance the feature representation and improve the generalization ability of the model to obtain an optimized overall model: Here, conv_bn_relu(·) indicates that the concatenated features are convolved, normalized, and relu function activated for final feature fusion, upsample(·) indicates that the feature vector is bilinearly interpolated, and sigmoid(·) indicates the activation function. represents the concat operation, and output represents the final output road network extraction graph; Step 4.5 uses the comprehensive loss function softDice and BCE to represent the optimized overall model, where softDice focuses on predicting the overlap between the road network graph and the label; BCE focuses on distinguishing between road and non-road areas: Here, y true represents the annotation label of the real image, Represents the road label predicted by the model, smooth represents a small smoothing constant to prevent the loss function from dividing by zero and causing errors. By restoring the image resolution, the extracted road network diagram is finally output.

6. A method for extracting road networks from high-resolution remote sensing images based on an asymmetric neural network, characterized in that: include: The encoder module is used in step 2 to gradually obtain two layers of low-level features and high-level features of the road by using the multi-layer pre-trained model ResNet34 as an encoder, wherein the low-level features include road edges and road surface texture features, and the high-level features include the overall shape of the road and the spatial relationship of the road network; The advanced feature optimization module AFOM is used in step 3 to input the advanced road features extracted in step 2 into the advanced feature optimization module AFOM to optimize and enhance the advanced road features and remove redundant information to obtain optimized advanced road features; The decoder module is used in step 4. The two-layer low-level features of the road obtained in step 2 and the optimized high-level features of the road obtained in step 3 are input into the decoder for optimization and fusion to obtain the optimized overall model. The optimized overall model is represented by the comprehensive loss function softDice and BCE. By restoring the image resolution, the extracted road network diagram is finally output.

7. A high-resolution remote sensing image road network extraction device based on an asymmetric neural network, characterized in that: include: A memory and a processor, wherein the memory stores a computer program so that the processor executes the high-resolution remote sensing image road network extraction method based on an asymmetric neural network.

8. A program storage medium for receiving user input, characterized in that: When the stored computer program is executed by a processor, it is possible to obtain an accurate road network diagram based on the high-resolution remote sensing image road network extraction method based on an asymmetric neural network.

Images