Remote sensing image road extraction method and system based on multi-scale context sensing network

Abstract

The invention discloses a remote sensing image road extraction method and system based on a multi-scale context-aware network, and belongs to the technical field of image processing.The method comprises the following steps that firstly, a branch with a context aggregation module is designed in an encoder to enhance context information extraction; and the problem of semantic fuzziness caused by shielding is relieved. Secondly, in order to solve the problem that semantic information between a coder and a decoder is not matched and improve the spatial information extraction capability of a model, a spatial feature extraction module is introduced into jump connection, and a method of directly copying coder features in SwinUnet is replaced. And finally, designing a feature contraction module in a down-sampling stage to reduce information loss in an encoder and enhance the segmentation capability of the network. Compared with an existing remote sensing image road segmentation method, the remote sensing image road segmentation method has the advantages that the precision and the performance of remote sensing image road extraction are effectively improved through global modeling, context enhancement and information matching optimization.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and relates to a remote sensing image road extraction method and system based on a multi-scale context-aware network. Background Art

[0002] Existing remote sensing image road extraction technologies are mainly divided into traditional image segmentation methods and deep learning-based image segmentation methods.

[0003] Traditional image segmentation methods usually extract roads based on some manually designed features and thresholds. For example, edge detection algorithms determine the boundaries of roads by detecting edges in images; region growing algorithms start from seed points and gradually merge similar regions to extract roads. However, traditional methods have low accuracy in road extraction under complex backgrounds. In actual remote sensing images, there are often a variety of interference factors, such as buildings, trees, shadows, etc. These complex backgrounds make it difficult for traditional methods to accurately identify roads. In addition, with the continuous development of remote sensing technology, the demand for high-resolution remote sensing images is increasing, and traditional methods often face problems such as large computational complexity and low efficiency when processing high-resolution images, which makes it difficult to meet the needs of practical applications.

[0004] In recent years, deep learning methods based on convolutional neural networks have made significant progress in remote sensing image processing. Deep learning models can automatically learn features in images, thereby effectively improving the accuracy of road extraction. For example, some methods based on fully convolutional neural networks can directly classify images at the pixel level to achieve road extraction. However, deep learning methods still have problems such as loss of local feature information and insufficient extraction of spatial information. In convolutional neural networks, as the number of network layers increases, local feature information may gradually be lost, resulting in inaccurate description of details. At the same time, the convolution operation mainly focuses on local areas, and the extraction of spatial information is not sufficient, which affects the integrity and accuracy of road extraction. Summary of the invention

[0005] The purpose of the present invention is to solve the problems of insufficient accuracy and loss of local features in road extraction under complex backgrounds and occlusions in the prior art, and to provide a remote sensing image road extraction method and system based on a multi-scale context-aware network.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention proposes a remote sensing image road extraction method based on a multi-scale context-aware network, comprising:

[0008] Acquire remote sensing image road data;

[0009] Swin Transformer is used as the backbone network to build a multi-scale context-aware network;

[0010] The remote sensing image road data is input into the multi-scale context-aware network to obtain the pixel-level features of the remote sensing image road and realize the remote sensing image road extraction.

[0011] Preferably, the multi-scale context-aware network includes an encoder and a decoder, the encoder and the decoder are connected to each other through a skip connection, the encoder includes a multi-scale context aggregation module, a Swin Transformer module, a linear embedding layer and a feature shrinkage module, and the decoder includes a spatial feature extraction module and a linear expansion layer;

[0012] The road data of the remote sensing image is divided into blocks by the encoder and converted into feature vectors through the linear embedding layer. The feature vectors are then passed layer by layer to the multi-scale context aggregation module, the Swin Transformer module and the feature contraction module for feature extraction. After passing through the bottleneck module, the features enter the decoder and are upsampled through the linear expansion layer. At the same time, feature fusion is combined with jump connections to restore the resolution of the feature map and output the final road pixel-level features.

[0013] Preferably, the multi-scale context aggregation module includes a global context aggregation branch and a local context processing branch;

[0014] The global context aggregation branch first performs a maximum pooling operation on the feature map F in Downsampling is performed, and then the pooled feature map is passed to the full attention module. The full attention module calculates the attention weights on the channel or space, and after calculating the global context features of the full attention module, outputs context information with global features;

[0015] The local context processing branch first transforms the feature map F in It is divided into multiple local blocks, and each local block is passed to the full attention module. The full attention module executes the attention mechanism within the local block, captures local features through the correlation between pixels, and outputs contextual information with local features.

[0016] Preferably, the spatial feature extraction module integrates a dilated spatial pyramid pooling and a convolutional block attention module;

[0017] The convolutional block attention module consists of a channel attention module and a spatial attention module, which respectively pay attention to the channel and spatial dimensions of the input feature map F. First, the feature map F is input into the channel attention module to obtain the intermediate feature map F. 中 , the intermediate feature map F 中Multiply it by the feature map F to obtain a new feature map F1; then, input the new feature map F1 into the spatial attention module; the output intermediate feature map is multiplied by the new feature map F1 to obtain the final feature map F2.

[0018] Preferably, in the channel attention module, the input feature map is compressed in channel and spatial dimensions through average pooling and maximum pooling operations;

[0019] In the spatial attention module, the channel information of the feature map F′ is aggregated through average pooling and maximum pooling operations to generate the average pooled feature map F Avg And the maximum pooling feature map F Max , and then concatenate and convolve through standard convolutional layers to generate a spatial attention map M s (F1); The standard convolutional layer includes convolution, normalization, and ReLU activation functions.

[0020] Preferably, the feature shrinkage module includes a global feature branch and a local feature branch;

[0021] The global feature branch extracts global features through maximum pooling, soft pooling and average pooling operations and performs full connection operations; the local feature branch uses 1×1 convolution and convolution operations with a stride of 2 to reduce the resolution of the feature map.

[0022] Preferably, the Swin Transformer module uses a shift window multi-head self-attention mechanism SW-MSA to achieve information exchange between different windows by introducing a shift operation between windows.

[0023] The present invention proposes a remote sensing image road extraction system based on a multi-scale context-aware network, comprising:

[0024] A parameter acquisition module, wherein the parameter acquisition module is used to acquire remote sensing image road data;

[0025] A model building module, wherein the model building module is used to use Swin Transformer as a backbone network to build a multi-scale context-aware network;

[0026] The feature extraction module is used to input the remote sensing image road data into a multi-scale context-aware network to obtain the pixel-level features of the remote sensing image road and realize remote sensing image road extraction.

[0027] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of a remote sensing image road extraction method based on a multi-scale context-aware network are implemented.

[0028] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a remote sensing image road extraction method based on a multi-scale context-aware network.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention proposes a method for extracting roads from remote sensing images based on a multi-scale context-aware network. The multi-scale context-aware network is constructed using Swin Transformer as the backbone network, which makes full use of its ability to capture long-range dependencies and significantly enhances the extraction effect of roads in complex backgrounds. The multi-scale context aggregation module in the multi-scale context-aware network is embedded in the encoder, which effectively solves the problem of decreased road extraction accuracy caused by occlusion and complex background. By introducing a spatial feature extraction module in the jump connection, the network further improves the efficiency of semantic information transmission between the encoder and the decoder, especially when processing small-scale roads. The feature shrinkage module reduces the loss of information in the downsampling process by combining multiple pooling operations and convolutional layers, enhances the prediction accuracy of edge pixels, and thus ensures the precise positioning of road boundaries. Overall, the present invention shows significant advantages in improving the accuracy and robustness of road extraction from remote sensing images.

[0031] The invention proposes a remote sensing image road extraction system based on a multi-scale context-aware network. By dividing the system into a parameter acquisition module, a model building module and a feature extraction module, the pixel-level features of the remote sensing image road are obtained to achieve remote sensing image road extraction. The modularization concept is adopted to make each module independent of each other, which is convenient for unified management of each module. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0033] Figure 1 This is a flow chart of the remote sensing image super-resolution reconstruction method based on cross-scale cosine attention of the present invention.

[0034] Figure 2 It is a schematic diagram of the CSwinUnet network structure of the present invention.

[0035] Figure 3 Schematic diagram of the structure of the multi-scale context aggregation module of the present invention.

[0036] Figure 4 It is a schematic diagram of the structure of the spatial feature extraction module of the present invention.

[0037] Figure 5 It is a schematic diagram of the characteristic shrink module structure of the present invention.

[0038] Figure 6 This is a diagram of the remote sensing image super-resolution reconstruction system based on cross-scale cosine attention of the present invention.

[0039] Figure 7 The figure is a schematic diagram of the structure of an electronic device of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0041] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0043] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0044] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0045] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0046] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0047] The present invention proposes a method for extracting roads from remote sensing images based on a multi-scale context-aware network. Figure 1 As shown, including:

[0048] S1. Acquire remote sensing image road data;

[0049] S2, using Swin Transformer as the backbone network to build a multi-scale context-aware network;

[0050] The multi-scale context-aware network is constructed, which includes an encoder and a decoder. The encoder and the decoder are connected to each other through a skip connection. The encoder includes a multi-scale context aggregation module, a Swin Transformer module, a linear embedding layer and a feature shrinkage module, and the decoder includes a spatial feature extraction module and a linear expansion layer.

[0051] The road data of the remote sensing image is divided into blocks by the encoder and converted into feature vectors through the linear embedding layer. The feature vectors are then passed layer by layer to the multi-scale context aggregation module, the Swin Transformer module and the feature contraction module for feature extraction. After passing through the bottleneck module, the features enter the decoder and are upsampled through the linear expansion layer. At the same time, feature fusion is combined with jump connections to restore the resolution of the feature map and output the final road pixel-level features.

[0052] The multi-scale context aggregation module includes a global context aggregation branch and a local context processing branch;

[0053] The global context aggregation branch first performs a maximum pooling operation on the feature map F in Downsampling is performed, and then the pooled feature map is passed to the full attention module. The full attention module calculates the attention weights on the channel or space, and after calculating the global context features of the full attention module, outputs context information with global features;

[0054] The local context processing branch first transforms the feature map F inIt is divided into multiple local blocks, and each local block is passed to the full attention module. The full attention module executes the attention mechanism within the local block, captures local features through the correlation between pixels, and outputs contextual information with local features.

[0055] The spatial feature extraction module integrates the dilated spatial pyramid pooling and convolutional block attention module;

[0056] The convolutional block attention module consists of a channel attention module and a spatial attention module, which respectively pay attention to the channel and spatial dimensions of the input feature map F. First, the feature map F is input into the channel attention module to obtain the intermediate feature map F. 中 , the intermediate feature map F 中 Multiply it by the feature map F to obtain a new feature map F1; then, input the new feature map F1 into the spatial attention module; the output intermediate feature map is multiplied by the new feature map F1 to obtain the final feature map F2.

[0057] In the channel attention module, the input feature map is compressed in channel and spatial dimensions through average pooling and maximum pooling operations;

[0058] In the spatial attention module, the channel information of the feature map F′ is aggregated through average pooling and maximum pooling operations to generate the average pooled feature map F Avg And the maximum pooling feature map F Max , and then concatenate and convolve through standard convolutional layers to generate a spatial attention map M s (F1); The standard convolutional layer includes convolution, normalization, and ReLU activation functions.

[0059] The feature shrinkage module includes a global feature branch and a local feature branch;

[0060] The global feature branch extracts global features through maximum pooling, soft pooling and average pooling operations and performs full connection operations; the local feature branch uses 1×1 convolution and convolution operations with a stride of 2 to reduce the resolution of the feature map.

[0061] S3. Input the remote sensing image road data into the multi-scale context-aware network to obtain the pixel-level features of the remote sensing image road and realize the remote sensing image road extraction.

[0062] The Swin Transformer module uses the shift window multi-head self-attention mechanism SW-MSA, which enables information exchange between different windows by introducing shift operations between windows.

[0063] The following is a detailed description of this method:

[0064] The present invention makes the following improvements on the basis of SwinUnet: 1) the input remote sensing image is divided into multiple 4×4 non-overlapping image blocks, and converted into feature vectors through a linear embedding layer; 2) Swin Transformer is used as an encoder. This module can effectively capture the local and global long-distance dependencies in the image through its unique shift window multi-head self-attention mechanism SW-MSA; 3) A multi-scale context aggregation module MSCAM is embedded in the encoder to enhance the extraction of context information, and the semantic ambiguity caused by occlusion is alleviated through pooling operations and attention mechanisms; 4) A spatial feature extraction module SFEM is introduced in the jump connection, and the problem of semantic information mismatch is solved by using the hole convolution and residual connection structure to enhance the extraction of spatial features; 5) The feature contraction module FCM is used in the downsampling process to reduce information loss through multiple pooling operations, ensure the accuracy of edge positioning and reduce noise points; 6): The resolution of the feature map is restored through the upsampling operation in the decoder to generate a pixel-level road extraction result.

[0065] Based on the multi-scale context-aware network MCSwinUnet, Swin Transformer is used as the backbone network, and its ability to capture long-range dependencies is used to enhance the extraction effect of roads in complex backgrounds; MSCAM is embedded in the encoder to solve the problem of reduced road extraction accuracy caused by occlusion and complex background. SFEM further improves the efficiency of semantic information transmission between the encoder and decoder by adding skip connections, and effectively improves the ability to segment small-scale roads; the feature shrinkage module reduces the loss of information in the encoding process through multiple pooling operations, enhances the prediction accuracy of edge pixels, and ensures the precise positioning of road boundaries; road feature extraction uses Swin Transformer and embedded MSCAM modules for downsampling.

[0066] The construction of a multi-scale context-aware network consists of two parts: the encoder and the decoder. The encoder includes a multi-scale context aggregation module, a Swin Transformer module, a linear embedding layer, and a feature contraction module, and the decoder includes a spatial feature extraction module and a linear expansion layer. The encoder is responsible for extracting features from the input image. The decoder and the encoder have a symmetrical structure and restore the spatial resolution of the image by gradually upsampling. Each stage in the decoder contains a Patch expansion module and a spatial feature extraction module (SFEM), which help map the features extracted by the encoder back to the high-resolution image space.

[0067] The encoder and decoder are connected to each other through skip connections, which helps to retain more spatial information during the decoding process and alleviate the loss of semantic information. The skip connection directly passes the features in the encoder to the corresponding stage of the decoder through the SFEM module, which improves the extraction and positioning accuracy of road details. Located between the encoder and the decoder, it consists of two consecutive Swin Transformer blocks. The bottleneck module captures long-range dependencies in the image while maintaining feature compactness, ensuring effective modeling of the integrity of the road structure. The image is first divided into small blocks by the encoder and converted into feature vectors through a linear embedding layer, and then passed to MSCAM, Swin Transformer and FCM layer by layer for feature extraction. After passing through the bottleneck module, the features enter the decoder, are upsampled through the Patch expansion module, and feature fusion is performed by combining the features of the skip connection. The decoder end restores the resolution of the feature map through a linear expansion layer, and finally outputs the final prediction result through a linear projection layer.

[0068] The multi-scale context aggregation module includes a global context aggregation branch and a local context processing branch;

[0069] The input of the global context aggregation branch is the feature map F in , which first performs a maximum pooling operation on F in Downsampling is performed to remove redundant information and retain global context features. Max pooling can effectively retain F in The pooled feature map is then passed to the full attention module (FA-Block), which aggregates the correlation between distant pixels in the feature map by calculating the attention weights in channels or space. This process ensures that the model can capture the long-distance dependencies between different regions in the image, which helps to extract global information such as the overall structure and macro layout of the road. After calculating the global context features of FA-Block, the output of this branch is context information with global features. The global context information is calculated through the full attention mechanism and is defined as:

[0070]

[0071] Among them, A i,j is the attention weight between the i-th pixel and the j-th pixel, Q i is the global context feature map Q i ∈R (H+W)×C×S , K j is the feature map K j ∈R (H+W)×C×S , c is the number of channels.

[0072] The local context processing branch receives the same input feature map F in , but the approach is different. First, it converts F in The local context processing branch is divided into multiple local blocks, and each block is passed to the FA-Block module. FA-Block performs a similar attention mechanism within the block to capture local features through the correlation between pixels. This processing method pays more attention to details and can finely capture the edges, textures and other details of the road, thereby improving the perception of subtle structures such as roads. The output of the local context processing branch is feature information rich in local details.

[0073] A spatial feature extraction module (SFEM) is introduced in the skip connection. The spatial feature extraction module (SFEM) adopts the combination of atrous spatial pyramid pooling (ASPP) and convolutional block attention module (CBAM) to enhance the model's ability to capture spatial information.

[0074] The CBAM module consists of a channel attention module (CAM) and a spatial attention module (SAM), which respectively pay attention to the channel and spatial dimensions of the input feature map. First, the feature map F is input into the CAM to obtain the intermediate feature map F. 中 , the intermediate feature map F 中 Multiply it by the feature map F to get the feature map F1. Then, input the feature map F1 into SAM; multiply the output intermediate feature map by the feature map F1 to get the final feature map F2. The main process is as follows:

[0075]

[0076] In CAM, the input feature map is compressed in channel and spatial dimensions through average pooling (AvgPool) and maximum pooling (MaxPool) operations:

[0077] M c (F)=σ(MLP(Avgpool(F))+MLP(Maxpool(F)))

[0078] Where M c (F) represents channel attention map, σ represents sigmoid function, F represents feature map, Avgpool(F) represents global average pooling, Maxpool(F) represents global maximum pooling, and MLP represents multi-layer perceptron.

[0079] In SAM, the channel information of the feature map is aggregated through AvgPool and MaxPool operations to generate two feature maps F Avg and F Max , and then connected and convolved through standard convolutional layers (including convolution, normalization and ReLU activation function) to generate the spatial attention map Ms (F1). The processing process is as follows:

[0080] M s (F1)=σ(f 7×7 ([Avgpool(F1)); Maxpool(F1)]))

[0081] Among them, M s (F1) represents the spatial attention map, f 7×7 represents a convolutional layer with a 7×7 kernel. σ represents the sigmoid function, F represents the feature map, Avgpool(F) represents the global average pooling, and Maxpool(F) represents the global maximum pooling.

[0082] The feature contraction module (FCM) combines multiple pooling operations and convolutional layers to reduce the loss of detail information during downsampling and improve the positioning accuracy of road edges. Specifically, FCM includes two parallel paths: one path uses average pooling (AvgPool) and maximum pooling (MaxPool) to calculate the statistical features of the feature map in the channel dimension through the global branch, and introduces the SoftPool operation to calculate the global weight, thereby retaining more key information during downsampling. The other path uses 1×1 convolution and convolution operations with a stride of 2 through the local branch to reduce the resolution of the feature.

[0083] In summary, MCSwinUnet adopts Swin Transformer as the backbone network and makes full use of its ability to capture long-range dependencies, which significantly enhances the extraction effect of roads in complex backgrounds. The MSCAM module is embedded in the encoder, which effectively solves the problem of reduced road extraction accuracy caused by occlusion and complex background. By introducing the spatial feature extraction module (SFEM) in the jump connection, the network further improves the efficiency of semantic information transmission between the encoder and the decoder, especially when processing small-scale roads. The feature contraction module (FCM) reduces the information loss in the downsampling process by combining multiple pooling operations and convolutional layers, enhances the prediction accuracy of edge pixels, and thus ensures the precise positioning of road boundaries. Overall, the present invention shows significant advantages in improving the accuracy and robustness of road extraction in remote sensing images. It has the following advantages: 1) It can more effectively capture global and local information in remote sensing images, especially in complex scenes, such as when roads are occluded or the background is complex. The present invention significantly improves the accuracy of road extraction, reduces information loss, and enhances the robustness of the model. 2) It can effectively simulate the long-distance spatial dependency between pixels, overcoming the localization problem that traditional convolutional neural networks are prone to when processing high-resolution images, thereby improving the global modeling capability of road structures. 3) It can effectively process high-resolution images without significantly increasing the computational complexity, ensuring the efficiency and adaptability of the model.

[0084] like Figure 2 The following is a schematic diagram of the CSwinUnet network structure. This structure shows a typical encoder-decoder structure, which is often used in image segmentation tasks. The entire network consists of three main modules: encoder, bottleneck layer and decoder. First, the size of the input image is That is, the height is H, the width is W, and the number of channels is 48. After the PatchPartition operation, the input image is divided into several small blocks, and the feature map is reduced to The height and width are reduced to one-fourth of the original size, and the number of channels is C. In the encoder part, the divided feature map is first passed through the linear embedding module and mapped to a high-dimensional space to capture richer features, maintaining the size of Next, the feature map passes through the Multi-Scale Channel Attention Module (MSCAM) and the Swin Transformer Module (STB) to further extract multi-scale contextual information and global features. The MSCAM module aggregates contextual information of different scales through the attention mechanism, while the STB module uses the Swin Transformer to focus on the global feature expression of the image. In this process, the spatial size of the data remains unchanged, but the feature expression is gradually enhanced after several layers of processing. Next, the data passes through the Global Context Module (FCM) to further extract the dependencies and global information between distant pixels. In the final stage of the encoder, the size of the feature map is reduced to, the height and width are halved again, and the number of channels is increased to 4 times the original. Next, entering the bottleneck layer, the feature map is further deeply processed by the STB module at this stage. The size of the feature map is maintained at At this time, the feature map carries rich semantic information and contains the global features of the image.

[0085] In the decoder part, the feature map is gradually upsampled through the Patch Expanding module to restore its original resolution layer by layer. After each block expansion operation, the spatial size of the feature map will double to gradually restore the spatial details of the image. At the same time, at each stage of the decoder, the selective feature enhancement module (SFEM) is combined with the information of the jump connection in the encoder to achieve effective fusion of global context and local features. Finally, after being upsampled layer by layer in the decoder, the feature map is restored to the spatial size of Hx W xC. In the final linear projection layer, the output size generated is H x W x Class, where Class represents the number of categories in image segmentation and is used to make classification predictions for each pixel.

[0086] like Figure 3The figure shows the structure of the multi-scale context aggregation module MSCAM, which shows an attention-based context block that fuses multi-scale information. The input features first pass through three adaptive convolution blocks (ACB) of different scales, labeled S1, S2, and S3, which represent different scales or feature extraction levels. The ACB block is used for multi-scale feature extraction or attention weighting operations, allowing the model to focus on important features at different levels of abstraction. The outputs from the three ACB blocks are spliced ​​together (indicated by "C" in the figure), combining features from different scales. The spliced ​​result passes through a 1x1 convolution layer, which may be used for feature compression or further feature refinement. Finally, the processed features are passed to the output layer to form the final output of the model. The lower part shows the internal structure of the ACB block. First, the input feature map undergoes a pooling operation and then passes to the FA block (full attention block), which applies a channel or spatial attention mechanism on the feature map to enhance the model's capture of important features. Next, the feature map passes through the pixel attention (PA) module to further focus on pixel-level feature details. After pixel attention processing, the feature maps are concatenated and passed to a 3x3 convolutional layer, which is used to further process and refine the feature maps.

[0087] like Figure 4 As shown in the figure, it is a schematic diagram of the structure of the spatial feature extraction module SFEM of the present invention, which shows the combination of atrous spatial pyramid pooling (ASPP) and convolutional block attention module (CBAM). ASPP improves the segmentation performance by adopting atrous convolution and parallel multi-branch structure to expand the effective field of view of the convolution kernel and enhance the contextual semantic information. However, too large a dilation ratio may lead to the loss of local feature information. Therefore, the dilated convolution selects a smaller dilation ratio (1, 2, 4, 6) to better capture road features of different scales and enhance the context perception ability of the network. At the same time, ASPP draws on the residual structure of ResNet

[27] , so that the network can retain the underlying features while learning new features, prevent feature loss and reduce the risk of gradient explosion or disappearance during deep network training. After passing through ASPP, the input image is passed to CBAM, which integrates the channel attention module (CAM) and the spatial attention module (SAM) to gradually refine the feature map. It infers the attention map separately and sequentially through the channel and spatial dimensions.

[0088] like Figure 5As shown, it is a schematic diagram of the feature contraction module FCM structure of the present invention, which shows a multi-branch neural network module structure. First, the input feature map F undergoes a reshaping operation to adjust its shape to a suitable input format, usually including width w, height h and number of channels c. Next, the feature map passes through a 1×1 convolution layer with a step size of 1 for preliminary feature extraction. Subsequently, it passes through a 3×3 convolution layer with a step size of 2 for further feature downsampling and extraction. After that, it passes through a 1×1 convolution layer with a step size of 1 for feature integration. Next, the network is divided into three parallel pooling branches, namely average pooling (AvgPool), maximum pooling (MaxPool) and soft pooling (SoftPool), and each pooling operation is connected to a fully connected layer (FC). After the outputs of these branches are fused with corresponding features (such as weighted summation or other fusion operations), they continue to pass through a 1×1 convolution layer to further process the output features. Finally, the output feature map undergoes a reshaping operation again to obtain the final output L. The entire module realizes the fusion of multiple pooling methods through this multi-branch structure to improve the richness of feature extraction and network performance.

[0089] Example 2

[0090] The present invention proposes a remote sensing image road extraction system based on a multi-scale context-aware network. Figure 6 As shown, including:

[0091] A parameter acquisition module, wherein the parameter acquisition module is used to acquire remote sensing image road data;

[0092] A model building module, wherein the model building module is used to use Swin Transformer as a backbone network to build a multi-scale context-aware network;

[0093] The feature extraction module is used to input the remote sensing image road data into a multi-scale context-aware network to obtain the pixel-level features of the remote sensing image road and realize remote sensing image road extraction.

[0094] Example 3

[0095] See also Figure 7 As shown, the present invention also provides an electronic device 100 for a remote sensing image road extraction method based on a multi-scale context-aware network; the electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on the at least one processor 102, and at least one communication bus 104.

[0096] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the remote sensing image road extraction method based on a multi-scale context-aware network described in Example 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101. The memory 101 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data (such as audio data) created according to the use of the electronic device 100, etc. In addition, the memory 101 can include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0097] The at least one processor 102 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 102 may be a microprocessor or any conventional processor, etc. The processor 102 is the control center of the electronic device 100, and uses various interfaces and lines to connect various parts of the entire electronic device 100.

[0098] The memory 101 in the electronic device 100 stores a plurality of instructions to implement a remote sensing image road extraction method based on a multi-scale context-aware network, and the processor 102 can execute the plurality of instructions to implement:

[0099] Acquire remote sensing image road data;

[0100] Swin Transformer is used as the backbone network to build a multi-scale context-aware network;

[0101] The remote sensing image road data is input into the multi-scale context-aware network to obtain the pixel-level features of the remote sensing image road and realize the remote sensing image road extraction.

[0102] Example 4

[0103] If the module / unit integrated in the electronic device 100 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory and read-only memory (ROM, Read-Only Memory).

[0104] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0106] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0108] Therefore, the solution proposed in the present invention, MCSwinUnet, adopts Swin Transformer as the backbone network, fully utilizes its ability to capture long-range dependencies, and significantly enhances the extraction effect of roads in complex backgrounds. The MSCAM module is embedded in the encoder, which effectively solves the problem of reduced road extraction accuracy caused by occlusion and complex background. By introducing the spatial feature extraction module SFEM in the jump connection, the network further improves the efficiency of semantic information transmission between the encoder and the decoder, especially when processing small-scale roads. The feature contraction module FCM reduces the loss of information in the downsampling process by combining multiple pooling operations and convolutional layers, enhances the prediction accuracy of edge pixels, and thus ensures the precise positioning of road boundaries. Overall, the present invention shows significant advantages in improving the accuracy and robustness of road extraction in remote sensing images.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for extracting roads from remote sensing images based on a multi-scale context-aware network, characterized in that: include: Acquire remote sensing image road data; Swin Transformer is used as the backbone network to build a multi-scale context-aware network; The remote sensing image road data is input into the multi-scale context-aware network to obtain the pixel-level features of the remote sensing image road and realize the remote sensing image road extraction.

2. The method for extracting roads from remote sensing images based on a multi-scale context-aware network according to claim 1, characterized in that: The multi-scale context-aware network is constructed, which includes an encoder and a decoder. The encoder and the decoder are connected to each other through a skip connection. The encoder includes a multi-scale context aggregation module, a Swin Transformer module, a linear embedding layer and a feature shrinkage module, and the decoder includes a spatial feature extraction module and a linear expansion layer. The road data of the remote sensing image is divided into blocks by the encoder and converted into feature vectors through the linear embedding layer. The feature vectors are then passed layer by layer to the multi-scale context aggregation module, the Swin Transformer module and the feature contraction module for feature extraction. After passing through the bottleneck module, the features enter the decoder and are upsampled through the linear expansion layer. At the same time, feature fusion is combined with jump connections to restore the resolution of the feature map and output the final road pixel-level features.

3. The method for extracting roads from remote sensing images based on a multi-scale context-aware network according to claim 2, characterized in that: The multi-scale context aggregation module includes a global context aggregation branch and a local context processing branch; The global context aggregation branch first performs a maximum pooling operation on the feature map F in Downsampling is performed, and then the pooled feature map is passed to the full attention module. The full attention module calculates the attention weights on the channel or space, and after calculating the global context features of the full attention module, outputs context information with global features; The local context processing branch first transforms the feature map F in It is divided into multiple local blocks, and each local block is passed to the full attention module. The full attention module executes the attention mechanism within the local block, captures local features through the correlation between pixels, and outputs contextual information with local features.

4. The method for extracting roads from remote sensing images based on a multi-scale context-aware network according to claim 2, characterized in that: The spatial feature extraction module integrates the dilated spatial pyramid pooling and convolutional block attention module; The convolutional block attention module consists of a channel attention module and a spatial attention module, which respectively pay attention to the channel and spatial dimensions of the input feature map F. First, the feature map F is input into the channel attention module to obtain the intermediate feature map F. 中 , the intermediate feature map F 中 Multiply it by the feature map F to obtain a new feature map F1; then, input the new feature map F1 into the spatial attention module; the output intermediate feature map is multiplied by the new feature map F1 to obtain the final feature map F2.

5. The method for extracting roads from remote sensing images based on a multi-scale context-aware network according to claim 4, characterized in that: In the channel attention module, the input feature map is compressed in channel and spatial dimensions through average pooling and maximum pooling operations; In the spatial attention module, the channel information of the feature map F′ is aggregated through average pooling and maximum pooling operations to generate the average pooled feature map F Avg And the maximum pooling feature map F Max , and then concatenate and convolve through standard convolutional layers to generate a spatial attention map M s (F1); The standard convolutional layer includes convolution, normalization, and ReLU activation functions.

6. The method for extracting roads from remote sensing images based on a multi-scale context-aware network according to claim 2, characterized in that: The feature shrinkage module includes a global feature branch and a local feature branch; The global feature branch extracts global features through maximum pooling, soft pooling and average pooling operations and performs full connection operations; the local feature branch uses 1×1 convolution and convolution operations with a stride of 2 to reduce the resolution of the feature map.

7. The method for extracting roads from remote sensing images based on a multi-scale context-aware network according to claim 1, characterized in that: The Swin Transformer module uses the shift window multi-head self-attention mechanism SW-MSA, which enables information exchange between different windows by introducing shift operations between windows.

8. A remote sensing image road extraction system based on a multi-scale context-aware network, characterized in that: include: A parameter acquisition module, wherein the parameter acquisition module is used to acquire road data from remote sensing images; A model building module, wherein the model building module is used to use Swin Transformer as a backbone network to build a multi-scale context-aware network; The feature extraction module is used to input the remote sensing image road data into a multi-scale context-aware network to obtain the pixel-level features of the remote sensing image road and realize remote sensing image road extraction.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the remote sensing image road extraction method based on a multi-scale context-aware network as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the remote sensing image road extraction method based on a multi-scale context-aware network as described in any one of claims 1 to 7 are implemented.

Images