Remote sensing image road extraction method and system based on VMamba and channel attention
By introducing the MambaDeepLab model with VMamba and channel attention, the problem of global context and multi-scale feature representation difficulties in road extraction from remote sensing images is solved, achieving efficient and accurate road extraction and vector map generation.
Patent Information
- Application Number
- CN202411766404.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2044-12-04
AI Technical Summary
Existing technologies for road extraction in high-resolution remote sensing images face challenges in representing global contextual information and multi-scale features. Traditional methods are time-consuming and labor-intensive, while models based on CNNs and Transformers suffer from high computational complexity or limitations in local receptive fields.
We employ the MambaDeepLab model based on VMamba and channel attention, combined with the DeepLabV3+ architecture. The VMamba module captures global context information and multi-scale features, the channel attention module refines the features, and connected component analysis is used for post-processing to generate high-quality road vector maps.
It improves the accuracy and efficiency of road extraction from remote sensing images, avoids the limitations of local receptive field in CNNs and the high computational complexity of Transformers, and generates high-quality road vector maps.
Smart Images
Figure CN119762499B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a remote sensing image road extraction method and system based on VMamba and channel attention, belonging to the technical field of remote sensing image and artificial intelligence. BACKGROUND
[0002] Remote sensing image road extraction is a basic task in remote sensing image processing, and has important practical application value in many fields such as city planning, traffic navigation, environmental monitoring, disaster assessment, etc. With the continuous development of remote sensing technology, the collection cycle of high-resolution remote sensing images is getting shorter and shorter, providing rich data sets. Therefore, extracting roads from these high-resolution remote sensing images has become a research hotspot.
[0003] Although extracting roads from remote sensing images has attracted widespread attention, this task still has challenges. Mainly including: 1) Remote sensing images usually show roads of different shapes, sizes, distributions and textures, which brings challenges to road extraction; 2) The occlusion and shadow caused by buildings, trees, vehicles also affect the integrity of road extraction; 3) The resolution and quality of remote sensing images can directly affect the accuracy of road extraction. In order to solve these challenges, effectively extracting global context information and multi-scale feature representation from remote sensing images is an important topic. The former helps the model to understand the overall structure and layout of the image, so as to better distinguish roads and other background elements. The latter can preserve local details and global information, enabling the model to more accurately extract roads of different sizes.
[0004] Traditional road extraction methods rely on inherent morphological features of input images and hand-crafted features developed by professionals for classification, and these methods are often not suitable for large-scale remote sensing images containing complex, multi-scale road networks. In this case, not only does the extraction of hand-crafted features require specialized knowledge, but it is also very time-consuming and laborious.
[0005] Compared with traditional methods, deep learning-based methods have superiority in automatic and robust extraction of semantic features. Convolutional neural networks (CNNs) as the first representative of deep learning have been widely applied in the task of road extraction from remote sensing images. Fully convolutional networks (FCN) generate pixel-wise pixel classification results by replacing fully connected layers with convolutional layers. However, the downsampling process of these models continuously reduces the spatial resolution of the image, ultimately leading to the loss of global context information. Later, the famous U-Net work solves this problem by fusing deep and shallow feature information through skip connections, and variants of U-Net have also been proposed for road extraction from remote sensing images; however, the limited local receptive field of UNet structure limits its ability to capture multi-scale information. In order to improve the model's ability to extract multi-scale information, the famous DeepLab work proposes an atrous spatial pyramid pooling (ASPP) to obtain multi-scale feature representation; however, the multi-scale features generated by ASPP cannot avoid containing repetitive or redundant spatial information. The introduction of attention mechanisms can effectively highlight important features and suppress redundant or irrelevant information, thereby reducing the impact of information redundancy. SE-DeepLab combines DeepLabv3+ with SE attention modules to enhance the weight of effective information, suppress weak information in the image, and improve the integrity of road extraction in the prediction process; RAANet embeds attention modules and residual structures into ASPP, constructing a new residual ASPP and improving the accuracy of road extraction from high-resolution remote sensing images; current CNN-based models have been able to effectively extract global context information and multi-scale features, but due to the characteristics of the local receptive field of convolutional operations, they have inherent limitations in processing global context information.
[0006] Compared with CNN, Transformer can focus on other positions directly through self-attention mechanism, effectively capturing global context information. On this basis, a series of road extraction models combined with Transformer have emerged, such as CNN and multi-scale Transformer fusion network CMTFNet, which extracts and fuses local information and multi-scale global context information from high-resolution remote sensing images; however, due to the quadratic complexity of self-attention mechanism, these improvements are accompanied by an increase in computational demand.
[0007] At present, the state space model Mamba provides an innovative solution for solving these problems due to its powerful modeling ability to remote context information and maintaining linear computational complexity, and the cross scanning module (CSM) introduced in VMamba further improves the applicability of Mamba in computer vision tasks. Therefore, how to fully utilize the linear long sequence modeling ability of Mamba and combine the classic segmentation framework to further improve the effect of remote sensing image road extraction has become an important problem. SUMMARY
[0008] Therefore, the present application provides a remote sensing image road extraction method, system, computer device and storage medium based on VMamba and channel attention, which can meet the requirements of high resolution remote sensing field for complex road extraction accuracy, rapidity and wide range of applicability.
[0009] The first object of the present application is to provide a remote sensing image road extraction method based on VMamba and channel attention.
[0010] The second object of the present application is to provide a remote sensing image road extraction system based on VMamba and channel attention.
[0011] The third object of the present application is to provide a computer device.
[0012] The fourth object of the present application is to provide a computer readable storage medium.
[0013] The first object of the present application can be achieved by adopting the following technical solutions:
[0014] A remote sensing image road extraction method based on VMamba and channel attention, the method comprises:
[0015] Acquiring a remote sensing image data set;
[0016] Pretreating the remote sensing images in the remote sensing image data set;
[0017] Based on the DeepLabV3+ architecture, a MambaDeepLab model is constructed in combination with the VMamba module;
[0018] The MambaDeepLab model is trained by using the pretreated remote sensing image data set, and a trained segmentation model is obtained;
[0019] The trained segmentation model is used to realize the extraction of roads in the remote sensing image of the scene to be segmented;
[0020] The extracted road results are subjected to broken line reconnection post-processing using connected component analysis;
[0021] The road result after the disconnection reconnection processing is rasterized to vector to generate a road vector map.
[0022] Further, the MambaDeepLab model is an asymmetric encoder-decoder structure network model based on a VMamba module and a channel attention module, and includes a block embedding layer, an encoder, a spatial pyramid pool, a channel attention module and a decoder.
[0023] The block embedding layer is configured to perform embedding processing on the input image to obtain a processed image.
[0024] The encoder is configured to construct a hierarchical feature extraction global context information for the processed image.
[0025] The spatial pyramid pool is configured to extract multi-scale information from deep features in the hierarchical features.
[0026] The channel attention module is configured to refine the multi-scale information.
[0027] The decoder is configured to fuse and up-sample shallow features in the hierarchical features and the refined multi-scale information to generate a segmentation map.
[0028] Further, the embedding processing on the input image to obtain a processed image specifically includes:
[0029] Dividing the input image into non-overlapping blocks with a size of N*N;
[0030] Mapping the divided image in dimensions to obtain a mapped image;
[0031] Performing normalization processing on the mapped image to obtain a processed image.
[0032] Further, the encoder includes three cascaded VSSLayer layers, the first two VSSLayer layers each include two VSSblock blocks and one PatchMerging block, and the third VSSLayer layer includes six VSSblock blocks and one PatchMerging block.
[0033] Each VSSblock block includes two layer normalization layers, one convolution layer, two linear layers, two SiLU activation layers and one SS2D module, the SS2D module includes a Cross-Scan module and a pure Mamba module, and the PatchMerging block includes a linear layer and a layer normalization layer connected in sequence.
[0034] Further, the spatial pyramid pool includes four different size segmentation windows, four corresponding VSSblock blocks, four linear layers and one final linear layer.
[0035] Each VSSblock block comprises two layer normalization layers, one convolution layer, two linear layers, two SiLU activation layers and one SS2D module, the SS2D module comprising one Cross-Scan module and one pure Mamba module.
[0036] Further, the channel attention module comprises one global average pooling layer, one ReLU activation layer, one Sigmoid activation layer and two fully connected layers.
[0037] Further, the decoder comprises three cascaded VSSLayer_up layers and one final projection layer, each VSSLayer_up layer comprising two VSSblock blocks and one PatchExpaning block, the final projection layer comprising one linear layer, one normalization layer and one convolution layer.
[0038] Each VSSblock block comprises two layer normalization layers, one convolution layer, two linear layers, two SiLU activation layers and one SS2D module, the SS2D module comprising one Cross-Scan module and one pure Mamba module; the PatchExpaning block comprises one linear layer and one layer normalization layer connected in sequence.
[0039] The second object of the application can be achieved by adopting the following technical scheme:
[0040] A remote sensing image road extraction system based on VMamba and channel attention, the system comprising:
[0041] A data acquisition unit for acquiring a remote sensing image dataset;
[0042] A data preprocessing unit for preprocessing remote sensing images in the remote sensing image dataset;
[0043] A model construction unit for constructing a MambaDeepLab model based on a DeepLabV3+ architecture combined with a VMamba module;
[0044] A model training unit for training the MambaDeepLab model using the preprocessed remote sensing image dataset to obtain a trained segmentation model;
[0045] A road extraction unit for extracting roads in the remote sensing image using the trained segmentation model;
[0046] A broken line reconnection post-processing unit for performing broken line reconnection post-processing on the extracted road results using connected component analysis;
[0047] The application discloses a road vector map production unit for raster-to-vector conversion of a road result after broken-line reconnection post-processing to generate a road vector map.
[0048] The third object of the application can be achieved by adopting the following technical solution:
[0049] A computer device comprises a processor and a memory for storing a program executable by the processor, and the processor implements the remote sensing image road extraction method when executing the program stored in the memory.
[0050] The fourth object of the application can be achieved by adopting the following technical solution:
[0051] A computer readable storage medium stores a program, and the program is executed by a processor to implement the remote sensing image road extraction method.
[0052] The application has the following beneficial effects relative to the prior art:
[0053] The application introduces a VMamba (VSS) block as a basic block to construct a DeepLabV3+ structure, which captures long sequence global context information and multi-scale information while maintaining linear computational complexity, avoids the lack of long-range information capturing ability caused by the limitation of local receptive field of the CNN-based model, and the high computational burden caused by the quadratic complexity of the self-attention mechanism of the Transformer-based model. The channel attention mechanism is introduced to refine the multi-scale information, improve the extraction accuracy of the model to the road, and use a connected component analysis to post-process the extracted road result and generate a high-quality road vector map. BRIEF DESCRIPTION OF DRAWINGS
[0054] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor based on the structures shown in the drawings.
[0055] Figure 1 The flowchart of the remote sensing image road extraction method based on VMamba and channel attention of the application embodiment 1.
[0056] Figure 2 The schematic diagram of the remote sensing image of the application embodiment 1.
[0057] Figure 3 The structural diagram of the MambaDeepLab model of the application embodiment 1.
[0058] Figure 4 Structure diagram of the block embedding layer of the embodiment 1 of the present application.
[0059] Figure 5 Structure diagram of the spatial pyramid pooling of the embodiment 1 of the present application.
[0060] Figure 6 Structure diagram of the channel attention module of the embodiment 1 of the present application.
[0061] Figure 7 Schematic diagram of the connected component analysis before merging of the embodiment 1 of the present application.
[0062] Figure 8 Schematic diagram of the connected component analysis after merging of the embodiment 1 of the present application.
[0063] Figure 9 Flow chart of the connected component analysis method of the embodiment 1 of the present application.
[0064] Figure 10 Schematic diagram of the road vector map of the embodiment 1 of the present application.
[0065] Figure 11 Structure block diagram of the remote sensing image road extraction device based on VMamba and channel attention of the embodiment 2 of the present application.
[0066] Figure 12 Structure block diagram of the computer device of the embodiment 3 of the present application. DETAILED DESCRIPTION
[0067] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme of the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.
[0068] Embodiment 1:
[0069] As shown in the drawings, the present embodiment provides a remote sensing image road extraction method based on VMamba and channel attention, which comprises the following steps: Figure 1
[0070] S101, acquiring a remote sensing image data set.
[0071] The remote sensing image and its corresponding real label map of ground objects of the present embodiment, the remote sensing image is as shown in the drawings Figure 2 As shown, the real mark map of the road is used to provide a reference for the prediction result of the model to calculate the loss function to optimize the model; the model performance can also be evaluated by comparing with the model prediction result.
[0072] S102, pre-process the remote sensing image in the remote sensing image data set.
[0073] The embodiment of the present application preprocesses the remote sensing image in the remote sensing image data set, including standardization, denoising, enhancement, etc. Specifically, first, the standardization scales the data to a mean of 0 and a variance of 1 by the Z-score method, so that the features have the same dimension, facilitating model training. Second, Gaussian filtering is used to reduce noise in the data and improve data quality. In addition, data augmentation generates transformed samples by methods such as random rotation, flipping, adjusting brightness and contrast, etc. The remote sensing image is segmented into several data slices of size 512x512, and is divided into training set and validation set according to the ratio of 8:2. The training set contains training images and training labels, and the validation set contains validation images and validation labels. Finally, the data set required for training and verifying the MambaDeepLab network model is obtained.
[0074] S103, based on the DeepLabV3+ architecture, a MambaDeepLab model is constructed by combining the VMamba module.
[0075] As shown in Figure 3 , the MambaDeepLab model is an asymmetric encoder-decoder structure network model based on the VMamba module and the channel attention module. The VMamba(VSS) block is introduced as the basic block to expand the DeepLabV3+ structure. The VMamba-based encoder captures global context information, and the Mamba-SPP based on the VMamba spatial pyramid pool captures multi-scale feature information. The MambaDeepLab model includes block embedding layer, encoder, spatial pyramid pool, channel attention module and decoder.
[0076] As shown in Figure 4 , the block embedding layer of the MambaDeepLab model, i.e. the input layer, has 3 feature mapping channels of the input sample feature data, and the output feature is x∈R H×W×3 ; where H represents the height of the image; W represents the width of the image; the block embedding layer is used to embed the input image to obtain the processed image, and the specific embedding process includes: dividing the input image into non-overlapping blocks of size NxN, where N=4 in this embodiment, i.e. dividing the input image into non-overlapping patches of size 4x4; performing dimension mapping on the divided image to obtain a mapping image; performing normalization processing on the mapping image to obtain a processed image.
[0077] The encoder includes three cascaded VSSLayer layers, the first two VSSLayer layers each include two VSSblock blocks and one PatchMerging block, and the third VSSLayer layer includes six VSSblock blocks and one PatchMerging block, and the three VSSLayer layers in the embodiment are respectively a first VSSLayer layer, a second VSSLayer layer and a third VSSLayer layer; in the embodiment, each VSSLayer layer of the encoder corresponds to a stage, and a block merging operation is used at the end of each stage to reduce the height and width of the input features while increasing the number of channels.
[0078] As shown in Figure 5 The spatial pyramid pooling is a Mamba-SPP based on VMamba (Mamba-SPP), which includes four different size (2, 4, 8 and 16) split window layers and four corresponding VSSblock blocks, four linear layers and a final linear layer, smaller windows are used to capture local details, while larger windows are used to collect global context information, and the four split window layers in the embodiment are respectively a first split window layer, a second split window layer, a third split window layer and a fourth split window layer.
[0079] As shown in Figure 6 The channel attention module is used to refine the multi-scale information, which includes a global average pooling layer, a ReLU activation layer, a Sigmoid activation layer and two fully connected layers; first, the spatial information of each channel is compressed into a scalar by global average pooling on the input feature map to generate a channel descriptor. Two fully connected layers are used to process the channel descriptor to learn the relationship between channels. The first fully connected layer reduces the dimension (using the ReLU activation function), and the second fully connected layer restores the dimension and generates channel weights (using the Sigmoid activation function). Finally, the learned channel weights are multiplied with the input feature map to generate a reweighted feature map.
[0080] The decoder is used for fusing and up-sampling the shallow features in the hierarchical features and the refined multi-scale information to generate a segmentation map, and includes three cascaded VSSLayer_up layers and a final projection layer, each VSSLayer_up layer includes two VSSblock blocks and a PatchExpaning block, the three VSSLayer_up layers in the embodiment are a first VSSLayer_up layer, a second VSSLayer_up layer and a third VSSLayer_up layer respectively, and the final projection layer includes a linear layer, a normalization layer and a convolution layer; in the embodiment, each VSSLayer_up layer of the decoder corresponds to a stage, and the block expansion operation is used at the beginning of each stage to reduce the number of feature channels and increase the height and width.
[0081] In the embodiment, each VSSblock block includes two layer normalization layers, a convolution layer, two linear layers, two SiLU activation layers and an SS2D module, the SS2D module includes a Cross-Scan module and a pure Mamba module; the PatchMerging block includes a linear layer and a layer normalization layer connected in sequence; and the PatchExpaning block includes a linear layer and a layer normalization layer connected in sequence.
[0082] In the embodiment, the step S104 specifically includes:
[0083] In the embodiment, the step S104 specifically includes:
[0084] Firstly, the training set batch is input to the block embedding layer; the input image is divided into non-overlapping patches with a size of 4x4, and the dimension of the image is mapped to C, and in the embodiment, the number of feature mapping channels is 96, that is, C is 96, to generate an embedded image and generate a feature matrix as the first skip connection layer; specifically, the block embedding layer divides the input image x e R H×W×3 The image is divided into non-overlapping blocks with a size of 4x4 through a convolution layer, and the dimension of the image is mapped to C (C is 96 in the embodiment). Through the process, an embedded image Then, the layer normalization layer is used to normalize x', and the generated feature matrix
[0085] After that, the block embedding layer is input to the encoder for hierarchical feature extraction; specifically, the feature matrix generated by the block embedding layer The first hierarchical feature matrix is obtained through the two VSSblock blocks of the first VSSLayer layer and the PatchMerging block for down-sampling operation The second hierarchical feature matrix is obtained by extracting features through two VSSblock blocks of the second VSSLayer layer and performing a downsampling operation through a PatchMerging block The third hierarchical feature matrix is obtained by extracting features through six VSSblock blocks of the third VSSLayer layer and performing a downsampling operation through a PatchMerging block
[0086]
[0087] Next, high-level features Multi-scale features are generated through a VMamba-based spatial pyramid pooling (Mamba-SPP); specifically, the high-level features The first scale feature is obtained by inputting the first window layer, extracting features through a VSSblock block with a 2x2 split window, and performing feature dimension transformation through a corresponding linear layer (here, the dimension is transformed to 256) The second scale feature is obtained by inputting the second window layer, extracting features through a VSSblock block with a 4x4 split window, and performing feature dimension transformation through a corresponding linear layer The third scale feature is obtained by inputting the third window layer, extracting features through a VSSblock block with an 8x8 split window, and performing feature dimension transformation through a corresponding linear layer The fourth scale feature is obtained by inputting the fourth window layer, extracting features through a VSSblock block with a 16x16 split window, and performing feature dimension transformation through a corresponding linear layer Then, the four scale features are spliced and dimension-transformed through a final linear layer to obtain the final multi-scale feature
[0088] The multi-scale feature is then input into a channel attention module to learn the relationship between channels and dynamically adjust the importance of each channel; specifically, the multi-scale feature A channel descriptor w1∈R is generated by first compressing the spatial information of each channel into a scalar through a global average pooling 1×1×C The channel descriptor is then processed through two fully connected layers to learn the relationship between channels and obtain a channel weight w2∈R 1×1×C The learned channel weight is multiplied with the input multi-scale feature to obtain a reweighted feature multi-scale feature
[0089] The shallow features in the hierarchical features generated by the encoder and the refined multi-scale information are input into the decoder for fusion and upsampling to generate a segmentation map; specifically, the multi-scale feature After being upsampled by four times, the shallow features are fused with the multi-scale feature The fused features are up-sampled by a first VSSLayer_up layer to obtain features The fused features are up-sampled by a first VSSLayer_up layer to obtain features The fused features are up-sampled by a first VSSLayer_up layer to obtain features The fused features are up-sampled by a first VSSLayer_up layer to obtain features H×W×64 .
[0090] The final output y is generated by a final projection layer H×W×numclass , where numclass is the number of classes to be segmented.
[0091] After the training is completed, the trained segmentation model is obtained, the verification set is input into the trained segmentation model, the extraction of the road in the remote sensing image is completed, and the result of the model method is evaluated according to the intersection over union (IoU). The definition of IoU is as follows:
[0092]
[0093] , where TP is True Positive, representing true positive: predicted positive, actual positive; FP is False Positive, representing false positive: predicted positive, actual negative; FN is False Negative, representing false negative: predicted negative, actual positive; TN is True Negative, representing true negative: predicted negative, actual negative; the larger the result of IoU, the more similar the predicted picture is to the provided true label.
[0094] S105, using the trained segmentation model to realize the extraction of the road in the remote sensing image of the scene to be segmented.
[0095] In this embodiment, the scene to be segmented can be the extraction of ground objects in a natural scene or the regional planning of a city; by obtaining the remote sensing image of the scene to be segmented and performing preprocessing, the remote sensing image is input into the trained segmentation model to complete the extraction of the road in the remote sensing image.
[0096] S106, using connected component analysis to process the extracted road result.
[0097] As shown in the red box in FIG. 8, by analyzing the characteristics (such as area, shape, etc.) of the connected components, noise and small false detections can be filtered out, disconnected road segments can be identified and reconnected, and the coherence and integrity of the road network are ensured. Figures 7-8
[0098] As shown in FIG. 8, this step S106 specifically includes: Figure 9
[0099] Reading and binarization processing: read the semantic segmentation result image and perform binarization processing.
[0100] Connected component detection and filtering: use the connectedComponentsWithStats() function of OpenCV to detect connected components and filter out components with small areas.
[0101] Line break reconnection: use the dilation operation to reconnect the broken lines.
[0102] S107, raster-to-vector processing of the road result after line break reconnection to generate a road vector map.
[0103] As shown in Figure 10 , the raster-to-vector processing is performed by the Polygonize() function in the GDAL library in Python to generate a road vector map for further analysis and application.
[0104] It should be noted that although the above method operations are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in that specific order, or that all the shown operations must be performed to achieve the desired results. On the contrary, the steps depicted can change the order of execution. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps.
[0105] Embodiment 2:
[0106] As shown in Figure 11 , the embodiment provides a remote sensing image road extraction system based on VMamba and channel attention, which includes a data acquisition unit 1101, a data preprocessing unit 1102, a model construction unit 1103, a model training unit 1104, a road extraction unit 1105, a line break reconnection post-processing unit 1106, and a road vector map production unit 1107. The specific functions of each unit are as follows:
[0107] The data acquisition unit 1101 is used to acquire a remote sensing image dataset.
[0108] The data preprocessing unit 1102 is used to preprocess the remote sensing images in the remote sensing image dataset.
[0109] The model construction unit 1103 is used to construct a MambaDeepLab model based on the DeepLabV3+ architecture combined with the VMamba module.
[0110] The model training unit 1104 is used to train the MambaDeepLab model using the preprocessed remote sensing image dataset to obtain a trained segmentation model.
[0111] The road extraction unit 1105 is used to extract roads from the remote sensing image using a trained segmentation model.
[0112] The disconnection and reconnection post-processing unit 1106 is used to perform disconnection and reconnection post-processing on the extracted road results using connectivity component analysis.
[0113] The road vector map production unit 1107 is used to convert the road results after disconnection and reconnection into raster-to-vector conversion to generate road vector maps.
[0114] The specific implementation of each unit in this embodiment can be found in Embodiment 1 above, and will not be repeated here. It should be noted that the system provided in this embodiment is only illustrated by the division of the above functional units. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure can be divided into different functional units to complete all or part of the functions described above.
[0115] Example 3:
[0116] This embodiment provides a computer device, such as... Figure 12 As shown, it includes a processor 1202, a memory, an input device 1203, a display device 1204, and a network interface 1205 connected via a device bus 1201. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium 1206 and internal memory 1207. The non-volatile storage medium 1206 stores operating devices, computer programs, and a database. The internal memory 1207 provides an environment for the operation of the operating devices and computer programs in the non-volatile storage medium. When the processor 1202 executes the computer program stored in the memory, it implements the remote sensing image road extraction method of Embodiment 1 described above, as follows:
[0117] Acquire a remote sensing image dataset; preprocess the remote sensing images in the dataset; build a MambaDeepLab model based on the DeepLabV3+ architecture and the VMamba module; train the MambaDeepLab model using the preprocessed remote sensing image dataset to obtain a trained segmentation model; use the trained segmentation model to extract roads from the remote sensing images of the scene to be segmented; perform post-processing of broken road connections using connected component analysis on the extracted road results; convert the post-processed road results from raster to vector to generate a road vector map.
[0118] Example 4:
[0119] The embodiment provides a computer readable storage medium storing a computer program, and the computer program is executed by a processor to implement the remote sensing image road extraction method in the embodiment 1, and the method is as follows:
[0120] An image dataset is acquired, remote sensing images in the image dataset are preprocessed, a MambaDeepLab model is constructed based on a DeepLabV3+ architecture and in combination with a VMamba module, the MambaDeepLab model is trained by using the preprocessed image dataset, a trained segmentation model is obtained, the trained segmentation model is used to implement extraction of roads in remote sensing images of a scene to be segmented, the extracted road results are post-processed by using connected component analysis for line breaking and reconnection, and a road vector map is generated by performing raster-to-vector conversion on the line breaking and reconnection post-processed road results.
[0121] It should be noted that the computer readable storage medium in the embodiment can be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0122] In the embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or apparatus. In the embodiment, the computer readable signal medium can include a data signal propagating in a baseband or as a carrier wave in a propagated data signal, which carries computer readable programs. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable storage medium other than the computer readable storage medium, which can send, propagate or transmit programs for use by or in combination with an instruction execution system, device or apparatus. The computer program contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to a wire, an optical cable, an RF (radio frequency) or the like, or any suitable combination of the above.
[0123] The computer readable storage medium described above can be written in one or more programming languages or combinations of languages for executing the computer program of the present embodiment, including object oriented programming languages such as Java, Python, C++, and conventional procedural programming languages such as C language or similar programming languages. The program can be executed entirely on a user computer, partially on a user computer, as a separate software package, partially on a user computer and partially on a remote computer, or entirely on a remote computer or server. In the case involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, through the Internet using an Internet service provider).
[0124] In summary, the present application introduces the VMamba (VSS) block as a basic block to build the DeepLabV3+ structure, which captures long sequence global context information and multi-scale information while maintaining linear computational complexity, avoids the lack of long-range information capture ability caused by the limitation of local receptive field of the CNN-based model, and the high computational burden caused by the quadratic complexity of the self-attention mechanism of the Transformer-based model. The channel attention mechanism is introduced to refine the multi-scale information, improve the extraction accuracy of the model to the road, and use the connected component analysis to post-process the extracted road results and generate high-quality road vector maps.
[0125] The above is only a preferred embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the present application within the scope disclosed by the present application, which belongs to the protection scope of the present application.
Claims
1. A method for road extraction from remote sensing images based on VMamba and channel attention, characterized in that, The method includes: Obtain remote sensing image datasets; Preprocess the remote sensing images in the remote sensing image dataset; Based on the DeepLabV3+ architecture, a MambaDeepLab model is built using the VMamba module; The MambaDeepLab model was trained using the preprocessed remote sensing image dataset to obtain a trained segmentation model; Using a trained segmentation model, roads can be extracted from remote sensing images of the scene to be segmented; The extracted road results are processed using connectivity component analysis to reconnect disconnected lines. The road results after disconnection and reconnection are converted from raster to vector to generate a road vector map. The MambaDeepLab model is an asymmetric encoder-decoder network model based on VMamba modules and channel attention modules, including a block embedding layer, encoder, spatial pyramid pool, channel attention module and decoder; The block embedding layer is used to embed the input image to obtain the processed image; The encoder is used to construct hierarchical features and extract global context information from the processed image; The spatial pyramid pool is used to extract multi-scale information from deep features in hierarchical features; The channel attention module is used to refine multi-scale information; The decoder is used to fuse shallow features and refined multi-scale information in the hierarchical features and upsample them to generate a segmentation map; The encoder comprises three cascaded VSSLayer layers. The first two VSSLayer layers each include two VSSblocks and one PatchMerging block, and the third VSSLayer layer includes six VSSblocks and one PatchMerging block. The spatial pyramid pool comprises four different sized segmentation windows and four corresponding VSSblocks, four linear layers, and one final linear layer. The decoder comprises three cascaded VSSLayer_up layers and one final projection layer. Each VSSLayer_up layer includes two VSSblocks and one PatchExpaning block, and the final projection layer includes one linear layer, one normalization layer, and one convolutional layer. Each VSSblock comprises two layer normalization layers, one convolutional layer, two linear layers, two SiLU activation layers, and one SS2D module, which includes a Cross-Scan module and a pure Mamba module; the PatchExpaning block comprises a sequentially connected linear layer and a layer normalization layer.
2. The method for road extraction from remote sensing images according to claim 1, characterized in that, The process of embedding the input image to obtain the processed image specifically includes: Divide the input image into non-overlapping blocks of size N×N; The segmented image is then dimension-mapped to obtain the mapped image; The mapped image is normalized to obtain the processed image.
3. The method for road extraction from remote sensing images according to claim 1, characterized in that, The channel attention module includes a global average pooling layer, a ReLU activation layer, a Sigmoid activation layer, and two fully connected layers.
4. A remote sensing image road extraction system based on VMamba and channel attention, characterized in that, The system includes: The data acquisition unit is used to acquire remote sensing image datasets. The data preprocessing unit is used to preprocess the remote sensing images in the remote sensing image dataset; The model building unit is used to build a MambaDeepLab model based on the DeepLabV3+ architecture and combined with the VMamba module. The model training unit is used to train the MambaDeepLab model using the preprocessed remote sensing image dataset to obtain a trained segmentation model. The road extraction unit is used to extract roads from the remote sensing image using a trained segmentation model. The disconnection and reconnection post-processing unit is used to perform disconnection and reconnection post-processing on the extracted road results using connectivity component analysis. The road vector map production unit is used to convert the raster to vector of the road results after disconnection and reconnection to generate road vector maps. The MambaDeepLab model is an asymmetric encoder-decoder network model based on VMamba modules and channel attention modules, including a block embedding layer, encoder, spatial pyramid pool, channel attention module and decoder; The block embedding layer is used to embed the input image to obtain the processed image; The encoder is used to construct hierarchical features and extract global context information from the processed image; The spatial pyramid pool is used to extract multi-scale information from deep features in hierarchical features; The channel attention module is used to refine multi-scale information; The decoder is used to fuse shallow features and refined multi-scale information in the hierarchical features and upsample them to generate a segmentation map; The encoder comprises three cascaded VSSLayer layers. The first two VSSLayer layers each include two VSSblocks and one PatchMerging block, and the third VSSLayer layer includes six VSSblocks and one PatchMerging block. The spatial pyramid pool comprises four different sized segmentation windows and four corresponding VSSblocks, four linear layers, and one final linear layer. The decoder comprises three cascaded VSSLayer_up layers and one final projection layer. Each VSSLayer_up layer includes two VSSblocks and one PatchExpaning block, and the final projection layer includes one linear layer, one normalization layer, and one convolutional layer. Each VSSblock comprises two layer normalization layers, one convolutional layer, two linear layers, two SiLU activation layers, and one SS2D module, which includes a Cross-Scan module and a pure Mamba module; the PatchExpaning block comprises a sequentially connected linear layer and a layer normalization layer.
5. A computer device, comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the remote sensing image road extraction method according to any one of claims 1-3.
6. A computer-readable storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the remote sensing image road extraction method according to any one of claims 1-3.