A flood remote sensing image segmentation method based on deep learning

By combining ResNet18 and Segformer networks, the dual-channel feature fusion encoding module and DSCA attention module are used to process flood remote sensing images, which solves the problem of inaccurate segmentation of flood areas in flood remote sensing images, and significantly improves segmentation accuracy.

CN118762190BActive Publication Date: 2024-11-05CHENGDU UNIV OF INFORMATION TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411244838.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2024-11-05
Estimated Expiration
2044-09-06

AI Technical Summary

Technical Problem

The prior art is difficult to accurately segment the flood area in flood remote sensing images, especially when the background is complex and the target distribution is uneven, and the problems of undersegment or oversegment are prone to occur.

Method used

The ResNet18 network and the Segformer network are used to extract high and low resolution feature maps, and feature fusion and attention mechanism processing are performed through the dual-channel feature fusion encoding module and the DSCA attention module to enhance the network's semantic understanding ability and segmentation accuracy.

Benefits of technology

The segmentation accuracy of the flood area is improved, and the problem of undersegment or oversegment of the flood area in the remote sensing image is avoided, which significantly improves the segmentation performance of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118762190B_ABST
    Figure CN118762190B_ABST
Patent Text Reader

Abstract

The present invention discloses a flood remote sensing image segmentation method based on deep learning, which relates to the field of image data processing technology, and includes the steps of: constructing a data set according to remote sensing images containing flood areas in different scenes; constructing a flood remote sensing image segmentation network, including a ResNet18 network, a Segformer network, a dual-path feature fusion encoding module, a DSCA attention module and a decoder; training the flood remote sensing image segmentation network with the data set until convergence, and obtaining a flood remote sensing image segmentation model; obtaining a remote sensing image of the flood area to be identified, and sending it to the flood remote sensing image segmentation model to output the identification result. The present invention is aimed at flood remote sensing images with complex backgrounds and uneven target distribution, and can better understand their detailed information, avoid the problem of under-segmentation or over-segmentation of flood areas in remote sensing images, and improve the segmentation accuracy of flood areas.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a flood remote sensing image segmentation method based on deep learning. Background Art

[0002] Floods can cause geological disasters such as collapse and mudslides, which not only submerge buildings, farmland and roads, but also have a serious impact on traffic, directly or indirectly causing huge economic losses, and even causing casualties and ecological damage. Therefore, it is necessary to detect flood disasters.

[0003] The current flood detection schemes mainly include: (1) Manual inspection: Manual patrol officers at the weather station conduct regular inspections at relevant water observation points and use the naked eye to judge the occurrence of floods. This method has high labor costs, strong subjectivity, and is not accurate enough. (2) Sensor equipment: RGB cameras are deployed on urban streets, infrared cameras or radars are deployed at key points in the suburbs, and rain gauges are placed at weather stations to obtain precipitation. The advantages of these methods are that they avoid manual contact and are more accurate, but they require a large number of sensor equipment to be deployed, and the data obtained by various devices need to be uploaded for summary analysis, which makes the cost more expensive. In addition, some remote areas do not have the conditions to deploy relevant equipment and cannot obtain data from the area. (3) Image method: Flood segmentation processing is performed based on the water area image obtained by the RGB camera. This method is low-cost and can actually observe the occurrence of floods at a certain point, but it cannot meet the needs of macro-regional flood detection, and it takes a certain amount of time to coordinate and analyze the results of each observation point. In order to obtain macro-flood conditions, only methods based on remote sensing images can be used. In remote sensing images, flood areas are distributed unevenly throughout the image, densely distributed in the main river channels and sparsely distributed in the flooded areas. The background of remote sensing images of urban floods is more complex, and remote sensing images have higher requirements for detail processing. Traditional image semantic segmentation networks are prone to over-segmentation or under-segmentation of flood areas.

[0004] Segformer, the full name of Segmentation Transformer, is an image segmentation model based on Transformer blocks. It effectively captures the global context information in the image by introducing the self-attention mechanism, thereby achieving high-precision pixel-level segmentation. In Segformer, Transformer blocks are used to extract image features. By stacking multiple layers of Transformer blocks, the model can gradually learn more abstract and rich feature representations. Summary of the invention

[0005] The purpose of the present invention is to provide a solution to the above-mentioned problem, which is to enable the network to better understand the detailed information of flood remote sensing images with complex background and uneven target distribution, and to improve the receptive field while maintaining the resolution of feature maps. A flood remote sensing image segmentation method based on deep learning is provided.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a flood remote sensing image segmentation method based on deep learning, comprising the following steps;

[0007] S1, construct the data set;

[0008] Obtain remote sensing images containing flood areas in different scenarios, mark the flood areas, obtain samples, and all samples constitute a data set;

[0009] S2, construct a flood remote sensing image segmentation network;

[0010] The flood remote sensing image segmentation network includes a ResNet18 network, a Segformer network, a dual-path feature fusion encoding module, a DSCA attention module and a decoder;

[0011] For sample X, the ResNet18 network is used to input X and output the first feature map F1 corresponding to X;

[0012] The Segformer network is used to input X and output a second feature map F2 corresponding to X, where the resolution of F1 is greater than that of F2;

[0013] The dual-path feature fusion encoding module is used to perform 3×3 convolution operations on F1 and F2 respectively to obtain the corresponding convolution features. , , and then , The third feature map F3 is obtained by splicing, and then the F3 channel is shuffled and split into the fourth feature map F4 and the fifth feature map F5. The F4 is dilated and convolved to obtain the feature , perform group convolution on F5 to obtain features , and then , Perform channel splicing to obtain the fused coding feature map F;

[0014] The DSCA attention module is used to perform a 3×3 convolution operation on the fused coding feature map F to obtain a first attention feature map M1, perform global average pooling, one-dimensional convolution and Sigmoid function operations on M1 to obtain a second attention feature map M2, multiply M1 and M2 element by element to restore them to the size of the fused coding feature map F to obtain a third attention feature map M3, and then add M1 and M3 to obtain the attention feature map M;

[0015] The decoder is used to input the attention feature map M and output the predicted flood area;

[0016] S3, using the data set to train the flood remote sensing image segmentation network until convergence, and obtain the flood remote sensing image segmentation model;

[0017] S4, obtaining a remote sensing image of the flood area to be identified, and sending it to a flood remote sensing image segmentation model to output the identification result.

[0018] Preferably, the dual-path feature fusion encoding module includes a 3×3 convolution layer, a first channel splicing layer, a channel shuffling layer, an expansion convolution layer, a grouped convolution layer and a second channel splicing layer;

[0019] The 3×3 convolutional layer is used to perform convolution operations on F1 and F2 and output the corresponding convolution features. , ;

[0020] The first channel splicing layer is used to , Perform channel splicing to obtain the third feature map F3;

[0021] The channel shuffling layer is used to shuffle the F3 channel and split it into a fourth feature map F4 and a fifth feature map F5;

[0022] The dilated convolution layer is used to perform dilated convolution on F4 to obtain the feature ;

[0023] The grouped convolution layer is used to perform grouped convolution on F5 to obtain features ;

[0024] The second channel splicing layer is used to , Channel splicing is performed to obtain the fused encoding feature map F.

[0025] Preferably, the expansion rate of the expansion convolution layer is 3, the number of groups of the grouped convolution layer is 2, and the convolution kernel size is 3×3.

[0026] Preferably, the DSCA attention module includes a 3×3 convolution layer, a global average pooling layer, a one-dimensional convolution layer, a Sigmoid function layer, a multiplication weighted layer and an addition layer;

[0027] The 3×3 convolution layer is used to perform a 3×3 convolution operation on the fused coding feature map F to obtain the first attention feature map M1;

[0028] The global average pooling layer is used to perform global average pooling on M1;

[0029] The one-dimensional convolution layer is used to perform a one-dimensional convolution operation with a convolution kernel size of 5 on the output of the global average pool;

[0030] The Sigmoid function layer is used to perform a Sigmoid function operation on the output of the one-dimensional convolutional layer to obtain a second attention feature map M2;

[0031] The multiplication weighted layer is used to perform element-by-element multiplication and addition on M1 and M2, and restore them to the size of the fused coding feature map F to obtain the third attention feature map M3.

[0032] The addition layer is used to add M1 and M3 to obtain the attention feature map M.

[0033] Compared with the prior art, the advantages of the present invention are:

[0034] The semantic information in the remote sensing images is extracted using the ResNet18 network and the Segformer network. This is because in remote sensing images, the flood areas are distributed throughout the image but unevenly, densely distributed in the main river channel, and sparsely distributed in the flooded area. In order to accurately segment the flood area, higher-level and richer semantic information is required. Therefore, the ResNet18 network and the Segformer network are used to encode them to obtain two feature maps with different resolutions, one high and one low, also known as deep features and shallow features.

[0035] A dual-path feature fusion encoding module is designed. Two feature maps with different information are concatenated and shuffled, and dilated convolution and grouped convolution are performed respectively. The dilation factor of dilated convolution is 3, and the grouping factor of grouped convolution is 2. The feature information at different levels is better fused by shuffling and combining channels. The convolution of the two channels not only expands the network's receptive field but also reduces the amount of parameter calculation while retaining the original feature information, which helps the model better obtain flood area information in remote sensing images and enhances the network's semantic understanding ability.

[0036] A DSCA attention module is designed to perform a dual-path segmentation channel attention mechanism on the output of the dual-path feature fusion encoding module, so that the network pays more attention to the relationship between the channels after the dual-path feature fusion, retains the semantic information of the remote sensing image, focuses on the spatial detail information, and outputs the feature map so that the model's decoder can directly upsample and segment the flood-covered area in the remote sensing image, thereby improving the model's segmentation accuracy.

[0037] In summary, the present invention can better understand the detailed information of flood remote sensing images with complex backgrounds and uneven target distribution, avoid the problem of under-segmentation or over-segmentation of flood areas in remote sensing images, and improve the segmentation accuracy of flood areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a network structure diagram of flood remote sensing image segmentation of the present invention;

[0039] Figure 2 It is the structural diagram of the dual-path feature fusion encoding module;

[0040] Figure 3 This is the structure diagram of the DSCA attention module. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings.

[0042] Example 1: See Figures 1 to 3 ,A flood remote sensing image segmentation method based on deep learning includes the following steps;

[0043] S1, construct the data set;

[0044] Obtain remote sensing images containing flood areas in different scenarios, mark the flood areas, obtain samples, and all samples constitute a data set;

[0045] S2, construct a flood remote sensing image segmentation network;

[0046] The flood remote sensing image segmentation network includes a ResNet18 network, a Segformer network, a dual-path feature fusion encoding module, a DSCA attention module and a decoder;

[0047] For sample X, the ResNet18 network is used to input X and output the first feature map F1 corresponding to X;

[0048] The Segformer network is used to input X and output a second feature map F2 corresponding to X, where the resolution of F1 is greater than that of F2;

[0049] The dual-path feature fusion encoding module is used to perform 3×3 convolution operations on F1 and F2 respectively to obtain the corresponding convolution features. , , and then , The third feature map F3 is obtained by splicing, and then the F3 channel is shuffled and split into the fourth feature map F4 and the fifth feature map F5. The F4 is dilated and convolved to obtain the feature , perform group convolution on F5 to obtain features , and then , Perform channel splicing to obtain the fused coding feature map F;

[0050] The DSCA attention module is used to perform a 3×3 convolution operation on the fused coding feature map F to obtain a first attention feature map M1, perform global average pooling, one-dimensional convolution and Sigmoid function operations on M1 to obtain a second attention feature map M2, multiply M1 and M2 element by element to restore them to the size of the fused coding feature map F to obtain a third attention feature map M3, and then add M1 and M3 to obtain the attention feature map M;

[0051] The decoder is used to input the attention feature map M and output the predicted flood area;

[0052] S3, using the data set to train the flood remote sensing image segmentation network until convergence, and obtain the flood remote sensing image segmentation model;

[0053] S4, obtaining a remote sensing image of the flood area to be identified, and sending it to a flood remote sensing image segmentation model to output the identification result.

[0054] The dual-path feature fusion encoding module includes a 3×3 convolution layer, a first channel splicing layer, a channel shuffling layer, an expansion convolution layer, a grouped convolution layer, and a second channel splicing layer;

[0055] The 3×3 convolutional layer is used to perform convolution operations on F1 and F2 and output the corresponding convolution features. , ;

[0056] The first channel splicing layer is used to , Perform channel splicing to obtain the third feature map F3;

[0057] The channel shuffling layer is used to shuffle the F3 channel and split it into a fourth feature map F4 and a fifth feature map F5;

[0058] The dilated convolution layer is used to perform dilated convolution on F4 to obtain the feature ;

[0059] The grouped convolution layer is used to perform grouped convolution on F5 to obtain features ;

[0060] The second channel splicing layer is used to , Channel splicing is performed to obtain the fused encoding feature map F.

[0061] The expansion rate of the expansion convolution layer is 3, the number of groups of the grouped convolution layer is 2, and the convolution kernel size is 3×3.

[0062] The DSCA attention module includes a 3×3 convolution layer, a global average pooling layer, a one-dimensional convolution layer, a sigmoid function layer, a multiplication weighted layer and an addition layer;

[0063] The 3×3 convolution layer is used to perform a 3×3 convolution operation on the fused coding feature map F to obtain the first attention feature map M1;

[0064] The global average pooling layer is used to perform global average pooling on M1;

[0065] The one-dimensional convolution layer is used to perform a one-dimensional convolution operation with a convolution kernel size of 5 on the output of the global average pool;

[0066] The Sigmoid function layer is used to perform a Sigmoid function operation on the output of the one-dimensional convolutional layer to obtain a second attention feature map M2;

[0067] The multiplication weighted layer is used to perform element-by-element multiplication and addition on M1 and M2, and restore them to the size of the fused coding feature map F to obtain the third attention feature map M3.

[0068] The addition layer is used to add M1 and M3 to obtain the attention feature map M.

[0069] The present invention uses a shallow feature extractor ResNet18 network and a deep feature extractor Segformer network to respectively extract semantic information in remote sensing images, and uses a dual-path feature fusion coding module structure diagram to fuse and encode the two-path features to enhance the semantic understanding ability of the network; the DSCA dual-path segmentation channel attention mechanism is used for the feature map output by the dual-path feature fusion coding module, so that the network pays more attention to the relationship between each channel after the dual-path feature fusion, and can better process the detailed information of flood distribution in remote sensing images.

[0070] In the dual-path feature fusion coding module, the number of groups in the grouped convolution is 2, and the size of the convolution kernel in the grouped convolution is 3×3. Therefore, if the number of channels before the grouped convolution is C1, the output feature map of each group becomes C1 / 2, and the parameters of the convolution kernel are 3×3×C1 / 2, which not only reduces the number of parameters and computation, but also fully extracts the information between the two groups.

[0071] Example 2: To better illustrate the effect of the present invention, we selected a dataset containing 3,000 flood remote sensing images from the IMPACT 2021 ETCI Competition on Flooding Detection and conducted 100 rounds of training, with the ratio of training set to validation set being 8:2. We used the same dataset to conduct comparative experiments with several existing technical methods and the present invention, and compared them with two commonly used semantic segmentation evaluation indicators, and obtained the following Table 1:

[0072] Table 1. Comparison of indicators of different methods

[0073] method Accuracy% mIoU % SegNet 71.46 60.32 PSPNet 77.52 64.51 U-Net 84.84 72.15 DeeplabV3+ 81.24 69.87 The present invention 86.36 77.87

[0074] In Table 1, Accuracy is the pixel accuracy, and mIoU is Mean Intersection over Union in English and Mean Intersection over Union in Chinese. SegNet, PSPNet, U-Net and DeeplabV3+ are all commonly used semantic segmentation models in the field of semantic segmentation. Compared with other segmentation models, the present invention outperforms all the compared existing technologies in terms of Accuracy and mIoU indicators of flood remote sensing image segmentation. The better model design of the present invention can accurately segment the flood area in the remote sensing image.

[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A flood remote sensing image segmentation method based on deep learning, characterized by: The steps include: S1, construct the data set; Obtain remote sensing images containing flood areas in different scenarios, mark the flood areas, obtain samples, and all samples constitute a data set; S2, construct a flood remote sensing image segmentation network; The flood remote sensing image segmentation network includes a ResNet18 network, a Segformer network, a dual-path feature fusion encoding module, a DSCA attention module and a decoder; For sample X, the ResNet18 network is used to input X and output the first feature map F1 corresponding to X; The Segformer network is used to input X and output a second feature map F2 corresponding to X, where the resolution of F1 is greater than that of F2; The dual-path feature fusion encoding module is used to perform 3×3 convolution operations on F1 and F2 respectively to obtain the corresponding convolution features. , , and then , The third feature map F3 is obtained by splicing, and then the F3 channel is shuffled and split into the fourth feature map F4 and the fifth feature map F5. The F4 is dilated and convolved to obtain the feature , perform group convolution on F5 to obtain features , and then , Perform channel splicing to obtain the fused coding feature map F; The DSCA attention module is used to perform a 3×3 convolution operation on the fused coding feature map F to obtain a first attention feature map M1, perform global average pooling, one-dimensional convolution and Sigmoid function operations on M1 to obtain a second attention feature map M2, multiply M1 and M2 element by element to restore them to the size of the fused coding feature map F to obtain a third attention feature map M3, and then add M1 and M3 to obtain the attention feature map M; The decoder is used to input the attention feature map M and output the predicted flood area; S3, using the data set to train the flood remote sensing image segmentation network until convergence, and obtain the flood remote sensing image segmentation model; S4, obtaining a remote sensing image of the flood area to be identified, and sending it to a flood remote sensing image segmentation model to output the identification result.

2. The flood remote sensing image segmentation method based on deep learning according to claim 1, characterized in that: The dual-path feature fusion encoding module includes a 3×3 convolution layer, a first channel splicing layer, a channel shuffling layer, an expansion convolution layer, a grouped convolution layer, and a second channel splicing layer; The 3×3 convolutional layer is used to perform convolution operations on F1 and F2 and output the corresponding convolution features. , ; The first channel splicing layer is used to , Perform channel splicing to obtain the third feature map F3; The channel shuffling layer is used to shuffle the F3 channel and split it into a fourth feature map F4 and a fifth feature map F5; The dilated convolution layer is used to perform dilated convolution on F4 to obtain the feature ; The grouped convolution layer is used to perform grouped convolution on F5 to obtain features ; The second channel splicing layer is used to , Channel splicing is performed to obtain the fused encoding feature map F.

3. The flood remote sensing image segmentation method based on deep learning according to claim 2 is characterized in that: The expansion rate of the expansion convolution layer is 3, the number of groups of the grouped convolution layer is 2, and the convolution kernel size is 3×3.

4. The flood remote sensing image segmentation method based on deep learning according to claim 1, characterized in that: The DSCA attention module includes a 3×3 convolution layer, a global average pooling layer, a one-dimensional convolution layer, a sigmoid function layer, a multiplication weighted layer and an addition layer; The 3×3 convolution layer is used to perform a 3×3 convolution operation on the fused coding feature map F to obtain the first attention feature map M1; The global average pooling layer is used to perform global average pooling on M1; The one-dimensional convolution layer is used to perform a one-dimensional convolution operation with a convolution kernel size of 5 on the output of the global average pool; The Sigmoid function layer is used to perform a Sigmoid function operation on the output of the one-dimensional convolutional layer to obtain a second attention feature map M2; The multiplication weighted layer is used to perform element-by-element multiplication and addition on M1 and M2, and restore them to the size of the fused coding feature map F to obtain the third attention feature map M3. The addition layer is used to add M1 and M3 to obtain the attention feature map M.