A multi-temporal water body detection method based on deep learning and active and passive remote sensing

Abstract

The present invention discloses a multi-temporal water body detection method based on deep learning and active and passive remote sensing, which belongs to the field of remote sensing identification. A TSAE-UNet model is designed to accurately identify satellite remote sensing images of cloudy areas, highly heterogeneous urban areas and phytoplankton interference areas that are often faced in water body detection tasks as water body and non-water body binary images with pixel-level resolution using a trained TSAE-UNet model; the TSAE-UNet model includes replacing the traditional convolution layer in U-Net with a depth-separable convolution, connecting the downsampling and upsampling through an improved jump connection module, and introducing an adaptive attention mechanism in the jump connection, adding a temporal convolution long short-term memory network between the interconnected downsampling and upsampling, and capturing and utilizing the temporal characteristics of multi-temporal remote sensing data through the temporal convolution long short-term memory network. The steps are simple, significantly improve the accuracy and efficiency of water body detection, and meet the growing needs of environmental monitoring and resource management.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image recognition, and in particular to a multi-temporal water body detection method based on deep learning and active and passive remote sensing. Technical Background

[0002] Remote sensing technology has become one of the main means of water body detection because it can provide large-scale, high-resolution surface information and has all-weather and all-day observation capabilities. Remote sensing technology is mainly divided into two categories: optical remote sensing and synthetic aperture radar (SAR) microwave remote sensing.

[0003] Optical remote sensing uses images in the visible and near-infrared bands to detect water bodies through differences in reflectivity. Common methods include the Normalized Difference Water Index (NDWI) and the Modified Normalized Difference Water Index (MNDWI). These methods are simple and intuitive, but they do not work well in conditions such as cloud cover and vegetation occlusion, and are greatly affected by lighting conditions. SAR microwave remote sensing uses the penetration and all-weather capabilities of microwaves to perform high-resolution imaging of the surface. SAR data can be observed on cloudy days and at night, and is not restricted by cloud cover and lighting conditions, so it has unique advantages in water body detection. However, SAR data processing is complex and easily affected by noise. Common methods include water body index and image segmentation algorithms based on dual-polarization data.

[0004] With the growing demand for environmental protection and resource management, the application of water body detection in environmental monitoring, disaster prevention and mitigation, and resource management has become increasingly important. Accurate detection and monitoring of water distribution and changes can effectively support water resource management, flood prevention and mitigation, water quality monitoring, and other tasks. Therefore, it is of great significance to develop efficient and accurate water body detection methods.

[0005] Traditional threshold segmentation-based methods are difficult to guarantee high accuracy and robustness in complex terrain and under multiple interference factors. Machine learning-based methods require a large amount of labeled data and are not effective when processing multi-temporal and multi-source remote sensing data. In addition, water body detection methods based on optical remote sensing usually rely on high-resolution image data, which is not only costly but also difficult to achieve multi-temporal water body detection tasks. Currently, most commonly used open source remote sensing data are medium- and low-resolution images, so there is an urgent need for a low-cost method to achieve multi-temporal water body detection.

[0006] For water body detection tasks, the most widely used technology is remote sensing. The detection of water bodies through remote sensing technology mainly relies on remote sensing satellite images. However, during the imaging process, satellite images will be interfered by the atmosphere, clouds, shadows of tall buildings, floating vegetation on the water surface, and noise, which will lead to misjudgment and omission of detected water bodies. Summary of the invention

[0007] In response to the above technical problems, the present invention proposes a multi-temporal water body detection method based on deep learning and active and passive remote sensing. By combining optical remote sensing images and SAR remote sensing images, the advantages of the two are complemented to effectively reduce interference and noise. A TSAE-UNet network model is designed to learn water body characteristics more efficiently; the time series data processing module, the time series convolutional long short-term memory network ConvLSTM, is used to reduce the misjudgment rate of water body detection on a time scale. The use of active and passive remote sensing data and the design of the TSAE-UNet model jointly reduce the above interference, improve the accuracy of water body detection, and prevent misjudgment and missed judgment in water body detection. The present invention significantly improves the accuracy and efficiency of water body detection to meet the growing needs of environmental monitoring and resource management.

[0008] To achieve the above technical objectives, the present invention discloses a multi-temporal water body detection method based on deep learning and active and passive remote sensing, which accurately identifies satellite remote sensing images of cloudy areas, highly heterogeneous urban areas and phytoplankton interference areas that are often faced by water body detection tasks as water body and non-water body binary images with pixel-level resolution; the steps are as follows:

[0009] The historical Sentinel-2 satellite data and historical Sentinel-1A satellite data of the same study area were collected according to the time phase. The normalized difference water index MNDWI and enhanced normalized difference water index E-MNDWI were calculated based on the Sentinel-2 satellite data; the dual vertical polarization radar data VV and vertical-horizontal polarization radar data VH were calculated based on the historical Sentinel-1A satellite data;

[0010] The Otsu threshold segmentation method was used to process the four groups of remote sensing data of MNDWI, E-MNDWI, VV and VH in the same phase, and the processing results of the four groups of remote sensing data in each month were obtained. The Otsu threshold segmentation method was used to perform water body threshold segmentation on the processing results of the four data of MNDWI, E-MNDWI, VV and VH, and the processed results were superimposed and the intersection was taken to obtain the initial binary images of water body and non-water body.

[0011] All the collected multi-temporal remote sensing data MNDWI, E-MNDWI, VV and VH processing results and their corresponding initial binary images of water bodies and non-water bodies are segmented into 256×256-sized water body and non-water body binary image data, referred to as binary image data, and the segmented binary image data are constructed into a training data set;

[0012] Based on the U-Net model, a temporal adaptive attention enhancement network model TSAE-UNet for multi-temporal water body detection is designed, including interconnected downsampling and upsampling. The traditional convolutional layer in U-Net is replaced by a depth-separable convolution SeparableConv2d. The downsampling and upsampling are connected through an improved skip connection module, and an adaptive attention mechanism Attention Block is introduced in the skip connection. A temporal convolution long short-term memory network ConvLSTM is added between the interconnected downsampling and upsampling. The temporal characteristics of multi-temporal remote sensing data are captured and utilized through the temporal convolution long short-term memory network ConvLSTM.

[0013] The training dataset is used to train the temporal adaptive attention enhancement network model TSAE-UNet for multi-temporal water body detection;

[0014] The downsampling includes sequentially connected input convolution, first layer downsampling, second layer downsampling, third layer downsampling and fourth layer downsampling; wherein the four layers of downsampling have the same structure, including sequentially set maximum pooling, depth separable convolution, batch normalization BatchNorm2d, ReLU activation function, depth separable convolution, BatchNorm2d, ReLU activation function, output; wherein the depth separable convolution includes sequentially connected Depthwise convolution and Pointwise convolution;

[0015] The upsampling includes a first layer upsampling, a second layer upsampling, a third layer upsampling and a fourth layer upsampling connected sequentially; wherein the four layers of upsampling have the same structure, and all include sequentially arranged transposed convolution, depthwise separable convolution, BatchNorm2d, ReLU activation function, depthwise separable convolution, BatchNorm2d, ReLU activation function, and output;

[0016] The multi-temporal Sentinel-1A VV and VH data covering the water area to be detected, as well as the Sentinel-2 image data, were obtained, and the MNDWI and E-MNDWI data were calculated. The VV, VH, MNDWI and E-MNDWI data were input into the trained TSAE-UNet model, and the binary images of the water and non-water bodies in the area were output through prediction by the TSAE-UNet model. The generated binary images of water and non-water bodies were read using ArcGIS software to obtain the distribution range and longitude and latitude positions of the water bodies, and the water area was calculated using ArcGIS.

[0017] A computer device, comprising: a processor, a memory and a network interface;

[0018] The processor is connected to the memory and the network interface, wherein the network interface is used to provide network communication function, the memory is used to store program code, and the processor is used to call the program code to execute the above-mentioned multi-temporal water body detection method based on deep learning and active and passive remote sensing.

[0019] A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and the computer program is suitable for a processor to load the above-mentioned multi-temporal water body detection method based on deep learning and active and passive remote sensing.

[0020] Beneficial effects: First, the TSAE-UNet model enhances the ability to capture temporal and spatial features. Compared with the traditional U-Net model, TSAE-UNet introduces a time series data processing module (ConvLSTM) in the encoding and decoding paths, which greatly improves the processing capabilities of multi-phase data, effectively captures the changing characteristics of water bodies in different time periods, and improves the accuracy and robustness of water body identification. Second, the TSAE-UNet model optimizes the model structure by introducing SeparableConv2d and AttentionBlock. The deep separable convolution significantly reduces the number of model parameters and computational complexity, and improves computational efficiency; the Attention Block enhances the feature selection and extraction capabilities, enabling the model to dynamically focus on important features, further improving the accuracy of water body detection. In addition, the model is trained through multi-source data fusion (such as MNDWI, E-MNDWI, VV, VH), which effectively removes noise and interference and improves the stability and reliability of the detection results. Third, the present invention proposes an efficient automatic label generation strategy, which uses the Otsu threshold segmentation method to generate high-quality training labels, and combines multi-source remote sensing data fusion technology to further improve detection accuracy and stability. This method not only simplifies the preparation process of training data, but also ensures the high quality of model training data, so that the model can show excellent water body detection capabilities under different terrain conditions. Through the above improvements, the TSAE-UNet model provided by the present invention performs well in water body detection tasks in multiple scenarios, can effectively remove noise and interference, improve the accuracy and efficiency of water body detection, and meet the needs of environmental monitoring and resource management and other fields for efficient and accurate water body detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of a multi-temporal water body detection method based on deep learning and active and passive remote sensing of the present invention;

[0022] Figure 2 This is a diagram of the TSAE-UNet network architecture proposed in an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a training data set obtained by the automatic label generation strategy in an embodiment of the present invention;

[0024] Figure 4 Schematic diagram of water body recognition results of the TSAE-UNet model in multiple scenarios in an embodiment of the present invention;

[0025] Figure 5 This is a schematic diagram of the multi-temporal water body detection results in Jining City from 2020 to 2022 in an embodiment of the present invention;

[0026] Figure 6 This is a trend chart of water area changes detected in Jining City in 2020, 2021, and 2022 in an embodiment of the present invention;

[0027] Figure 7 This is a schematic diagram of the accuracy analysis of water body detection results in the four seasons of spring, summer, autumn and winter in Jining City in 2022 in an embodiment of the present invention; DETAILED DESCRIPTION

[0028] The present invention is further described below with reference to the accompanying drawings and specific implementation examples.

[0029] like Figure 1 As shown, a multi-temporal water body detection method based on deep learning and active and passive remote sensing is provided. The steps are as follows:

[0030] Acquire Sentinel-2 data from 2020 to 2022, obtain one scene per month, and calculate MNDWI and E-MNDWI;

[0031] Acquire Sentinel-1A data from 2020 to 2022, one scene per month, and obtain VV and VH data;

[0032] The Otsu threshold segmentation method was used to process the MNDWI, E-MNDWI, VV and VH data of the same month, and the processing results of the four indexes of each month were obtained. The data were fused to obtain a binary image.

[0033] The MNDWI, E-MNDWI, VV and VH processing results of all months and their corresponding binary images were segmented into image data of size 256 × 256 to construct a training data set;

[0034] Based on the U-Net network, a temporal adaptive attention enhancement network TSAE-UNet for multi-temporal water body detection is designed;

[0035] Use the constructed training data set to input into the TSAE-UNet model to complete the training;

[0036] The trained TSAE-UNet model is used to detect the water body.

[0037] Further, the MNDWI is calculated using the following formula:

[0038]

[0039] Among them, B03(ρ GREEN ) represents the green light band of Sentinel-2, B11(ρ SWIR1 ) represents the shortwave infrared band 1 of Sentinel-2;

[0040] The specific calculation formula of E-MNDWI is as follows:

[0041]

[0042] Among them, B03(ρ GREEN ) represents the green light band of Sentinel-2, B11(ρ SWIR1 ) represents the short-wave infrared band 1 of Sentinel-2, B12 (ρ SWIR2 ) represents the shortwave infrared band 2 of Sentinel-2;

[0043] The VV and VH are obtained by processing Sentinel-1A data and can be obtained by https: / / browser.dataspace.copernicus.eu / Download directly to get it;

[0044] The Otsu threshold segmentation method is used to perform water body threshold segmentation on MNDWI, E-MNDWI, VV and VH, and the processed results are superimposed and the intersection is taken to obtain the initial binary image;

[0045] Split MNDWI, E-MNDWI, VV, VH and the corresponding binary images into 256×256 images with a step size of 64;

[0046] Repeat the above steps to get enough samples of the training data set.

[0047] Furthermore, the TSAE-UNet network model includes downsampling, time series data processing modules and upsampling arranged in sequence, and the downsampling and upsampling are connected by an improved skip connection module;

[0048] The downsampling includes sequentially connected input convolution, first layer downsampling, second layer downsampling, third layer downsampling and fourth layer downsampling; wherein the four layers of downsampling have the same structure, including sequentially set maximum pooling, depthwise separable convolution (Depthwise convolution and Pointwise convolution), BatchNorm2d, ReLU activation function, depthwise separable convolution, BatchNorm2d, ReLU activation function, and output;

[0049] The upsampling includes a first layer upsampling, a second layer upsampling, a third layer upsampling and a fourth layer upsampling connected in sequence; wherein the four layers of upsampling have the same structure, and all include sequentially arranged transposed convolution, depthwise separable convolution (Depthwise convolution and Pointwise convolution), BatchNorm2d, ReLU activation function, depthwise separable convolution (Depthwise convolution and Pointwise convolution), BatchNorm2d, ReLU activation function, output, and concatenation;

[0050] Further, downsampling is specifically:

[0051] Input convolution block: Input MNDWI, E-MNDWI, VV and VH image data with size a×a and number of channels 4, numbered 1, first passes through a depthwise convolution with kernel size M×M, step size step_length, padding size padding, and kernel size Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, outputs features of size a×a and number of channels n_channels Figure 2 ; After that, a depthwise convolution with a convolution kernel size of M×M, a step size of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, outputs features of size a×a and number of channels n_channels Figure 3 ;

[0052] First layer downsampling: input features Figure 3 , first through the P×P maximum pooling, the number of channels remains unchanged, and the size is reduced by half, Get Features Figure 4 ; Then, after the convolution kernel size is M×M, the depthwise convolution, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Features with 2×n_channels channels Figure 5 ; Again, the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Features with 2×n_channels channels Figure 6 ;

[0053] Second layer downsampling: Features Figure 6 First, after the P×P maximum pooling, the number of channels remains unchanged and the size is reduced by half. Get Features Figure 7 ; Then, after the convolution kernel size is M×M, the depthwise convolution, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 8 with a channel number of 4×n_channels; again, depthwise convolution with a convolution kernel size of M×M, a step size of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 9 with a channel number of 4×n_channels;

[0054] The third layer of downsampling: Feature map 9 first undergoes P×P maximum pooling, the number of channels remains unchanged, and the size is reduced by half. Get feature map 10; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 11 with a channel number of 8×n_channels; again, it undergoes a depthwise convolution with a convolution kernel size of M×M, a step length of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 12 with channel number 8×n_channels;

[0055] Fourth layer downsampling: Feature map 12 first undergoes P×P maximum pooling, the number of channels remains unchanged, and the size is reduced by half, Get feature map 13; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 14 with a channel number of 16×n_channels; again, depthwise convolution with a convolution kernel size of M×M, a step size of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 15 with a channel number of 16×n_channels;

[0056] Furthermore, using Pytorch's view operation, feature map 15 adds a time dimension, and the output size is The number of channels is 16×n_channels, and the feature map with a time dimension of 1 is 16;

[0057] Furthermore, the specific steps of the time series processing module are as follows: the feature map 16 is input into ConvLSTM, and the output size is The number of channels is 4×n_channels and the feature map of time dimension is 17; remove the time dimension and the output size is The number of channels is 4×n_channels feature map 18;

[0058] Furthermore, the specific steps of upsampling and skip connection are:

[0059] First layer upsampling: Upsampling starts from feature map 18, using a transposed convolution with a kernel size of Q×Q and a step size of 2. The feature map size is doubled, the number of channels remains unchanged, and the output size is The number of channels is 4×n_channels. Feature map 19; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 20 with a channel number of 8×n_channels; again, it passes through the depthwise convolution with a convolution kernel size of M×M, a step size of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 21 with a channel number of 8×n_channels; the feature map 12 in the downsampling stage and the feature map 21 in the upsampling stage are input into the Attention Block for concatenation, and the output size is Feature map 22 with channel number 8×n_channels;

[0060] Second layer upsampling: Feature map 22 is upsampled by transposed convolution with kernel size Q×Q and step size 2. The feature map size is doubled, the number of channels remains unchanged, and the output size is The number of channels is 8×n_channels. Feature map 23; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 24 with a channel number of 4×n_channels; again, it passes through the depthwise convolution with a convolution kernel size of M×M, a step size of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The number of channels is 4×n_channels feature map 25; the feature map 9 in the downsampling stage and the feature map 25 in the upsampling stage are input to the Attention Block for series connection, and the output size is Feature map 26 with channel number 4×n_channels;

[0061] The third layer upsampling: The feature map 26 is upsampled by a transposed convolution with a kernel size of Q×Q and a step size of 2. The feature map size is doubled, the number of channels remains unchanged, and the output size is The number of channels is 4×n_channels. Feature map 27; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 28 with a channel number of 2×n_channels; again, it passes through the depthwise convolution with a convolution kernel size of M×M, a step size of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 29 with 2×n_channels channels; Features at the downsampling stage Figure 6 And the feature map 29 of the upsampling stage are input into Attention Block for series connection, and the output size is The number of channels is 2×n_channels feature map 30;

[0062] The fourth layer of upsampling: feature map 30 is upsampled by transposed convolution with a kernel size of Q×Q and a step size of 2. The feature map size is doubled, the number of channels remains unchanged, and the output size is a×a, the number of channels is 2×n_channels feature map 31; then, it is subjected to depthwise convolution with a kernel size of M×M, a step size of step_length, a padding size of padding, and a kernel size of Pointwise convolution with a step size of step_length, followed by BatchNorm2d and ReLU activation functions, outputting a feature map of size a×a and n_Channels channels 32; Depthwise convolution with a kernel size of M×M, a step size of step_length, padding size of padding, and kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, outputs a feature map of size a×a and n_channels; the feature map of the downsampling stage Figure 3 The feature map 33 of the upsampling stage is input into the Attention Block for series connection, and the output feature map 34 with a size of a×a and a channel number of n_channels is output;

[0063] The skip connection plays a role in maintaining information flow in the network, so that deep features can be directly combined with shallow features, enhancing the expressiveness of features; in the above upsampling process, it can be observed that the feature map obtained by the previous downsampling is connected in series with the feature map after upsampling; these connection processes are to achieve the skip connection;

[0064] Output convolution: After all upsampling steps are completed, the feature map 34 enters an output convolution layer; here, a 1×1 convolution kernel is applied to reduce the number of channels from n_channels channels to the target channel n_classes, and the final output is a feature map 35 of size a×a and the number of channels is n_classes;

[0065] Furthermore, the constructed TSAE-UNet model is trained using the training data;

[0066] The MNDWI, E-MNDEI, VV and VH image data are used as the input data of the TSAE-UNet model, and the corresponding binary image data obtained by the Otsu threshold method is used as the output data of the TSAE-UNet model, that is, the label data, to train the network; the model training adopts the BCEWithLogitsLoss loss function, and its calculation formula is as follows:

[0067]

[0068] in, is the loss term, N is the number of samples in each batch of model training, and Y i is the target label of the i-th training sample, which takes a value of 0 or 1. is the output of the i-th sample model, is the Sigmoid activation function, and X is the input sample.

[0069] Furthermore, the trained TSAE-UNet network model is used for multi-temporal water body detection;

[0070] The MNDWI, E-MNDWI, VV and VH images of the area to be detected are input into the trained TSAE-UNet model, and the corresponding binary image results (0 and 1, where 1 represents water body and 0 represents non-water body) are predicted according to the trained TSAE-UNet model, thus completing the water body detection.

[0071] Embodiment 1

[0072] 1. Overview of the study area and data description:

[0073] Jining City, Shandong Province, China was selected as the research object, and the data used mainly include Sentinel-2 data covering Jining City from 2020 to 2022 (download address: https: / / browser.dataspace.copernicus.eu / ) and Sentinel-1A data, and MNDWI, E-MNDWI, VV and VH data were obtained by calculation; among them, VV and VH data were directly obtained from https: / / browser.dataspace.copernicus.eu / Just download it; the calculation formulas for MNDWI and E-MNDWI are as follows:

[0074]

[0075] Among them, B03(ρ GREEN ) represents the green light band of Sentinel-2, B11(ρ SWIR1 ) represents the shortwave infrared band 1 of Sentinel-2;

[0076]

[0077] Among them, B12 represents the short-wave infrared band 2 of Sentinel-2;

[0078] 2. Construction of training data set

[0079] The MNDWI, E-MNDWI, VV and VH data were obtained, and the water body threshold segmentation processing was performed on MNDWI, E-MNDWI, VV and VH using the Otsu threshold segmentation method. The processed results were superimposed and the intersection was taken to obtain the initial binary image (0 and 1, 1 represents water body, 0 represents non-water body), and the MNDWI, E-MNDWI, VV, VH data and the corresponding initial binary image were obtained;

[0080] The MNDWI, E-MNDWI, VV, VH data and the corresponding initial binary images are divided into small blocks of 256×256. Each segmented MNDWI, E-MNDWI, VV, VH data and the corresponding binary image constitute a sample. Repeat the operation to obtain 10,000 training samples, thereby obtaining the TSAE-UNet model training data set, as shown in Figure 2 As shown;

[0081] 3. Constructing the TSAE-UNet network model for multi-temporal water body detection

[0082] like Figure 3 As shown, the TSAE-UNet network model includes downsampling, upsampling and skip connection parts arranged in sequence:

[0083] The downsampling mainly includes four layers, and the specific steps are as follows:

[0084] Input convolution block: The input size is 256×256 and the number of channels is 4. The MNDWI, E-MNDWI, VV and VH image data is numbered 1. First, it passes through a depthwise convolution with a convolution kernel size of 3×3, a stride of 1, a padding size of 1, and a pointwise convolution with a convolution kernel size of 1×1, and a stride of 1. Then it is processed by BatchNorm2d and ReLU activation functions, and the output size is 256×256 and the number of channels is 64. Figure 2 ; After that, it passes through a depthwise convolution with a kernel size of 3×3, a stride of 1, a padding size of 1, and a pointwise convolution with a kernel size of 1×1, a stride of 1, and then passes through BatchNorm2d and ReLU activation functions to output features with a size of 256×256 and 64 channels. Figure 3 ;

[0085] First layer downsampling: input features Figure 3 First, after 2×2 maximum pooling, the number of channels remains unchanged and the size is reduced by half to 128×128 to obtain the feature Figure 4 ; Then, after the depthwise convolution with a kernel size of 3×3, a stride of 1, a padding size of 1, and a pointwise convolution with a kernel size of 1×1, a stride of 1, and then processed by BatchNorm2d and ReLU activation functions, the output size is 128×128 and the number of channels is 128. Figure 5; Again, it passes through the Depthwise convolution with a kernel size of 3×3, a stride of 1, a padding size of 1, and a Pointwise convolution with a kernel size of 1×1, a stride of 1, and then passes through the BatchNorm2d and ReLU activation functions to output features with a size of 128×128 and 128 channels. Figure 6 ;

[0086] Second layer downsampling: Features Figure 6 First, after 2×2 maximum pooling, the number of channels remains unchanged and the size is reduced by half to 64×64 to obtain the feature Figure 7 ; Then, after the Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then processed by BatchNorm2d and ReLU activation functions, the output size is 64×64 and the number of channels is 256 feature map 8; again after the Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then processed by BatchNorm2d and ReLU activation functions, the output size is 64×64 and the number of channels is 256 feature map 9;

[0087] The third layer of downsampling: Feature map 9 first undergoes 2×2 maximum pooling, the number of channels remains unchanged, and the size is reduced by half to 32×32, obtaining feature map 10; then, it undergoes Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then is processed by BatchNorm2d and ReLU activation functions, and the output size is 32×32 and the number of channels is 512. Feature map 11; it undergoes Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then is processed by BatchNorm2d and ReLU activation functions, and the output size is 32×32 and the number of channels is 512. Feature map 12;

[0088] Fourth layer downsampling: Feature map 12 first undergoes 2×2 maximum pooling, the number of channels remains unchanged, and the size is reduced by half to 16×16, obtaining feature map 13; then, it undergoes Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then is processed by BatchNorm2d and ReLU activation functions, and the output size is 16×16 and the number of channels is 1024. Feature map 14; again, it undergoes Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then is processed by BatchNorm2d and ReLU activation functions, and the output size is 16×16 and the number of channels is 1024. Feature map 15;

[0089] Next, use the view operation of Pytorch to add a time dimension to feature map 15, and output feature map 16 with a size of 16×16, a number of channels of 1024, and a time dimension of 1;

[0090] Next, the specific steps of the time series processing module are as follows: feature map 16 is input into ConvLSTM, and the output size is 16×16, the number of channels is 256, and the feature map 17 of the time dimension is 1; the time dimension is removed, and the output size is 16×16, and the number of channels is 256. Feature map 18.

[0091] Next, upsampling and skip connection are performed. The specific steps of upsampling and skip connection are as follows:

[0092] First layer upsampling: Upsampling starts from feature map 18, and transposed convolution with a kernel size of 2×2 and a stride of 2 is used for upsampling. The feature map size is doubled, the number of channels remains unchanged, and the output size is 32×32, and the number of channels is 256. Feature map 19; then, after the depthwise convolution with a kernel size of 3×3, a stride of 1, a padding size of 1, and a pointwise convolution with a kernel size of 1×1, a stride of 1, and then after BatchNorm2d and ReLU activation functions The feature map 20 with a size of 32×32 and a number of channels of 512 is output; the feature map 20 is processed again with a depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a pointwise convolution with a convolution kernel size of 1×1 and a step size of 1, and then processed by BatchNorm2d and ReLU activation functions, and the feature map 21 with a size of 32×32 and a number of channels of 512 is output; the feature map 12 in the downsampling stage and the feature map 21 in the upsampling stage are input into the Attention Block in series, and the feature map 22 with a size of 32×32 and a number of channels of 512 is output;

[0093] The second layer of upsampling: feature map 22 is upsampled by transposed convolution with a convolution kernel size of 2×2 and a stride of 2. The feature map size is doubled, the number of channels remains unchanged, and the output size is 64×64, the number of channels is 512, feature map 23; then, it is processed by Depthwise convolution with a convolution kernel size of 3×3, a stride of 1, a padding size of 1, and Pointwise convolution with a convolution kernel size of 1×1, a stride of 1, and then processed by BatchNorm2d and ReLU activation functions. Output feature map 24 with a size of 64×64 and 256 channels; again pass through Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then pass through BatchNorm2d and ReLU activation function processing, output feature map 25 with a size of 64×64 and 256 channels; feature map 9 in the downsampling stage and feature map 25 in the upsampling stage are input into the Attention Block in series, and output feature map 26 with a size of 64×64 and 256 channels;

[0094] The third layer of upsampling: Feature map 26 is upsampled by transposed convolution with a convolution kernel size of 2×2 and a step size of 2. The feature map size is doubled, the number of channels remains unchanged, and the output size is 128×128, and the number of channels is 256. Feature map 27; then, it is processed by Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then processed by BatchNorm2d and ReLU activation functions, and the output size is 128×128, and the number of channels is 128. Feature map 28; again, it is processed by Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then processed by BatchNorm2d and ReLU activation functions, and the output size is 128×128, and the number of channels is 128. Feature map 29 in the downsampling stage Figure 6 The feature map 29 in the upsampling stage is input into the AttentionBlock for series connection, and the output feature map 30 with a size of 128×128 and a channel number of 128 is output;

[0095] Fourth layer upsampling: Feature map 30 is upsampled by transposed convolution with a convolution kernel size of 2×2 and a stride of 2. The feature map size is doubled, the number of channels remains unchanged, and the output size is 256×256, the number of channels is 128, feature map 31; then, it is processed by Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then processed by BatchNorm2d and ReLU activation functions, and the output size is 256×256, the number of channels is 64, feature map 32; it is again processed by Depthwise convolution with a convolution kernel size of 3×3, a step size of 1, a padding size of 1, and a Pointwise convolution with a convolution kernel size of 1×1, a step size of 1, and then processed by BatchNorm2d and ReLU activation functions, and the output size is 256×256, the number of channels is 64, feature map 33; the features of the downsampling stage Figure 3 The feature map 33 in the upsampling stage is input into the AttentionBlock together for series connection, and the output feature map 34 with a size of 256×256 and a number of channels of 64 is output;

[0096] Output convolution: After all upsampling steps, feature map 34 enters an output convolution layer; here, a 1×1 convolution kernel is applied to reduce the number of channels from 64 channels to the target channel 1, and the final output is a feature map 35 of size 256×256 and a channel number of 1;

[0097] 4. Train the training data set

[0098] The mainstream deep learning framework pytorch1.12.1 is used to train the above TSAE-UNet network. The computer hardware and software configuration is shown in Table 1. Figure 1As shown, the Adam optimizer is used for network training, the BCEWithLogitsLoss loss function is selected, the initial learning rate is 0.0001, the batch size is 4, and the number of training epochs is 100. The MNDWI, E-MNDWI, VV and VH image data are used as the model input for forward propagation, and the TSAE-UNet model predicts the results. With the BCEWithLogitsLoss loss function, the loss term is calculated, the gradient is calculated, and then back-propagated. The network is trained through the Adam optimizer, and then the next round of training is carried out; the MNDWI, E-MNDWI, VV and VH image data are used as the input data of the TSAE-UNet model, and the MNDWI, E-MNDWI, VV and VH image data are segmented and fused by Otsu threshold to obtain the corresponding binary image as the output data of the TSAE-UNet model, that is, the label data, to train the network; the model training uses the BCEWithLogitsLoss loss function, and its calculation formula is as follows:

[0099]

[0100] in, is the loss term, N is the number of samples in each batch of model training, and Y i is the target label of the i-th training sample, which takes a value of 0 or 1. is the output of the i-th sample model, is the Sigmoid activation function, and X is the input sample.

[0101] 1 Computer hardware and software configuration

[0102]

[0103]

[0104] 5. Input the MNDWI, E-MNDWI, VV and VH image data of the water area to be detected into the trained TSAE-UNet model to complete multi-temporal water body detection

[0105] This experiment mainly carried out water body detection experiments in multiple scenarios (such as Figure 4 ), Jining City 2020-2022 multi-temporal water body detection experiment (as shown in Figure 5 and Figure 6 As shown) and water body detection and precision analysis experiments in Jining City in the four seasons of spring, summer, autumn and winter in 2022;

[0106] like Figure 4As shown in the figure, the water body detection experiment in multiple scenarios selects the difficult detection areas (cloudy areas, highly heterogeneous urban areas and phytoplankton interference areas) that are often faced by water body detection tasks, and inputs the MNDWI, E-MNDWI, VV and VH image data of the corresponding areas into the TSAE-UNet model to obtain the corresponding water body detection results, thus completing the water body detection task. Figure 4 It can be seen that TSAE-UNet is not affected by cloud cover, heterogeneous urban agglomerations, and phytoplankton on the surface of water bodies. It can meet the existing real-world water body detection needs and solve the problem that water body detection is easily interfered by multiple factors.

[0107] like Figure 5 As shown in the figure, the TSAE-UNet model is used to detect the water area in Jining City from 2020 to 2022, and the water area of ​​each month is counted. The detected water area change trend is as follows Figure 6 As shown, it can be seen that the water body change trends in 2020, 2021 and 2022 are highly consistent in space, and the corresponding time points are also consistent, which realizes the practical needs of multi-temporal water body detection;

[0108] like Figure 7 As shown, water body detection was carried out in Jining City in spring, summer, autumn and winter of 2022, and the confusion matrix was used to evaluate the performance of the TSAE-UNet model; in the spring of 2022, the TSAE-UNet model had a user accuracy of 99.61%, a producer accuracy of 99.45%, an overall accuracy of 99.08%, and a Kappa coefficient of 0.83 for distinguishing water bodies; in the summer of 2022, the TSAE-UNet model had a user accuracy of 99.67%, a producer accuracy of 9 9.70%, overall accuracy 99.39%, Kappa coefficient 0.88; in the autumn of 2022, the TSAE-UNet model distinguished water bodies with a user accuracy of 99.79%, a producer accuracy of 99.95%, an overall accuracy of 99.74%, and a Kappa coefficient of 0.95; in the winter of 2022, the TSAE-UNet model distinguished water bodies with a user accuracy of 99.89%, a producer accuracy of 99.88%, an overall accuracy of 99.78%, and a Kappa coefficient of 0.96. This accuracy can meet the needs of most scientific research and practical applications.

[0109] The present invention provides a multi-temporal water body detection method based on deep learning and active and passive remote sensing, which uses an advanced TSAE-UNet model to predict data. The TSAE-UNet model introduces a time series data processing module and an AttentionBlock module in the encoding and decoding paths, enhances the ability to capture spatiotemporal features, and optimizes the design of the convolutional layer to ensure high accuracy of the prediction results.

[0110] After multiple verifications and comparisons, the data predicted by the TSAE-UNet model is highly consistent with the actual data, proving the superiority and robustness of the present invention. In addition, high-precision water body detection results are extremely valuable for achieving efficient and high-precision water body distribution monitoring in important fields such as environmental monitoring, disaster prevention and mitigation, and resource management. The present invention not only improves the accuracy and robustness of water body detection, but also generates high-quality training labels through the Otsu threshold segmentation method, and combines multi-source data fusion technology (such as MNDWI, E-MNDWI, VV, VH) to effectively remove noise and multi-factor interference, thereby improving the stability and reliability of the detection results.

[0111] The method of the present invention is simple to operate and highly efficient. It is suitable for multi-temporal water body detection tasks in different scenarios and can help related research and applications achieve more accurate data analysis and processing. The accuracy of water body detection is determined by using the confusion matrix and the extracted water body area. The detection result we obtain is a binary image of water body and non-water body, which can be seen in Figure 5 , white represents detected water bodies, and black represents non-water bodies. TSAE-UNet predicts pixel by pixel. If the pixel is a water body, it will mark it as a water body, otherwise it is a non-water body, so a more accurate water body boundary can be obtained. It adopts a method that combines deep learning with remote sensing technology. It is suitable for water body detection tasks in areas of any size. Compared with traditional methods, it can achieve water body detection tasks more efficiently. From the perspective of the TSAE-UNet model, the use of a lightweight attention mechanism module can effectively reduce the complexity of model calculation while fully extracting features.

Claims

1. A multi-temporal water body detection method based on deep learning and active and passive remote sensing, characterized in that: The satellite remote sensing images of cloudy areas, highly heterogeneous urban areas, and phytoplankton interference areas that are often encountered in water body detection tasks are accurately identified as binary images of water bodies and non-water bodies with pixel-level resolution; the steps are as follows: The historical Sentinel-2 satellite data and historical Sentinel-1A satellite data of the same study area were collected according to the time phase, and the normalized difference water index (MNDWI) and enhanced normalized difference water index (E-MNDWI) were calculated based on the Sentinel-2 satellite data. Dual vertical polarization radar data VV and vertical-horizontal polarization radar data VH are calculated based on historical Sentinel-1A satellite data; The Otsu threshold segmentation method was used to process the four groups of remote sensing data of MNDWI, E-MNDWI, VV and VH in the same phase, and the processing results of the four groups of remote sensing data in each month were obtained. The Otsu threshold segmentation method was used to perform water body threshold segmentation on the processing results of the four data of MNDWI, E-MNDWI, VV and VH, and the processed results were superimposed and the intersection was taken to obtain the initial binary images of water body and non-water body. All the collected multi-temporal remote sensing data MNDWI, E-MNDWI, VV and VH processing results and their corresponding initial binary images of water bodies and non-water bodies are segmented into 256×256-sized water body and non-water body binary image data, referred to as binary image data, and the segmented binary image data are constructed into a training data set; Based on the U-Net model, a temporal adaptive attention enhancement network model TSAE-UNet for multi-temporal water body detection is designed, including interconnected downsampling and upsampling. The traditional convolutional layer in U-Net is replaced by a depth-separable convolution SeparableConv2d. The downsampling and upsampling are connected through an improved skip connection module, and an adaptive attention mechanism Attention Block is introduced in the skip connection. A temporal convolution long short-term memory network ConvLSTM is added between the interconnected downsampling and upsampling. The temporal characteristics of multi-temporal remote sensing data are captured and utilized through the temporal convolution long short-term memory network ConvLSTM. The training dataset is used to train the temporal adaptive attention enhancement network model TSAE-UNet for multi-temporal water body detection; The downsampling includes sequentially connected input convolution, first layer downsampling, second layer downsampling, third layer downsampling and fourth layer downsampling; wherein the four layers of downsampling have the same structure, including sequentially set maximum pooling, depth separable convolution, batch normalization BatchNorm2d, ReLU activation function, depth separable convolution, BatchNorm2d, ReLU activation function, output; wherein the depth separable convolution includes sequentially connected Depthwise convolution and Pointwise convolution; The upsampling includes a first layer upsampling, a second layer upsampling, a third layer upsampling and a fourth layer upsampling connected sequentially; wherein the four layers of upsampling have the same structure, and all include sequentially arranged transposed convolution, depthwise separable convolution, BatchNorm2d, ReLU activation function, depthwise separable convolution, BatchNorm2d, ReLU activation function, and output; The multi-temporal Sentinel-1A VV and VH data covering the water area to be detected, as well as the Sentinel-2 image data, were obtained, and the MNDWI and E-MNDWI data were calculated. The VV, VH, MNDWI and E-MNDWI data were input into the trained TSAE-UNet model, and the binary images of the water and non-water bodies in the area were output through prediction by the TSAE-UNet model. The generated binary images of water and non-water bodies were read using ArcGIS software to obtain the distribution range and longitude and latitude positions of the water bodies, and the water area was calculated using ArcGIS.

2. The multi-temporal water body detection method based on deep learning and active and passive remote sensing according to claim 1 is characterized in that: MNDWI, The calculation process of the four index data of E-MNDWI, VV and VH is as follows: MNDWI is calculated using the following formula: Among them, B03 represents the green light band ρ of Sentinel-2 GREEN , B11 represents the short-wave infrared band 1ρ of Sentinel-2 SWIR1 ; E-MNDWI is calculated using the following formula: Among them, B03 represents the green light band ρ of Sentinel-2 GREEN , B11 represents the short-wave infrared band 1ρ of Sentinel-2 SWIR1 , B12 represents the short-wave infrared band 2ρ of Sentinel-2 SWIR2 ; The VV and VH are https: / / browser.dataspace.copernicus.eu / Direct query acquisition; The Otsu threshold segmentation method is used to perform water body threshold segmentation on MNDWI, E-MNDWI, VV and VH, and the processed results are superimposed and the intersection is taken to obtain the initial binary image; Split MNDWI, E-MNDWI, VV, VH and the corresponding binary images into 256×256 images with a step size of 64; Repeat the above steps to get enough samples of the training data set.

3. The multi-temporal water body detection method based on deep learning and active and passive remote sensing according to claim 2 is characterized in that: The downsampling in the temporal adaptive attention enhancement network model TSAE-UNet for multi-temporal water body detection is as follows: Input convolution block: Input MNDWI, E-MNDWI, VV and VH image data with size a×a and channel number 4 as feature map 1, first pass a depthwise convolution with kernel size M×M, step length step_length, padding size padding, and kernel size Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, outputting feature map 2 with a size of a×a and n_channels channels; then, a Depthwise convolution with a kernel size of M×M, a step length of step_length, a padding size of padding, and a kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, outputs feature map 3 with a size of a×a and n_channels channels; First layer downsampling: Input feature map 3 is first subjected to P×P maximum pooling, the number of channels remains unchanged, and the size is reduced by half, Get feature map 4; then, after the convolution kernel size is M×M, the depthwise convolution, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 5 with a channel number of 2×n_channels; again after a depthwise convolution with a convolution kernel size of M×M, a step length of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 6 with 2×n_channels channels; Second layer downsampling: Feature map 6 first undergoes P×P maximum pooling, with the number of channels remaining the same and the size reduced by half, Get feature map 7; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 8 with a channel number of 4×n_channels; again, depthwise convolution with a convolution kernel size of M×M, a step size of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 9 with a channel number of 4×n_channels; The third layer of downsampling: Feature map 9 first undergoes P×p maximum pooling, the number of channels remains unchanged, and the size is reduced by half. Get feature map 10; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 11 with a channel number of 8×n_channels; again after the depthwise convolution with a convolution kernel size of M×M, its step length is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 12 with channel number 8×n_channels; Fourth layer downsampling: Feature map 12 first undergoes P×P maximum pooling, the number of channels remains unchanged, and the size is reduced by half, Get feature map 13; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 14 with a channel number of 16×n_channels; again, depthwise convolution with a convolution kernel size of M×M, a step length of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is Feature map 15 with a channel number of 16×n_channels.

4. The multi-temporal water body detection method based on deep learning and active and passive remote sensing according to claim 3 is characterized in that: Using Pytorch's view operation, feature map 15 adds a time dimension, and the output size is The number of channels is 16×n_channels and the time dimension is 16 feature maps.

5. The multi-temporal water body detection method based on deep learning and active and passive remote sensing according to claim 4 is characterized in that: Specific steps of the time series processing module: Feature map 16 is input into ConvLSTM, and the output size is The number of channels is 4×n_channels and the feature map of time dimension is 17; remove the time dimension and the output size is The number of channels is 4×n_channels.

6. The multi-temporal water body detection method based on deep learning and active and passive remote sensing according to claim 5 is characterized in that: The specific steps of upsampling and skip connection in the temporal adaptive attention enhancement network model TSAE_UNet for multi-temporal water body detection are: First layer upsampling: Upsampling starts from feature map 18, using transposed convolution with kernel size Q×Q and step length 2×step_length. The feature map size is doubled, the number of channels remains unchanged, and the output size is The number of channels is 4×n_channels. Feature map 19; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 20 with a channel number of 8×n_channels; again, it undergoes a depthwise convolution with a convolution kernel size of M×M, a step size of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 21 with a channel number of 8×n_channels; the feature map 12 in the downsampling stage and the feature map 21 in the upsampling stage are input into the Attention Block for concatenation, and the output size is Feature map 22 with channel number 8×n_channels; Second layer upsampling: Feature map 22 is upsampled by transposed convolution with kernel size Q×Q and step length 2×step_length. The feature map size is doubled, the number of channels remains unchanged, and the output size is The number of channels is 8×n_channels. Feature map 23; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 24 with a channel number of 4×n_channels; again, it passes through the depthwise convolution with a convolution kernel size of M×M, a step size of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The number of channels is 4×n_channels feature map 25; the feature map 9 in the downsampling stage and the feature map 25 in the upsampling stage are input to the Attention Block for series connection, and the output size is Feature map 26 with channel number 4×n_channels; The third layer upsampling: Feature map 26 is upsampled by transposed convolution with a kernel size of Q×Q and a step length of 2×step_length. The feature map size is doubled, the number of channels remains unchanged, and the output size is The number of channels is 4×n_channels. Feature map 27; then, after the convolution kernel size is M×M, the step size is step_length, the padding size is padding, and the convolution kernel size is Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The feature map 28 with a channel number of 2×n_channels; again, it undergoes a depthwise convolution with a convolution kernel size of M×M, a step length of step_length, a padding size of padding, and a convolution kernel size of Pointwise convolution with a step length of step_length, followed by BatchNorm2d and ReLU activation functions, and the output size is The number of channels is 2×n_channels feature map 29; the feature map 6 in the downsampling stage and the feature map 29 in the upsampling stage are input to the Attention Block for concatenation, and the output size is The number of channels is 2×n_channels feature map 30; Fourth layer upsampling: Feature map 30 is upsampled by transposed convolution with kernel size Q×Q and step length 2×step_length. The feature map size is doubled, the number of channels remains unchanged, and the output size is a×a, the number of channels is 2×n_channels feature map 31; then, it is subjected to depthwise convolution with kernel size M×M, step length step_length, padding size padding, and kernel size Pointwise convolution with a step size of step_length, followed by BatchNorm2d and ReLU activation functions, outputting a feature map of size a×a and n_channels 32; Depthwise convolution with a kernel size of M×M, a step size of step_length, padding size of padding, and a kernel size of Pointwise convolution with a step size of step_length is then processed by BatchNorm2d and ReLU activation functions, and the output feature map 33 has a size of a×a and n_channels. The feature map 3 in the downsampling stage and the feature map 33 in the upsampling stage are input into the Attention Block in series, and the output feature map 34 has a size of a×a and n_channels.

7. The multi-temporal water body detection method based on deep learning and active and passive remote sensing according to claim 6 is characterized in that: The specific steps inside the Attention Block are as follows: the upsampled output of the decoder part is used as the input of the AttentionBlock, the feature map with a size of H×W and the number of channels is C, and the output size is H×W with a channel number of C after a two-dimensional convolution with a convolution kernel size of 1×1 and a step size of 1. The feature map is then passed through BatchNorm2d, and the output size is H×W, and the number of channels is Similarly, the output of the corresponding layer of the encoder is also used as the input of the Attention Block. After the same operation as above, the output size is H×W and the number of channels is The feature map of the two output results (size is H×W, the number of channels is The feature map of is fused to obtain a new size of H×W and the number of channels is After the ReLU activation function, the output size is H×W and the number of channels is The feature map of is passed through the designed Psi branch (1×1 convolution, BatchNorm2d, Sigmoid activation function) to obtain a weight map with a size of H×W and a channel number of 1, which is then multiplied with the feature map output by the corresponding layer of the encoder part, and finally the feature map with a size of H×W and a channel number of C is output; The jump connection plays a role in maintaining the flow of information in the network, so that deep features can be directly combined with shallow features, enhancing the expressiveness of features; in the above upsampling process, it can be observed that the feature map obtained by the previous downsampling is connected in series with the feature map after upsampling; these connection processes are the realization of jump connections; Output convolution: After all upsampling steps, the feature map 34 enters an output convolution layer; here, a 1×1 convolution kernel is applied to reduce the number of channels from n_channels channels to the target channel n_classes, and the final output is a feature map 35 of size a×a and n_classes channels.

8. The multi-temporal water body detection method based on deep learning and active and passive remote sensing according to claim 1, characterized in that: Use the training data to train the designed TSAE-UNet model; The Adam optimizer is used for network training, and the BCEWithLogitsLoss loss function is selected. After setting the initial learning rate, batch size, and number of training rounds, the MNDWI, E-MNDWI, VV, and VH image data are used as model input for forward propagation. The TSAE-UNet model predicts the results, and the BCEWithLogitsLoss loss function is used to calculate the loss term and gradient, and then back-propagates. The network is trained using the Adam optimizer, and then the next round of training is performed. The MNDWI, E-MNDWI, VV, and VH image data are used as the input data of the TSAE-UNet model, and the MNDWI, E-MNDWI, VV, and VH image data are segmented and fused by Otsu threshold to obtain the corresponding binary image as the output data of the TSAE-UNet model, that is, the label data, to train the network. The model training uses the BCEWithLogitsLoss loss function, and its calculation formula is as follows: in, is the loss term, N is the number of samples in each batch of model training, and Y i is the target label of the i-th training sample, which takes a value of 0 or 1. is the output of the i-th sample model, is the Sigmoid activation function, and X is the input sample.

9. A computer device, characterized in that: include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide network communication function, the memory is used to store program code, and the processor is used to call the program code to execute the multi-temporal water body detection method based on deep learning and active and passive remote sensing as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and executing the multi-temporal water body detection method based on deep learning and active and passive remote sensing as described in any one of claims 1 to 7.

Images