A remote sensing water extraction method and device based on a U-Net and Transformer fusion architecture

Abstract

This application discloses a remote sensing water body extraction method and apparatus based on a U-Net and Transformer fusion architecture, relating to the fields of intelligent remote sensing image processing and artificial intelligence. The method first acquires multispectral remote sensing images and knowledge product data, dividing them into blocks to obtain multispectral image blocks and spatial attention layer blocks. Then, a trained hybrid model is used for inference: the U-Net encoder extracts spatial features, the Transformer module fuses spatiotemporal features and focuses on water body regions, and the decoder outputs a probability map which is processed to generate a binary mask image. Finally, the mask images are stitched together to generate a monthly dataset. This method can achieve multi-source data fusion and spatiotemporal modeling, improve the accuracy of small water body and boundary identification, achieve large-area efficient coverage at high resolution, and meet the requirements of high accuracy and high timeliness.

Description

Technical Field

[0001] This application relates to the fields of remote sensing image intelligent processing and artificial intelligence technology, and in particular to a remote sensing water body extraction method and apparatus based on a U-Net and Transformer fusion architecture. Background Technology

[0002] Water bodies, as a crucial environmental element on the Earth's surface, hold significant research and application value in remote sensing monitoring. Their spatiotemporal dynamic information is widely used in water resource management, flood disaster early warning, and aquatic ecological environment assessment. In recent years, with the stable operation and open data sharing of medium- and high-resolution remote sensing satellites such as the Landsat and Sentinel series, it has become possible to acquire remote sensing image data covering large areas and long time series, greatly promoting the development of automated water body extraction technology.

[0003] Existing water body extraction methods mainly include threshold-based spectral index methods (such as NDWI, MNDWI, etc.), traditional machine learning methods (such as support vector machines, random forests, etc.), and deep learning methods that have been widely studied in recent years. Among them, convolutional neural networks, represented by U-Net, are widely used in remote sensing image semantic segmentation tasks due to their powerful spatial feature extraction capabilities, and have also been used in related research on water body extraction. However, in practical applications, existing models generally lack effective temporal modeling capabilities and multi-source heterogeneous image adaptation capabilities, making it difficult to meet the current comprehensive requirements of "high timeliness, high accuracy, and strong adaptability" for remote sensing water body extraction. Summary of the Invention

[0004] The purpose of this application is to provide a remote sensing water body extraction method and device based on the U-Net and Transformer fusion architecture, which is applicable to water body change monitoring tasks with large range and high temporal resolution, and can generate spatially detailed and temporally continuous high-resolution water body datasets, achieving high-precision and high-time-efficiency water body extraction.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides a remote sensing water body extraction method based on a U-Net and Transformer fusion architecture, including:

[0007] Acquire multispectral remote sensing images of the target area for the month to be extracted, as well as the corresponding knowledge product data, wherein the knowledge product data includes digital elevation model data and river network product vector data;

[0008] The multispectral remote sensing image and the knowledge product data are processed into blocks to obtain several image blocks, wherein the image blocks include multispectral remote sensing image blocks and spatial attention layer blocks;

[0009] The image patch is used as input, and a trained water body extraction hybrid model is used to output a binary water body mask. The water body extraction hybrid model is a deep learning model composed of a U-Net encoder module, a Transformer module with spatial attention mechanism, and a U-Net decoder module. The U-Net encoder module extracts the spatial features of the multispectral remote sensing image patch. The Transformer module outputs a fusion feature sequence that fuses time dependence and spatial attention based on the spatial features and the spatial attention layer patch. The U-Net decoder module outputs a water body distribution probability map based on the fusion feature sequence. The water body distribution probability map is then normalized using a Sigmoid activation function and processed using a thresholding method to obtain the binary water body mask.

[0010] Each of the aforementioned binary water body mask images is stitched together according to its spatial location to obtain a complete water body distribution layer covering the target area for the month to be extracted;

[0011] Organize the complete water body distribution layers for each month to be extracted in chronological order to generate a monthly water body distribution dataset for the target region.

[0012] In a second aspect, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the remote sensing water extraction method based on the U-Net and Transformer fusion architecture described in the first aspect above.

[0013] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the remote sensing water extraction method based on the U-Net and Transformer fusion architecture described in the first aspect above.

[0014] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the remote sensing water extraction method based on the U-Net and Transformer fusion architecture described in the first aspect.

[0015] According to the specific embodiments provided in this application, this application has the following technical effects:

[0016] This application provides a remote sensing water body extraction method and apparatus based on a U-Net and Transformer fusion architecture. First, multispectral remote sensing images and knowledge product data of the target area are acquired and processed into blocks to obtain multispectral image patches and spatial attention layer patches. The extracted multi-source data lays the foundation for subsequent feature fusion. Second, a trained hybrid deep learning model is used for inference: the U-Net encoder module extracts spatial detail features from the multispectral image patches, addressing the problem of insufficient spatial feature extraction; the Transformer module with a spatial attention mechanism combines spatial attention layer patches and spatial features, capturing the temporal dependence of water bodies through temporal modeling, while guiding the model to focus on salient water body areas, balancing temporal dynamics and spatial attention; the U-Net decoder module outputs a water body distribution probability map based on the fused feature sequence, which is then processed by Sigmoid normalization and thresholding to generate a binary water body mask map, enhancing the recognition accuracy of small water bodies and boundaries. Finally, the binary water body mask images output in blocks are stitched together according to spatial location to form a complete water body distribution layer, and monthly datasets are generated in chronological order. Through block processing and stitching strategies, high resolution is ensured while achieving efficient coverage of a large area, balancing efficiency and accuracy.

[0017] In summary, this method achieves effective fusion of multi-source data and collaborative modeling of spatiotemporal features, accurately capturing spatial details and temporal dynamic changes of water bodies, and significantly improving the ability to identify water body boundaries in small water bodies and complex backgrounds. At the same time, through block processing and stitching strategies, it achieves complete coverage of a large area while ensuring high spatial resolution. The resulting monthly water body distribution dataset has the characteristics of spatial detail and temporal continuity, meeting the needs of water resource monitoring, ecological assessment and other fields for high-precision and high-timeliness water body data. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a remote sensing water extraction method based on a U-Net and Transformer fusion architecture, provided in Embodiment 1 of this application;

[0020] Figure 2 A schematic diagram illustrating the concept of a remote sensing water extraction method based on a U-Net and Transformer fusion architecture, provided in Embodiment 1 of this application;

[0021] Figure 3 This is a schematic diagram illustrating the training performance of the hybrid deep learning model in Embodiment 1 of this application;

[0022] Figure 4 This is a schematic diagram of the structure of a computer device provided in Embodiment 2 of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Research has revealed the following shortcomings in existing remote sensing water body extraction technologies:

[0025] (1) Difficulty in fusing multi-source data: Multi-source remote sensing images have significant differences in spatial resolution, temporal distribution and spectral band structure. Direct fusion can easily lead to inconsistencies in information and affect the generalization ability of the model.

[0026] (2) Insufficient time series modeling capability: Most methods focus on static analysis of single-phase images, making it difficult to capture the dynamic evolution characteristics of water bodies in the time dimension.

[0027] (3) Class imbalance and poor adaptability: Water areas usually account for a very small proportion in remote sensing images, resulting in obvious class imbalance. Existing methods are not capable of distinguishing small water bodies and boundary areas, and are prone to misidentification or missed detection.

[0028] (4) Limited spatial boundary recognition: When faced with complex backgrounds (such as water bodies and vegetation, bare land, and urban areas), the traditional U-Net structure has limited ability to express detailed edges, and it is difficult to simultaneously meet the balance between extraction efficiency and spatial resolution under large coverage in actual deployment.

[0029] To this end, this embodiment proposes a remote sensing water body extraction method that can integrate multi-temporal and multi-source remote sensing data, has the ability to extract local spatial details and model global temporal features, and has the ability to adapt to class imbalance and update the model, so as to improve the stability, accuracy and intelligent application level of the model in complex remote sensing scenarios.

[0030] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Example 1

[0032] See Figure 1 This embodiment provides a remote sensing water body extraction method based on a U-Net and Transformer fusion architecture, including:

[0033] S1, acquire multispectral remote sensing images of the target area for the month to be extracted and corresponding knowledge product data, wherein the knowledge product data includes digital elevation model data and river network product vector data;

[0034] S2, the multispectral remote sensing image and the knowledge product data are processed into blocks to obtain several image blocks, wherein the image blocks include multispectral remote sensing image blocks and spatial attention layer blocks;

[0035] S3, taking the image patch as input, the trained water body extraction hybrid model outputs a binary water body mask image. The water body extraction hybrid model is a deep learning model composed of a U-Net encoder module, a Transformer module with spatial attention mechanism, and a U-Net decoder module. The U-Net encoder module extracts the spatial features of the multispectral remote sensing image patch. The Transformer module outputs a fusion feature sequence that integrates time dependence and spatial attention based on the spatial features and the spatial attention layer patch. The U-Net decoder module outputs a water body distribution probability map based on the fusion feature sequence. The water body distribution probability map is then normalized using a Sigmoid activation function and processed using a thresholding method to obtain the binary water body mask image.

[0036] S4, each of the binary water body mask images is stitched together according to its spatial location to obtain a complete water body distribution layer covering the target area for the month to be extracted;

[0037] S5. Organize the complete water body distribution layers for each month to be extracted in chronological order to generate a monthly water body distribution dataset for the target area.

[0038] The following is combined Figure 2 The implementation process of the remote sensing water extraction method based on the U-Net and Transformer fusion architecture in this embodiment is described in detail.

[0039] like Figure 2 The remote sensing water extraction method based on the U-Net and Transformer fusion architecture shown includes the training process and application process of the water extraction hybrid model. Specifically, the method includes:

[0040] (1) Obtain 10m spatial resolution multispectral remote sensing images of each target month in the target area within the target year, published 30m spatial resolution JRCGSW water product images, and related knowledge product data.

[0041] The 10-meter spatial resolution multispectral remote sensing image is a preprocessed monthly-scale fused remote sensing image. The preprocessing includes atmospheric correction, cloud detection and declouding, as well as median fusion of multiple images within the same month. The multispectral information includes, but is not limited to, the following spectral bands that are sensitive to water body identification: blue band (center wavelength approximately 0.45–0.52 μm), green band (approximately 0.53–0.59 μm), red band (approximately 0.64–0.67 μm), near-infrared band (approximately 0.85–0.88 μm), shortwave infrared band 1 (approximately 1.57–1.65 μm), and shortwave infrared band 2 (approximately 2.10–2.30 μm).

[0042] The 30-meter spatial resolution JRC Global Surface Water (JRC GSW) water body product image is a water body identification time series product released by the Joint Research Centre of the European Union based on the Landsat series of remote sensing images. It has a monthly time resolution, and its pixels are divided into water body pixels, non-water body pixels, and missing pixels according to the water body identification results of that month.

[0043] The knowledge-based product data includes digital elevation model (DEM) data and river network product vector data.

[0044] (2) The multispectral remote sensing images, water product images and knowledge product data obtained in step (1) are preprocessed, and a sample set with complete spatial coverage and continuous time series is randomly selected and divided into training set and test set according to a certain ratio.

[0045] The preprocessing in step (2) includes converting the river network product vector data into raster data and using bilinear interpolation to resample the JRC GSW water product image, DEM data and river network product data to a spatial resolution of 10m consistent with the Sentinel-2 multispectral remote sensing image to eliminate spatial scale differences and ensure spatial consistency of subsequent feature extraction.

[0046] The sample set consists of image sequences randomly selected from monthly multispectral remote sensing images, possessing complete spatial coverage and continuous temporal phases. Each sample comprises three parts: the original image sequence, a spatial attention layer, and a corresponding ground truth label layer. The spatial size of the image sequence is 128×128 pixels, and the time duration is 12 consecutive months; the corresponding tensor dimension is 12×C×128×128, where C represents the number of multispectral channels. The spatial attention layer consists of DEM data and river network product data, with a corresponding tensor dimension of 2×128×128. The ground truth label layer consists of JRC GSW monthly water body product images, with a corresponding tensor dimension of 12×128×128.

[0047] The data is divided into training and test sets according to a certain ratio: 70% is allocated to the training set and 30% to the test set.

[0048] (3) Construct a hybrid deep learning model structure consisting of a U-Net model structure and a Transformer module containing a spatial attention mechanism. The model structure receives the data to be processed by the U-Net encoder module, processes it sequentially by the Transformer module, and finally outputs it by the U-Net decoder module.

[0049] The hybrid deep learning model, consisting of the U-Net model structure and a Transformer module containing a spatial attention mechanism, includes:

[0050] 1) U-Net encoder module: For each image sequence X t ∈R C×128×128 Spatial feature extraction is performed on (t = 1, 2, ..., 12). The encoder includes a 3-layer convolutional structure, each layer consisting of a single 3×3 convolution, batch normalization, ReLU activation function, and 2×2 max pooling. The output feature expression of the l-th layer is:

[0051]

[0052] in The final output expression for the third layer is:

[0053]

[0054] 2) Transformer module for spatial attention mechanism: This module handles spatial attention layer information M∈R. 2×128×128 Spatial guided features M′∈R are obtained after three layers of 3×3 convolution and 2×2 max pooling downsampling operations. 64×16×16 Spatial attention maps are generated using 1×1 convolutions and a sigmoid activation function, expressed as follows:

[0055] A guide =σ(Conv 1×1 (M′));

[0056] Where σ(·) represents the Sigmoid activation function.

[0057] Encode the features F of each frame t Flattened into vector z t ∈R D D = 64 × 16 × 16, forming the input order matrix:

[0058] Z = [z1, z2, ..., z 12 ]∈R 12×D ;

[0059] Sequence modeling is performed using a standard Transformer encoder. To guide the attention mechanism to focus on spatially salient regions, a spatial bias term B is constructed. spatial ∈R 12×12 , by A guide Generated using methods such as average pooling and linear transformation, it is used to adjust the weight calculation process in the Transformer's self-attention.

[0060]

[0061] Where Q, K, and V are the query, key, and value matrices, respectively, and d k The dimension of the key vector.

[0062] Ultimately, the Transformer output incorporates a feature sequence that combines temporal dependencies and spatial concerns:

[0063] Z′=[z′1,z′2,…,z′ 12 ]∈R 12×D .

[0064] 3) U-Net decoder module: Reconstructs the multi-frame fused feature sequence Z′ output by the Transformer into a spatial feature map frame by frame. First, it reconstructs the spatial features from the vector of each frame:

[0065]

[0066] The data is then fed into the U-Net decoder, which uses 3 layers of transposed convolutions, batch normalization, and ReLU activation function for upsampling, outputting a probability map of size 1×128×128.

[0067]

[0068] After normalization using the Sigmoid activation function, a binary water mask image is generated using a thresholding method.

[0069]

[0070] The final output is a sequence of 12 binary water images.

[0071] (4) The weighted binary cross-entropy (WBCE) loss function and the Dice loss function are jointly optimized, and the training set is used to train the hybrid deep learning model to obtain the trained model, namely the water body extraction hybrid model.

[0072] The combined form of the weighted binary cross-entropy (WBCE) and the Dice loss function, expressed as the total loss function, is as follows:

[0073] Loss total =λ1·WBCE+λ2·DicLoss;

[0074] Where λ1 and λ2 are adjustable hyperparameters used to control the weights of the two types of losses during training, satisfying λ1,λ2≥0; WBCE is the weighted binary cross-entropy loss value; DicLoss is the Dice loss value.

[0075] The weighted binary cross-entropy loss (WBCE) is used to alleviate the problem of extreme imbalance in the ratio of positive to negative samples, and its expression is as follows:

[0076]

[0077] in, The output of the hybrid deep learning model is a sequence of binary water body images; M∈{0,1} is the corresponding ground truth label layer in the sample set; β is a class weighting factor used to increase the weighting of images of the minority class (water bodies), and is defined as:

[0078]

[0079] Where, N pos With N neg These represent the number of pixels in water bodies and non-water bodies, respectively.

[0080] The Dice loss function is used to measure the degree of regional overlap between the predicted map and the ground truth map, and it is defined as follows:

[0081]

[0082] Here, ∈ is a constant smoothing term used to avoid the denominator being zero. This term can effectively enhance the model's performance in boundary discrimination and small target recognition.

[0083] (5) The classification performance of the water body extraction mixture model is evaluated using the test set, and the accuracy and generalization ability of the model are verified by using F1 score, precision and recall.

[0084] (6) The water body extraction hybrid model is applied to the 10-meter spatial resolution multispectral remote sensing image in the target area. The image is segmented and reasoned using a sliding window method. The image is stitched and reconstructed by fusion of overlapping areas and edge smoothing. The water body distribution information of the target area is extracted, and a monthly water body distribution dataset with a 10-meter spatial resolution is generated.

[0085] The trained model was applied to multi-source remote sensing images with a spatial resolution of 10 meters within the target area. A sliding window approach was used to segment the entire image into patches. The sliding window was 128×128 pixels in size, with a time length of 12, and the window slid spatially in 10-pixel increments. Each image patch included corresponding multi-band remote sensing image information and corresponding knowledge product data, including DEM data and river network product data, which were input into the trained model as a spatial guide layer. The water distribution prediction results of each image patch were stitched together and reconstructed according to their spatial positions in the original images to form a water layer that completely covers the target area. Finally, the water distribution layers corresponding to each time slice were organized chronologically to generate a monthly water distribution dataset with a spatial resolution of 10 meters.

[0086] The above steps (1)-(6) achieve the following: Multi-source data fusion: By uniformly resampling to 10m resolution, the problem of spatial differences in multi-source data is solved, and the consistency of fusion is improved; Spatiotemporal feature joint modeling: U-Net extracts spatial details, Transformer captures temporal dependencies, and combined with spatial attention mechanism to focus on key areas of water bodies, thereby improving dynamic monitoring capabilities; Class imbalance optimization: WBCE and Dice loss are jointly optimized to alleviate the imbalance between positive and negative samples and enhance the accuracy of small water bodies and boundary identification; Efficient large-area coverage: Sliding window block inference achieves large-area coverage while maintaining 10m high resolution, balancing accuracy and efficiency.

[0087] The following specific example explains the implementation process of the remote sensing water extraction method based on the U-Net and Transformer fusion architecture in this embodiment.

[0088] We have existing Sentinel-2 monthly multispectral remote sensing images of the Yellow River Basin with a spatial resolution of 30m from 2018 to 2021, JRC GSW monthly water body product images of the Yellow River Basin with a spatial resolution of 30m from 2018 to 2021, ASTER GDEM V3 digital elevation model data of the Yellow River Basin, and GRWL river network product data, to obtain a 10m spatial resolution water body distribution dataset of the Yellow River Basin from 2018 to 2021.

[0089] SA acquired 10m spatial resolution Sentinel-2 multispectral remote sensing images of the Yellow River Basin from 2018 to 2021 using the Google Earth Engine (GEE) remote sensing cloud platform. These images underwent atmospheric correction, cloud detection and declouding, and median fusion of multiple images within the same month. The images include blue, green, red, near-infrared, shortwave infrared 1, and shortwave infrared 2 bands. The acquisition also included 30m spatial resolution JRC GSW monthly water product images of the Yellow River Basin from 2018 to 2021, as well as ASTER GDEM V3 digital elevation model data and GRWL river network product data.

[0090] SB preprocesses the data acquired by SA, converting GRWL river network product vector data into raster data. Bilinear interpolation is then used to resample JRC GSW water product images, ASTER GDEM V3 digital elevation model data, and GRWL river network product data to a spatial resolution of 10m, consistent with Sentinel-2 multispectral remote sensing images. A sample set is constructed, with each sample consisting of three parts: the original image sequence, a spatial attention layer, and a corresponding ground truth label layer. Image sequences with complete spatial coverage and continuous temporal phases are randomly selected from monthly multispectral remote sensing images. The spatial dimensions of the image sequence are 128×128 pixels, and the time period is 12 consecutive months; the corresponding tensor dimensions are 12×6×128×128. The spatial attention layer consists of ASTER GDEM V3 digital elevation model data and GRWL river network product data corresponding to the image sequence, with a corresponding tensor dimension of 2×128×128. The ground truth label layer consists of JRC GSW monthly water body product images corresponding to the image sequence, with a corresponding tensor dimension of 12×128×128.

[0091] SC constructs a hybrid deep learning model structure consisting of a U-Net model structure and a Transformer module with a spatial attention mechanism. The model structure receives data input from the U-Net encoder module, processes it sequentially through the Transformer module with spatial attention, and finally outputs it from the U-Net decoder module. The U-Net encoder module includes a three-layer convolutional structure, each layer consisting of a set of 3×3 convolutions, batch normalization, ReLU activation function, and 2×2 max pooling operations, used to extract image features layer by layer and perform downsampling. The Transformer module with spatial attention first extracts spatial attention layer information from the input encoded features, then performs downsampling through three layers of 3×3 convolutions and 2×2 max pooling to generate spatially guided features. The U-Net decoder module first performs multi-frame reconstruction on the fused feature sequence. Then, it generates a spatial attention map using a 1×1 convolution combined with a sigmoid activation function. The encoded features of each frame are flattened into vectors to form an input sequence matrix, which is then fed into a standard Transformer encoder for sequence modeling. Simultaneously, a spatial bias term is constructed using average pooling and linear mapping, and introduced into the Transformer's self-attention mechanism to adjust its attention weights, thereby outputting a feature sequence that fuses temporal dependence and spatial attention. The U-Net decoder module includes a three-layer upsampling structure, where each layer consists of transposed convolution, batch normalization, and a ReLU activation function to restore spatial resolution. The final output is a probability map with a size of 1×128×128. After normalization using a sigmoid activation function, a thresholding method is used to generate a binary water body mask map, resulting in a 12-frame sequence of binary water body images.

[0092] SD employs a weighted binary cross-entropy (WBCE) loss function and a Dice loss function for joint optimization. The hybrid deep learning model is trained on the training set, and the results are as follows: Figure 3 As shown: The AdamW optimizer is used during training, with the initial learning rate set to 3e-4, and cosine decay learning rate scheduling is combined until the error converges to the preset threshold. Training ends and the trained model, i.e., the water body extraction hybrid model, is obtained.

[0093] SE uses the test set to evaluate the classification performance of the water body extraction hybrid model, and uses F1 score, precision, and recall to verify the model's accuracy and generalization ability. The evaluation results show that the F1 score, precision, and recall of the imputed data are all within the expected range, verifying the accuracy and reliability of the imputed data.

[0094] SF applied the trained model to monthly multi-source Sentinel-2 remote sensing images of the Yellow River Basin with a spatial resolution of 10 meters. A sliding window approach was used to segment the entire image into patches. The sliding window was 128×128 pixels in size, with a time length of 12, and the window slid spatially in 10-pixel increments. Each image patch included corresponding multi-band remote sensing image information and corresponding knowledge product data, including ASTER GDEM V3 digital elevation model data and GRWL river network product data, which were input into the trained model as a spatial guide layer. The predicted water distribution results of each image patch were stitched together according to their spatial location in the original image to form a water layer that completely covered the target area. Finally, the water distribution layers corresponding to each time slice were organized chronologically to generate a 10-meter spatial resolution water distribution dataset for the Yellow River Basin from 2018 to 2021.

[0095] The method presented in this embodiment combines the spatial feature extraction capabilities of U-Net with the temporal modeling advantages of Transformer to achieve refined and dynamic extraction of water body information. By introducing a spatial attention mechanism to enhance boundary perception, and employing joint optimization with weighted cross-entropy and the Dice loss function, it effectively alleviates the class imbalance problem and improves the recognition accuracy for small-scale water bodies and complex background regions. Simultaneously, this embodiment possesses a high temporal update frequency and spatial resolution, capable of generating highly timely, monthly-scale water body distribution data products, exhibiting good generalization ability and practical application value. This method is suitable for large-scale, high-temporal-resolution water body change monitoring tasks, and can generate spatially detailed, temporally continuous high-resolution water body datasets.

[0096] Example 2

[0097] This embodiment provides a computer device, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 4As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores data related to a remote sensing water extraction method based on a U-Net and Transformer fusion architecture. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the remote sensing water extraction method based on a U-Net and Transformer fusion architecture as described in Embodiment 1.

[0098] Those skilled in the art will understand that Figure 4 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0099] Example 3

[0100] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a remote sensing water extraction method based on a U-Net and Transformer fusion architecture as described in Embodiment 1.

[0101] Example 4

[0102] This embodiment provides a computer program product, including a computer program that, when executed by a processor, implements a remote sensing water extraction method based on a U-Net and Transformer fusion architecture as described in Embodiment 1.

[0103] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0104] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0105] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0106] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0107] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A remote sensing water body extraction method based on a U-Net and Transformer fusion architecture, characterized in that, The remote sensing water body extraction method based on the U-Net and Transformer fusion architecture comprises: acquiring multi-spectral remote sensing images of a target region in a to-be-extracted month and corresponding knowledge product data, wherein the knowledge product data comprises digital elevation model data and river network product vector data; performing block processing on the multi-spectral remote sensing images and the knowledge product data to obtain a plurality of image blocks, wherein the image blocks comprise multi-spectral remote sensing image blocks and spatial attention map layer blocks; inputting the image blocks to output a binary water body mask graph by using a trained water body extraction hybrid model, wherein the water body extraction hybrid model is a deep learning model composed of a U-Net encoder module, a Transformer module with a spatial attention mechanism and a U-Net decoder module, the U-Net encoder module is used to extract spatial features of the multi-spectral remote sensing image blocks, the Transformer module is used to output a fusion feature sequence that fuses time dependence and spatial attention according to the spatial features and the spatial attention map layer blocks, and the U-Net decoder module is used to output a water body distribution probability graph according to the fusion feature sequence; the water body distribution probability graph is normalized by a Sigmoid activation function and processed by a threshold method to obtain the binary water body mask graph; splicing each binary water body mask graph according to spatial positions to obtain a complete water body distribution graph layer covering the target region in the to-be-extracted month; organizing the complete water body distribution graph layers of each to-be-extracted month in chronological order to generate a monthly water body distribution data set of the target region; the Transformer module comprises a spatial guide feature extraction layer, a spatial attention map generation layer, a sequence feature flattening layer, a spatial bias item construction layer and a Transformer encoder layer; the spatial guide feature extraction layer is used to perform 3-layer convolution operation on the spatial attention map layer blocks, each layer of convolution operation is matched with maximum pooling for down-sampling to generate spatial guide features; the spatial attention map generation layer is used to adjust channels of the spatial guide features by a convolution layer and generate a spatial attention map by combining a Sigmoid activation function layer to mark significant water body areas; the sequence feature flattening layer is used to flatten each frame of spatial features output by the U-Net encoder into a vector to form an input sequence matrix containing multi-frame time sequence information; the spatial bias item construction layer is used to generate a spatial bias item based on the spatial attention map to adjust self-attention weights; the Transformer encoder layer is used to adopt a standard Transformer encoder structure to model the input sequence matrix in time sequence by introducing a self-attention mechanism of the spatial bias item to output a fusion feature sequence that fuses time dependence and spatial attention.

2. The method of claim 1, wherein the U-Net and Transformer fusion architecture-based remote sensing water body extraction method is characterized by, The training process of the water body extraction hybrid model comprises: Acquire an initial sample data set, wherein the initial sample data set includes multispectral remote sensing image sample data, water body product image sample data, and knowledge product sample data of each target month in a target area and a target year; Preprocess the initial sample data set to obtain a training set; Train a water body extraction hybrid model based on the training set using a joint loss function until the water body extraction hybrid model converges, wherein the joint loss function includes a weighted binary cross-entropy loss function and a Dice loss function.

3. The method of claim 1, wherein the U-Net and Transformer fusion architecture-based remote sensing water body extraction method is characterized by, The U-Net encoder module includes three layers of convolution structures, and each layer of the convolution structure is composed of a convolution layer, a batch normalization layer, a ReLU activation function layer, and a maximum pooling layer.

4. The method of claim 1, wherein the U-Net and Transformer fusion architecture-based remote sensing water body extraction method is characterized by, The U-Net decoder module includes a feature reconstruction layer and three layers of up-sampling structures, and each layer of the up-sampling structure is composed of a transposed convolution, a batch normalization, and a ReLU activation function. The feature reconstruction layer is used to reshape the fusion feature sequence frame by frame vector to a spatial feature map and recover it to a spatial feature with the same dimension as the U-Net encoder output; The three layers of up-sampling structures are used to expand the spatial size of the spatial feature map through layer-by-layer up-sampling, and output a water body distribution probability map with the same spatial size as the multispectral remote sensing image.

5. The method of claim 2, wherein the U-Net and Transformer fusion architecture-based remote sensing water body extraction method is characterized by, The expression of the joint loss function is: ； wherein, is a joint loss function value; and is a tunable hyper-parameter satisfying ; is a weighted binary cross-entropy loss value; is a Dice loss value; The expression of the weighted binary cross-entropy loss function is: ； wherein, is a binary water mask map; is a corresponding ground truth label layer in the initial sample data set; is a class weight factor; The expression of the Dice loss function is: ； wherein, is a constant smoothing term.

6. The method of claim 2, wherein the U-Net and Transformer fusion architecture-based remote sensing water body extraction method is characterized by, The preprocessing includes converting river network product vector sample data into river network product raster data, and resampling water body product image sample data, digital elevation model sample data, and river network product raster data using bilinear interpolation until the spatial resolution of the water body product image sample data, the digital elevation model sample data, and the river network product raster data is consistent with the spatial resolution of the multispectral remote sensing image sample data.

7. A computer device comprising: Memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that the processor executes the computer program to implement the remote sensing water body extraction method based on the U-Net and Transformer fusion architecture of any one of claims 1-6.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the remote sensing water body extraction method based on the U-Net and Transformer fusion architecture of any one of claims 1-6.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the remote sensing water body extraction method based on the U-Net and Transformer fusion architecture of any one of claims 1-6.

Images