Remote sensing water body data set filling method based on multi-source knowledge and deep learning model

Abstract

The invention discloses a remote sensing water body data set filling method based on multi-source knowledge data and a deep learning model, and the method comprises the steps: obtaining a remote sensing image related to water body recognition in each target month in a target historical year of a target region through a remote sensing data platform, and combining the rainfall data in each target month in the target historical year of the target region; performing preliminary water body filling on information missing pixels in each remote sensing image by adopting a dynamic threshold value method; constructing a training data set by combining knowledge product data of a target area, training a constructed hybrid model structure in which a convolutional neural network CNN and a long short-term memory network LSTM are connected in series based on the training data set, and performing fine filling on each remote sensing image by using the trained model; and obtaining a remote sensing image without water body information loss in each target month in the target historical year in the target area. The method overcomes the defects of a traditional water body data filling technology, shows remarkable advantages under the conditions of complex terrains and changeable climates, and has wide application prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing data processing and water body change detection, and in particular relates to a remote sensing water body data set filling method based on multi-source knowledge and deep learning models. Background Art

[0002] With the global climate change and the intensification of human activities, the management and monitoring of water resources are facing increasingly severe challenges. As an effective monitoring method, remote sensing technology can provide large-scale and high-temporal resolution water data, which is widely used in water resources management, environmental monitoring and other fields. However, due to the influence of various factors, remote sensing water data sets with high temporal resolution, especially monthly-scale data sets, often have data missing problems. These problems will affect the integrity and accuracy of water data, thereby restricting its effectiveness in practical applications.

[0003] The main sources of remote sensing water data sets include optical remote sensing satellites (such as Landsat and Sentinel-2), radar remote sensing satellites (such as Sentinel-1), thermal infrared remote sensing satellites and other platforms. Although optical remote sensing data has high spatial and spectral resolution, it is unable to obtain surface information when there is severe cloud cover or atmospheric interference, resulting in missing water data in some areas. In addition, the limitations of sensor resolution make it difficult to accurately identify some small water bodies or water bodies in complex terrain, resulting in the inability to accurately record water data in these areas. The dynamic changes of water bodies (such as precipitation, evaporation, and seasonal changes) further aggravate the spatiotemporal inconsistency of the data and increase the difficulty of processing remote sensing data.

[0004] In order to deal with these problems, filling in missing data has become a key task in remote sensing water body data processing. At present, the methods for filling in water body data are mainly divided into traditional interpolation algorithms and modern machine learning-based methods. Although traditional interpolation methods such as linear interpolation and Kriging interpolation are simple to calculate and highly efficient, their filling accuracy is often not ideal when dealing with complex terrain and changing climate conditions. In contrast, modern methods based on machine learning, especially deep learning technology, can significantly improve the filling accuracy by learning the spatiotemporal characteristics of water bodies through a large amount of training data. However, existing deep learning methods mostly rely on a single data source, ignoring the important influence of terrain characteristics and river network structure on water body distribution, resulting in deficiencies in spatial consistency and physical rationality of the filling results.

[0005] Therefore, combining multi-source knowledge data such as precipitation data, digital elevation model (DEM), river network data, etc. with deep learning technology to fill water body data has become an important research direction in the current field of remote sensing data processing. By introducing multi-source data, not only can the accuracy of data filling be improved, but also the spatiotemporal consistency and physical rationality of water body data can be enhanced, especially under complex terrain and variable climate conditions. This method can generate high-quality and complete water body data sets, further enhancing its application value in fields such as water resources management and environmental monitoring. Summary of the invention

[0006] In order to overcome the problems existing in existing water body data filling methods, such as insufficient filling accuracy, poor spatial consistency, and ineffective response to data missing problems under complex terrain and changeable climate conditions, the present invention provides a remote sensing water body dataset filling method based on multi-source knowledge data and deep learning models.

[0007] To achieve the above object, the present invention adopts the following technical solutions: A remote sensing water body dataset filling method based on multi-source knowledge data and deep learning model, the method includes the following steps: Step 1: Obtain remote sensing images for water body identification in each target month in each target historical year of the target area; Step 2: Obtain the precipitation data for each target month in each target historical year in the target area, and calculate the seasonal precipitation characteristics of the target area; Step 3: Perform preliminary water body filling on each remote sensing image obtained in step 1 based on the dynamic threshold method to obtain a remote sensing image set with preliminary water body filling; Step 4: preprocess the knowledge product data in the target historical year period of the target area, wherein the knowledge product data includes the river network product data and digital elevation model data of the target area, and combine the preprocessed river network product data and digital elevation model data with the remote sensing image set of preliminary water body filling to form a training data set; Step 5: Construct a hybrid model structure of a convolutional neural network (CNN) model structure and a long short-term memory (LSTM) model structure in series: data is input from the input layer of the CNN model structure, the CNN model structure is connected in series with the LSTM model structure, the output of the LSTM model structure is connected to a fully connected layer, the fully connected layer is connected to the output layer, and the data processing result is output by the output layer; Step 6: Based on the training data set, the remote sensing images of preliminary water body filling in each target month in each target historical year, the preprocessed river network product data and the digital elevation model data are used as input, and the water body distribution position images in each target month in each target historical year are used as output to train the hybrid model structure to obtain a trained model, i.e., a water body filling hybrid model; Step 7: Use the water body filling hybrid model to finely fill the remote sensing image set with preliminary water body filling, and obtain remote sensing images with no missing water body information in each target month in the target historical year of the target area.

[0008] Further, the step 1 is specifically as follows: The remote sensing images obtained through the public remote sensing data platform include water body pixels, non-water body pixels and information missing pixels; among them, water body pixels are marked as water bodies, and non-water body pixels such as houses and vegetation are marked as non-water bodies; due to factors such as cloud obstruction and atmospheric interference, some pixel data may be missing, and these pixels are marked as information missing pixels.

[0009] According to the precipitation data of each target month in the target historical year of the target area, the precipitation anomaly percentage is calculated to represent the seasonal precipitation characteristics of the target area. The calculation formula is as follows: in, i Represents each target historical year, j Represents each target month, D i,j is the precipitation anomaly percentage for each target month in each target historical year, P i,j is the actual precipitation in each target month in each target historical year, P avg,j It is the long-term average precipitation corresponding to each target month in the target historical year period.

[0010] Further, the step three is specifically as follows: According to the seasonal precipitation characteristics of the target area, the dynamic threshold method is used to preliminarily fill in the information missing pixels in each remote sensing image obtained in step 1; the dynamic threshold method dynamically updates the thresholds of water bodies and non-water bodies based on the climate characteristics and terrain conditions of the target area and the seasonal precipitation characteristic values ​​of the target area; the dynamic threshold method includes two stages: dynamic threshold setting and dynamic threshold judgment: S1. The setting process of the dynamic threshold is as follows: ①First, based on the climate characteristics of the target area, the preliminary benchmark water body thresholds and benchmark non-water body thresholds for different seasons are preset for the different seasons of the target area; ② On this basis, the benchmark water body threshold and non-water body threshold are dynamically adjusted according to the seasonal precipitation characteristic values ​​of the target area to obtain the adjusted water body threshold and adjusted non-water body threshold for each target month in each target historical year. The adjustment method is: Adjusted water body threshold j = Base water body threshold j- D i,j / 10, Adjusted non-water body threshold j = Baseline non-water body threshold j + D i,j / 10; in, D i,j The range of is limited to [-1, 1] to prevent over-adjustment; S2. In the dynamic threshold judgment stage, all pixels in each remote sensing image are counted to calculate the proportion of water and non-water bodies. The filling rules are as follows: ① If the proportion of water bodies exceeds the adjusted water body threshold, the information missing pixels in the corresponding remote sensing image will be filled with water bodies; ② If the proportion of non-water bodies exceeds the adjusted non-water body threshold, the information missing pixels in the corresponding remote sensing image will be filled as non-water bodies; ③ If the proportions of both do not reach their respective adjusted thresholds, the pixel will be kept as an information-missing pixel and no filling will be performed.

[0011] Further, the step 4 is specifically as follows: The river network product data is vector data, and the digital elevation model data is raster data. The river network product data and the digital elevation model data are preprocessed. The preprocessing process includes converting the river network product vector data into raster data, cropping, resampling and projection conversion of the converted river network product data and digital elevation model data, and unifying their spatial resolution and coordinate system with the remote sensing image set of preliminary water body filling to ensure the precise alignment of all data in spatial position; the preprocessed river network product data, digital elevation model data and the remote sensing image set of preliminary water body filling are combined to form a training data set.

[0012] Furthermore, the hybrid model structure constructed in step 5 is specifically as follows: The CNN model structure includes an input layer, a convolution layer and a pooling layer. Data is input from the input layer, and multimodal features are extracted through convolution and pooling. The convolution layer uses 64 3×3 convolution kernels. The convolution operation can be expressed as: in x is the input data, W conv is the convolution kernel weight, b conv is the bias term, RELU is the activation function; The MaxPooling layer is used to reduce the spatial dimension. The pooling kernel size used is 2×2 and the stride is 2. The output after pooling is expressed as: The spatial features output by the CNN model structure are flattened and input into the LSTM model structure for temporal feature extraction. The LSTM model structure includes an LSTM layer, which contains 128 cell units. Each cell unit is divided into a forget gate, an input gate and an output gate. The expressions of the control coefficients of the forget gate, the input gate and the output gate are: {f}_{t}=\sigma ({W}_{f}\cdot [{h}_{t-1},{x}_{t}]+{b}_{f}) {i}_{t}=\sigma ({W}_{i}\cdot [{h}_{t-1},{x}_{t}]+{b}_{i}) {O}_{t}=\sigma ({W}_{o}\cdot [{h}_{t-1},{x}_{t}]+{b}_{o}) in, f t is the control coefficient of the forget gate, W f is the weight matrix of the forget gate, h t-1 is the implicit layer state information of the previous moment, x t is the input value at the current moment, b f is the bias top of the forget gate, σ is the activation function Sigmoid, i t is the control coefficient of the input gate, W i is the weight matrix of the input gate, b i is the bias top of the input gate, W o is the weight matrix of the output gate, b o is the bias top of the output gate; The forget gate and the input gate are combined to obtain the cell state at the current moment. The expression of the cell state at the current moment is as follows: {C}_{t}={f}_{t}\ast {i}_{t-1}+{i}_{t}\ast tanh({W}_{C}\cdot [{h}_{t-1},{x}_{t}]+{b}_{C}) in, C t is the cell state at the current moment, C t-1 is the cell state at the previous moment, W c is the weight matrix of the cell state at the current moment, tanh() is the activation function, b C is the offset top of the cell state at the current moment; Based on the current cell state and combined with the output gate, the current hidden layer state is obtained. The expression of the current hidden layer state is as follows: The output of the LSTM model structure is connected to the fully connected layer, and the prediction results are combined and output through the fully connected layer. The combination process can be expressed as: W fc and b fc is the weight and bias process of the fully connected layer, h is the output of the LSTM layer, σ is the activation function.

[0013] Further, the hybrid model structure constructed in step 5 is trained to obtain a trained model. The training process is as follows: Based on the training data set, combined with the minimization of the mean square error (MSE) loss function, the hybrid model structure is trained using the back propagation algorithm and the Adam optimizer. The model parameters are updated through multiple iterations until the error converges to a preset threshold. The training is completed and the trained model, i.e., the water body filling hybrid model, is obtained. The loss function is expressed as follows: ，y i is the actual value, is the model prediction value, n is the number of samples.

[0014] Furthermore, the water filling hybrid model is applied to finely fill the remote sensing image set with preliminary water filling. The specific process is as follows: S1. Perform preliminary prediction: Apply the water filling hybrid model to the remote sensing image set of preliminary water filling to generate preliminary prediction results; S2. Perform random missing evaluation: randomly select some non-missing pixels from the preliminary prediction results, set them as missing values, simulate random missing pixels, and reapply the water body filling hybrid model to fill them; compare the filling results with the corresponding preliminary water body filling remote sensing images, calculate the error, and evaluate the model filling effect; S3, iterative repair: According to the error between the preliminary prediction results and the remote sensing image filled with preliminary water bodies, the model input is adjusted and the model is retrained for iterative repair. The formula is: ,in, y new is the updated prediction result, y initial is the initial forecast value, y observed is the observation value of the remote sensing image for preliminary water filling, α is the learning rate, is the predicted value missing at random; S4. Verification of spatiotemporal consistency: After each iterative repair, evaluate the spatiotemporal consistency of the filling results to ensure the continuity and rationality of the filled data in time and space dimensions, and finally obtain a set of remote sensing images with no missing water body information in each target month within the target historical year of the target area.

[0015] Furthermore, by comparing the remote sensing images with no missing water body information in the target month in the target historical year of the target area with the Sentinel satellite images of water body information in the target area and the hydrological measured data of the same period, a variety of evaluation indicators including mean square error (MSE) and correlation coefficient (R²) were used to evaluate the accuracy of water body filling and the overall water body data availability to verify the accuracy and reliability of the filled data.

[0016] Compared with the prior art, the technical solution of the present invention has the following beneficial effects: The remote sensing water body dataset filling method based on multi-source knowledge data and deep learning models provided by the present invention effectively overcomes the problems of insufficient filling accuracy, poor spatial consistency and poor filling effect under complex terrain in the existing methods; by introducing seasonal precipitation characteristics, river network product data and DEM data, the accuracy and applicability of the filling model are enhanced; the high-precision remote sensing image set with no missing water body information finally generated has broad application prospects in water resources management, environmental monitoring, remote sensing data analysis and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of the method of the present invention;

[0018] Figure 2 A hybrid model architecture diagram in which a CNN model structure and an LSTM model structure constructed by the present invention are connected in series;

[0019] Figure 3 It is a schematic diagram of the water filling process in a specific implementation manner of the present invention. DETAILED DESCRIPTION

[0020] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0021] like Figure 1 As shown, the remote sensing water body data set filling method based on multi-source knowledge data and deep learning model provided by the present invention comprises the following steps: Step 1: Obtain remote sensing images for water body identification in each target month in each target historical year of the target area; Step 2: Obtain the precipitation data for each target month in each target historical year in the target area, and calculate the seasonal precipitation characteristics of the target area; Step 3: Perform preliminary water body filling on each remote sensing image obtained in step 1 based on the dynamic threshold method to obtain a remote sensing image set with preliminary water body filling; Step 4: preprocess the knowledge product data in the target historical year period of the target area, wherein the knowledge product data includes the river network product data and digital elevation model data of the target area, and combine the preprocessed river network product data and digital elevation model data with the remote sensing image set of preliminary water body filling to form a training data set; Step 5: Construct a hybrid model structure of a convolutional neural network (CNN) model structure and a long short-term memory (LSTM) model structure in series: data is input from the input layer of the CNN model structure, the CNN model structure is connected in series with the LSTM model structure, the output of the LSTM model structure is connected to a fully connected layer, the fully connected layer is connected to the output layer, and the data processing result is output by the output layer; Step 6: Based on the training data set, the remote sensing images of preliminary water body filling in each target month in each target historical year, the preprocessed river network product data and the digital elevation model data are used as input, and the water body distribution position images in each target month in each target historical year are used as output to train the hybrid model structure to obtain a trained model, i.e., a water body filling hybrid model; Step 7: Use the water body filling hybrid model to finely fill the remote sensing image set with preliminary water body filling, and obtain remote sensing images with no missing water body information in each target month in the target historical year of the target area.

[0022] Further, the step 1 is specifically as follows: The remote sensing images obtained through the public remote sensing data platform include water body pixels, non-water body pixels and information missing pixels; among them, water body pixels are marked as water bodies, and non-water body pixels such as houses and vegetation are marked as non-water bodies; due to factors such as cloud obstruction and atmospheric interference, some pixel data may be missing, and these pixels are marked as information missing pixels.

[0023] Further, the step 2 is specifically as follows: According to the precipitation data of each target month in the target historical year of the target area, the precipitation anomaly percentage is calculated to represent the seasonal precipitation characteristics of the target area. The calculation formula is as follows: in, i Represents each target historical year, j Represents each target month, D i,j is the precipitation anomaly percentage for each target month in each target historical year, P i,j is the actual precipitation in each target month in each target historical year, P avg,j It is the long-term average precipitation corresponding to each target month in the target historical year period.

[0024] Further, the step three is specifically as follows: According to the seasonal precipitation characteristics of the target area, the dynamic threshold method is used to preliminarily fill in the information missing pixels in each remote sensing image obtained in step 1; the dynamic threshold method dynamically updates the thresholds of water bodies and non-water bodies based on the climate characteristics and terrain conditions of the target area and the seasonal precipitation characteristic values ​​of the target area; the dynamic threshold method includes two stages: dynamic threshold setting and dynamic threshold judgment: S1. The setting process of the dynamic threshold is as follows: ①First, based on the climate characteristics of the target area, the preliminary benchmark water body thresholds and benchmark non-water body thresholds for different seasons are preset for the different seasons of the target area; ② On this basis, the benchmark water body threshold and non-water body threshold are dynamically adjusted according to the seasonal precipitation characteristic values ​​of the target area to obtain the adjusted water body threshold and adjusted non-water body threshold for each target month in each target historical year. The adjustment method is: Adjusted water body threshold j = Base water body threshold j - D i,j / 10, Adjusted non-water body threshold j = Baseline non-water body threshold j+ D i,j / 10; in, D i,j The range of is limited to [-1, 1] to prevent over-adjustment; S2. In the dynamic threshold judgment stage, all pixels in each remote sensing image are counted to calculate the proportion of water and non-water bodies. The filling rules are as follows: ① If the proportion of water bodies exceeds the adjusted water body threshold, the information missing pixels in the corresponding remote sensing image will be filled with water bodies; ② If the proportion of non-water bodies exceeds the adjusted non-water body threshold, the information missing pixels in the corresponding remote sensing image will be filled as non-water bodies; ③ If the proportions of both do not reach their respective adjusted thresholds, the pixel will be kept as an information-missing pixel and no filling will be performed.

[0025] Further, the step 4 is specifically as follows: The river network product data is vector data, and the digital elevation model data is raster data. The river network product data and the digital elevation model data are preprocessed. The preprocessing process includes converting the river network product vector data into raster data, cropping, resampling and projection conversion of the converted river network product data and digital elevation model data, and unifying their spatial resolution and coordinate system with the remote sensing image set of preliminary water body filling to ensure the precise alignment of all data in spatial position; the preprocessed river network product data, digital elevation model data and the remote sensing image set of preliminary water body filling are combined to form a training data set.

[0026] Furthermore, the hybrid model structure constructed in step 5 is specifically as follows: The CNN model structure includes an input layer, a convolution layer and a pooling layer. Data is input from the input layer, and multimodal features are extracted through convolution and pooling. The convolution layer uses 64 3×3 convolution kernels. The convolution operation can be expressed as: in x is the input data, W conv is the convolution kernel weight, b conv is the bias term, RELU is the activation function; The MaxPooling layer is used to reduce the spatial dimension. The pooling kernel size used is 2×2 and the stride is 2. The output after pooling is expressed as: The spatial features output by the CNN model structure are flattened and input into the LSTM model structure for temporal feature extraction. The LSTM model structure includes an LSTM layer, which contains 128 cell units. Each cell unit is divided into a forget gate, an input gate, and an output gate. The expressions of the control coefficients of the forget gate, the input gate, and the output gate are respectively: {f}_{t}=\sigma ({W}_{f}\cdot [{h}_{t-1},{x}_{t}]+{b}_{f}) {i}_{t}=\sigma ({W}_{i}\cdot [{h}_{t-1},{x}_{t}]+{b}_{i}) {O}_{t}=\sigma ({W}_{o}\cdot [{h}_{t-1},{x}_{t}]+{b}_{o}) in, f t is the control coefficient of the forget gate, W f is the weight matrix of the forget gate, h t-1 is the implicit layer state information of the previous moment, x t is the input value at the current moment, b f is the bias top of the forget gate, σ is the activation function Sigmoid, i t is the control coefficient of the input gate, W i is the weight matrix of the input gate, b i is the bias top of the input gate, W o is the weight matrix of the output gate, b o is the bias top of the output gate; The forget gate and the input gate are combined to obtain the cell state at the current moment. The expression of the cell state at the current moment is as follows: {C}_{t}={f}_{t}\ast {i}_{t-1}+{i}_{t}\ast tanh({W}_{C}\cdot [{h}_{t-1},{x}_{t}]+{b}_{C}) in, C t is the cell state at the current moment, Ct-1 is the cell state at the previous moment, W c is the weight matrix of the cell state at the current moment, tanh() is the activation function, b C is the offset top of the cell state at the current moment; Based on the current cell state and combined with the output gate, the current hidden layer state is obtained. The expression of the current hidden layer state is as follows: The output of the LSTM model structure is connected to the fully connected layer, and the prediction results are combined and output through the fully connected layer. The combination process can be expressed as: W fc and b fc is the weight and bias process of the fully connected layer, h is the output of the LSTM layer, σ is the activation function.

[0027] Further, the hybrid model structure constructed in step 5 is trained to obtain a trained model. The training process is as follows: Based on the training data set, combined with the minimization of the mean square error (MSE) loss function, the hybrid model structure is trained using the back propagation algorithm and the Adam optimizer. The model parameters are updated through multiple iterations until the error converges to a preset threshold. The training is completed and the trained model, i.e., the water body filling hybrid model, is obtained. The loss function is expressed as follows: ，y i is the actual value, is the model prediction value, n is the number of samples.

[0028] Furthermore, the water filling hybrid model is applied to finely fill the remote sensing image set with preliminary water filling. The specific process is as follows: S1. Perform preliminary prediction: Apply the water filling hybrid model to the remote sensing image set of preliminary water filling to generate preliminary prediction results; S2. Perform random missing evaluation: randomly select some non-missing pixels from the preliminary prediction results, set them as missing values, simulate random missing pixels, and reapply the water body filling hybrid model to fill them; compare the filling results with the corresponding preliminary water body filling remote sensing images, calculate the error, and evaluate the model filling effect; S3, iterative repair: According to the error between the preliminary prediction results and the remote sensing image filled with preliminary water bodies, the model input is adjusted and the model is retrained for iterative repair. The formula is: ,in, y new is the updated prediction result, y initial is the initial forecast value, y observed is the observation value of the remote sensing image for preliminary water filling, α is the learning rate, is the predicted value missing at random; S4. Verification of spatiotemporal consistency: After each iterative repair, evaluate the spatiotemporal consistency of the filling results to ensure the continuity and rationality of the filled data in time and space dimensions, and finally obtain a set of remote sensing images with no missing water body information in each target month within the target historical year of the target area.

[0029] Furthermore, by comparing the remote sensing images with no missing water body information in the target month in the target historical year of the target area with the Sentinel satellite images of water body information in the target area and the hydrological measured data of the same period, a variety of evaluation indicators including mean square error (MSE) and correlation coefficient (R²) were used to evaluate the accuracy of water body filling and the overall water body data availability to verify the accuracy and reliability of the filled data. Example

[0030] The existing target area has remote sensing images for water body identification in each month from 1990 to 2021 with a spatial resolution of 30m, precipitation data in each month from 1990 to 2021, river network product data and digital elevation model (DEM) data in the corresponding years of the target area; according to the method of the present invention, the water body data filling process of the area is as follows: Figure 3 As shown, specifically: Step 1: Obtain remote sensing images of water body identification in the target area for each month from 1990 to 2021 through a public remote sensing data platform; the remote sensing images are based on long-term satellite observations of the earth's surface and are processed by professional classification algorithms to provide the distribution of water and non-water areas in each month; water body pixels are marked as water bodies, and non-water areas such as houses and vegetation are marked as non-water bodies; due to factors such as cloud cover and atmospheric interference, some pixel data may be missing, and these pixel data are marked as information-missing pixels; Step 2: Based on the monthly precipitation data of the target area from 1990 to 2021, the precipitation anomaly percentage is calculated to capture the seasonal precipitation characteristics of the target area; Step 3: Based on the seasonal precipitation characteristics of the target area, the dynamic threshold method is used to perform preliminary water filling of information-missing pixels in each remote sensing image, and a remote sensing image set with preliminary water filling is obtained, which lays the foundation for further deep learning model processing; Step 4. The knowledge product data includes river network product data and digital elevation model (DEM) data of the target area. The river network product data is vector data, and the digital elevation model data is raster data. The river network product data and the digital elevation model data are preprocessed. The preprocessing process includes converting the river network product vector data into raster data, clipping, resampling and projecting the converted river network product data and digital elevation model data, and unifying their spatial resolution and coordinate system with the remote sensing image set of preliminary water body filling to ensure that all data are accurately aligned in spatial position; the preprocessed river network product data, digital elevation model data and the remote sensing image set of preliminary water body filling are combined to form a training data set to provide input data for the hybrid model structure of subsequent water body filling to ensure that the hybrid model structure can fully utilize the spatial and geographic feature information of the target area; Step 5. Construct a hybrid model structure in which a CNN model structure and an LSTM model structure are connected in series to capture and simulate the complex spatiotemporal characteristics of water body distribution; the CNN model structure includes an input layer, a convolution layer and a maximum pooling layer. Data is input from the input layer of the CNN model structure and output to the LSTM model structure after convolution and pooling. The LSTM model structure includes an LSTM layer, which contains 128 cell units, each of which is composed of a forget gate, an input gate and an output gate. The output of the LSTM model structure is connected to a fully connected layer, and the fully connected layer is connected to the output layer; the hybrid model structure performs convolution processing on the input preliminary filled water body data set, river network product data and digital elevation model (DEM) data through the CNN model structure, extracts its spatial features in a multi-modal manner, and reduces the spatial dimension of the data through a maximum pooling layer; then, the spatial features output by the CNN model structure are flattened and input into the LSTM model structure. The LSTM model structure captures the dynamic changes and long-term dependencies of water body data through its time series processing capabilities; finally, the output is combined through a fully connected layer; Step 6: Based on the training data set, combined with the minimization of the mean square error (MSE) loss function, the back propagation algorithm and the Adam optimizer are used to train the hybrid model structure. The model parameters are updated through multiple iterations until the error converges to the preset threshold. The training is completed and the trained model, i.e., the water body filling hybrid model, is obtained. Step 7. Use the water body filling hybrid model to fine-fill the remote sensing image set with preliminary water body filling: first, apply the water body filling hybrid model to the remote sensing image set with preliminary water body filling to generate preliminary prediction results; then, randomly select some non-missing pixels from the preliminary prediction results, reapply the water body filling hybrid model to fill in, simulate random missing pixels, and compare with the preliminary water body filled remote sensing images to evaluate the error; finally, through iterative repair and spatiotemporal consistency verification, ensure the accuracy and continuity of the filled water body data; finally, obtain remote sensing images of the target area with no missing water body information in each month from 1990 to 2021.

[0031] Finally, by comparing the remote sensing images of the target area without missing water body information in each month from 1990 to 2021 with the Sentinel satellite images of water body information in the target area from 1990 to 2021 and the hydrological measured data of the same period, the error value was calculated and the accuracy of the filling results was evaluated; the evaluation results showed that the mean square error (MSE) and correlation coefficient (R²) of the filled data were within the expected range, verifying the accuracy and reliability of the filled data.

[0032] The remote sensing image set finally restored by this method can provide high-precision support in practical applications such as water resources management and environmental monitoring, reflecting its effectiveness and application value in actual scenarios.

Claims

1. A remote sensing water body dataset filling method based on multi-source knowledge and deep learning model, characterized by: The method comprises the following steps: Step 1: Obtain remote sensing images for water body identification in each target month in each target historical year of the target area; Step 2: Obtain the precipitation data for each target month in each target historical year in the target area, and calculate the seasonal precipitation characteristics of the target area; Step 3: Perform preliminary water body filling on each remote sensing image obtained in step 1 based on the dynamic threshold method to obtain a remote sensing image set with preliminary water body filling; Step 4: preprocess the knowledge product data in the target historical year period of the target area, wherein the knowledge product data includes the river network product data and digital elevation model data of the target area, and combine the preprocessed river network product data and digital elevation model data with the remote sensing image set of preliminary water body filling to form a training data set; Step 5: Construct a hybrid model structure of a convolutional neural network (CNN) model structure and a long short-term memory (LSTM) model structure in series: data is input from the input layer of the CNN model structure, the CNN model structure is connected in series with the LSTM model structure, the output of the LSTM model structure is connected to a fully connected layer, the fully connected layer is connected to the output layer, and the data processing result is output by the output layer; Step 6: Based on the training data set, the remote sensing images of preliminary water body filling in each target month in each target historical year, the preprocessed river network product data and the digital elevation model data are used as input, and the water body distribution position images in each target month in each target historical year are used as output to train the hybrid model structure to obtain a trained model, i.e., a water body filling hybrid model; Step 7: Use the water body filling hybrid model to finely fill the remote sensing image set with preliminary water body filling, and obtain remote sensing images with no missing water body information in each target month in the target historical year of the target area.

2. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 1 is characterized by: Each remote sensing image obtained in step 1 includes water body pixels, non-water body pixels and information missing pixels.

3. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 2 is characterized by: The step 2 is specifically as follows: According to the precipitation data of each target month in the target historical year of the target area, the precipitation anomaly percentage is calculated to represent the seasonal precipitation characteristics of the target area. The calculation formula is as follows: in, i Represents each target historical year, j Represents each target month, D i,j is the precipitation anomaly percentage for each target month in each target historical year, P i,j is the actual precipitation in each target month in each target historical year, P avg,j It is the long-term average precipitation corresponding to each target month in the target historical year period.

4. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 3 is characterized by: The step three is specifically as follows: According to the seasonal precipitation characteristics of the target area, the dynamic threshold method is used to preliminarily fill in the information missing pixels in each remote sensing image obtained in step 1; the dynamic threshold method dynamically updates the thresholds of water bodies and non-water bodies based on the climate characteristics and terrain conditions of the target area and the seasonal precipitation characteristic values ​​of the target area; the dynamic threshold method includes two stages: dynamic threshold setting and dynamic threshold judgment: S1. The setting process of the dynamic threshold is as follows: ①First, based on the climate characteristics of the target area, the preliminary benchmark water body thresholds and benchmark non-water body thresholds for different seasons are preset for the different seasons of the target area; ② On this basis, the benchmark water body threshold and non-water body threshold are dynamically adjusted according to the seasonal precipitation characteristic values ​​of the target area to obtain the adjusted water body threshold and adjusted non-water body threshold for each target month in each target historical year. The adjustment method is: Adjusted water body threshold j = Base water body threshold j - D i,j / 10, Adjusted non-water body threshold j = Baseline non-water body threshold j + D i,j / 10; in, D i,j The range of is limited to [-1, 1]; S2. In the dynamic threshold judgment stage, all pixels in each remote sensing image are counted to calculate the proportion of water and non-water bodies. The filling rules are as follows: ① If the proportion of water bodies exceeds the adjusted water body threshold, the information missing pixels in the corresponding remote sensing image will be filled with water bodies; ② If the proportion of non-water bodies exceeds the adjusted non-water body threshold, the information missing pixels in the corresponding remote sensing image will be filled as non-water bodies; ③ If the proportions of both do not reach their respective adjusted thresholds, the pixel will be kept as an information-missing pixel and no filling will be performed.

5. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 4 is characterized by: The step 4 is specifically as follows: The river network product data is vector data, and the digital elevation model data is raster data. The river network product data and the digital elevation model data are preprocessed. The preprocessing process includes converting the river network product vector data into raster data, clipping, resampling and projecting the converted river network product data and digital elevation model data, and unifying their spatial resolution and coordinate system with the remote sensing image set of preliminary water body filling to ensure accurate alignment of all data in spatial position; The preprocessed river network product data, digital elevation model data and remote sensing image sets with preliminary water body filling are combined to form a training data set.

6. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 5 is characterized by: The hybrid model structure constructed in step 5 is specifically as follows: The CNN model structure includes an input layer, a convolution layer and a pooling layer. Data is input from the input layer, and multimodal features are extracted through convolution and pooling. The convolution layer uses 64 3×3 convolution kernels. The convolution operation can be expressed as: in x is the input data, W conv is the convolution kernel weight, b conv is the bias term, RELU is the activation function; The MaxPooling layer is used to reduce the spatial dimension. The pooling kernel size used is 2×2 and the stride is 2. The output after pooling is expressed as: ； The spatial features output by the CNN model structure are flattened and input into the LSTM model structure for temporal feature extraction. The LSTM model structure includes an LSTM layer, which contains 128 cell units. Each cell unit is divided into a forget gate, an input gate, and an output gate. The expressions of the control coefficients of the forget gate, the input gate, and the output gate are respectively: in, f t is the control coefficient of the forget gate, W f is the weight matrix of the forget gate, h t-1 is the implicit layer state information of the previous moment, x t is the input value at the current moment, b f is the bias top of the forget gate, σ is the activation function Sigmoid, i t is the control coefficient of the input gate, W i is the weight matrix of the input gate, b i is the bias top of the input gate, W o is the weight matrix of the output gate, b o is the bias top of the output gate; The forget gate and the input gate are combined to obtain the cell state at the current moment. The expression of the cell state at the current moment is as follows: in, C t is the cell state at the current moment, C t-1 is the cell state at the previous moment, W c is the weight matrix of the cell state at the current moment, tanh() is the activation function, b C is the offset top of the cell state at the current moment; Based on the current cell state and combined with the output gate, the current hidden layer state is obtained. The expression of the current hidden layer state is as follows: The output of the LSTM model structure is connected to the fully connected layer, and the prediction results are combined and output through the fully connected layer. The combination process can be expressed as: W fc and b fc is the weight and bias process of the fully connected layer, h is the output of the LSTM layer, σ is the activation function.

7. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 6 is characterized by: The hybrid model structure constructed in step 5 is trained to obtain a trained model. The training process is as follows: Based on the training data set, combined with the minimization of the mean square error (MSE) loss function, the hybrid model structure is trained using the back propagation algorithm and the Adam optimizer. The model parameters are updated through multiple iterations until the error converges to a preset threshold. The training is completed and the trained model, i.e., the water body filling hybrid model, is obtained. The loss function is expressed as follows: ，y i is the actual value, is the model prediction value, n is the number of samples.

8. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 7 is characterized by: The water filling hybrid model is applied to finely fill the remote sensing image set with preliminary water filling. The specific process is as follows: S1. Perform preliminary prediction: Apply the water filling hybrid model to the remote sensing image set of preliminary water filling to generate preliminary prediction results; S2. Perform random missing evaluation: randomly select some non-missing pixels from the preliminary prediction results, set them as missing values, and reapply the water body filling hybrid model to fill them; compare the filling results with the corresponding preliminary water body filling remote sensing images, calculate the error, and evaluate the model filling effect; S3, iterative repair: According to the error between the preliminary prediction results and the remote sensing image of the preliminary water body filling, adjust the model input and retrain the model, and perform multiple iterative repairs. The formula is: ,in, y new is the updated prediction result, y initial is the initial forecast value, y observed is the observation value of the remote sensing image for preliminary water filling, α is the learning rate, is the predicted value missing at random; S4. Verification of spatiotemporal consistency: After each iterative repair, evaluate the spatiotemporal consistency of the filling results to ensure the continuity and rationality of the filled data in time and space dimensions, and finally obtain a set of remote sensing images with no missing water body information in each target month within the target historical year of the target area.

9. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 8 is characterized by: The remote sensing images without missing water body information in the target month of the target historical year of the target area are compared with the Sentinel satellite images of water body information in the target area and the hydrological measured data of the same period. Evaluation indicators including mean square error (MSE) and correlation coefficient (R²) are used to evaluate the water body filling accuracy and the overall water body data availability.

Images