Remote sensing water body dataset filling method based on multi-source knowledge and deep learning model

Abstract

The present invention discloses a method for filling a remote sensing water body dataset based on multi-source knowledge data and a deep learning model. First, remote sensing images for water body identification in target regions and target months within target historical years are obtained through a remote sensing data platform. Combined with precipitation data for target regions and target months within target historical years, a dynamic threshold method is used to perform preliminary water body filling for pixels with missing information in each remote sensing image. A training dataset is constructed based on the target region's knowledge product data. A hybrid model structure, consisting of a convolutional neural network (CNN) and a long short-term memory (LSTM) network connected in series, is trained based on the training dataset. The trained model is used to perform fine filling of each remote sensing image. Remote sensing images with no missing water body information in target months within target historical years are obtained. This method overcomes the shortcomings of traditional water body data filling technologies, exhibits significant advantages in complex terrain and variable climate conditions, and has broad application prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing data processing and water body change detection, and in particular relates to a remote sensing water body dataset filling method based on multi-source knowledge and deep learning models. Background Art

[0002] With global climate change and intensified human activities, water resource management and monitoring face increasingly severe challenges. Remote sensing technology, as an effective monitoring tool, can provide large-scale, high-temporal-resolution water data, making it widely used in water resource management and environmental monitoring. However, due to various factors, remote sensing water datasets with high temporal resolution, particularly those at the monthly scale, often suffer from data gaps. These issues affect the integrity and accuracy of water data, thereby limiting its effectiveness in practical applications.

[0003] The main sources of remote sensing water datasets include optical remote sensing satellites (such as Landsat and Sentinel-2), radar remote sensing satellites (such as Sentinel-1), thermal infrared remote sensing satellites, and other platforms. Although optical remote sensing data has high spatial and spectral resolution, it cannot obtain surface information when there is cloud cover or severe atmospheric interference, resulting in missing water data in some areas. In addition, the limitations of sensor resolution make it difficult to accurately identify some small water bodies or water bodies in complex terrain, resulting in the inability to accurately record water data in these areas. The dynamic changes in water bodies (such as precipitation, evaporation, and seasonal changes) further exacerbate the spatiotemporal inconsistencies of the data, increasing the difficulty of processing remote sensing data.

[0004] In order to address these problems, filling in missing data has become a key task in remote sensing water body data processing. At present, the methods for filling in water body data are mainly divided into traditional interpolation algorithms and modern machine learning-based methods. Traditional interpolation methods such as linear interpolation and kriging interpolation, although simple to calculate and efficient, often have unsatisfactory filling accuracy when dealing with complex terrain and changing climatic conditions. In contrast, modern methods based on machine learning, especially deep learning technology, can significantly improve filling accuracy by learning the spatiotemporal characteristics of water bodies through a large amount of training data. However, existing deep learning methods mostly rely on a single data source, ignoring the important influence of terrain characteristics and river network structure on water body distribution, resulting in deficiencies in the spatial consistency and physical rationality of the filling results.

[0005] Therefore, combining multi-source knowledge data such as precipitation, digital elevation models (DEMs), and river network data with deep learning techniques for water data infill has become a key research direction in remote sensing data processing. Introducing multi-source data not only improves data infill accuracy but also enhances the spatiotemporal consistency and physical plausibility of water data, demonstrating significant advantages particularly in complex terrain and variable climate conditions. This approach can generate high-quality, complete water datasets, further enhancing their application value in fields such as water resources management and environmental monitoring. Summary of the Invention

[0006] In order to overcome the problems of insufficient filling accuracy, poor spatial consistency, and ineffective response to data missing problems under complex terrain and changeable climate conditions in existing water body data filling methods, the present invention provides a remote sensing water body dataset filling method based on multi-source knowledge data and deep learning models.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A remote sensing water body dataset filling method based on multi-source knowledge and deep learning model includes the following steps:

[0009] Step 1: Obtain remote sensing images for water body identification in each target month of each target historical year in the target area;

[0010] Step 2: Obtain precipitation data for each target month in each target historical year in the target area, and calculate the seasonal precipitation characteristics of the target area;

[0011] Step 3: Based on the seasonal precipitation characteristics of the target area, the thresholds of water bodies and non-water bodies in the target area are dynamically adjusted using a dynamic threshold method. By counting the proportion of water body / non-water body pixels in each remote sensing image obtained in step 1, the filling type of information-missing pixels is determined. Preliminary water body filling is performed on each remote sensing image obtained in step 1 to obtain a remote sensing image set with preliminary water body filling.

[0012] Step 4: Preprocess the knowledge product data within the target historical year period of the target area. The knowledge product data includes the river network product data and digital elevation model data of the target area. The preprocessed river network product data and digital elevation model data are combined with the remote sensing image set of the preliminary water body filling to form a training data set.

[0013] Step 5: Construct a hybrid model structure that connects a convolutional neural network (CNN) model structure and a long short-term memory (LSTM) model structure in series: data is input from the input layer of the CNN model structure, the CNN model structure and the LSTM model structure are connected in series, the output of the LSTM model structure is connected to a fully connected layer, the fully connected layer is connected to the output layer, and the data processing result is output by the output layer;

[0014] Step 6: Based on the training data set, the remote sensing images of preliminary water body filling in each target month in each target historical year, the preprocessed river network product data and the digital elevation model data are used as input, and the water body distribution position images in each target month in each target historical year are used as output to train the hybrid model structure to obtain a trained model, i.e., a water body filling hybrid model;

[0015] Step 7. Use the water body filling hybrid model to predict the water body distribution position of each remote sensing image in the remote sensing image set with preliminary water body filling. Through random missing evaluation calculation error and iterative repair, fine filling is achieved to obtain remote sensing images with no water body information missing in each target month in the target historical year of the target area.

[0016] Furthermore, the step 1 is specifically as follows:

[0017] The remote sensing images obtained through the public remote sensing data platform include water body pixels, non-water body pixels and information-missing pixels; among them, water body pixels are marked as water bodies, and non-water body pixels such as houses and vegetation are marked as non-water bodies; due to factors such as cloud obstruction and atmospheric interference, some pixel data may be missing, and these pixels are marked as information-missing pixels.

[0018] Furthermore, the step 2 is specifically as follows:

[0019] Based on the precipitation data for each target month in the target historical year of the target area, the precipitation anomaly percentage is calculated to represent the seasonal precipitation characteristics of the target area. The calculation formula is as follows:

[0020]

[0021] Among them, i represents each target historical year, j represents each target month, D i,j is the precipitation anomaly percentage for each target month in each target historical year, P i,j is the actual precipitation in each target month in each target historical year, P avg,j It is the long-term average precipitation corresponding to each target month in the target historical year period.

[0022] Furthermore, the step three is specifically as follows:

[0023] Based on the seasonal precipitation characteristics of the target area, a dynamic threshold method is used to preliminarily fill in the missing pixels in each remote sensing image obtained in step 1. The dynamic threshold method dynamically updates the thresholds of water bodies and non-water bodies based on the climate characteristics and topographic conditions of the target area and the seasonal precipitation characteristic values of the target area. The dynamic threshold method includes two stages: dynamic threshold setting and dynamic threshold judgment:

[0024] S1. The process of setting the dynamic threshold is as follows:

[0025] ① First, based on the climate characteristics of the target area and the different seasons divided into the target area, preset preliminary benchmark water body thresholds and benchmark non-water body thresholds for different seasons;

[0026] ② On this basis, the benchmark water body threshold and non-water body threshold are dynamically adjusted according to the seasonal precipitation characteristic values of the target area to obtain the adjusted water body threshold and adjusted non-water body threshold for each target month in each target historical year. The adjustment method is:

[0027] Adjusted water body threshold j =Baseline water body threshold j -D i,j / 10,

[0028] Adjusted non-water body threshold j = Benchmark non-water body threshold j +D i,j / 10;

[0029] Among them, D i,j The range of is limited to [-1,1] to prevent over-adjustment;

[0030] S2. In the dynamic threshold judgment stage, all pixels in each remote sensing image are counted and the proportion of water and non-water bodies is calculated. The filling rules are as follows:

[0031] ① If the water body ratio exceeds the adjusted water body threshold, the information missing pixels in the corresponding remote sensing image will be filled with water bodies;

[0032] ② If the proportion of non-water bodies exceeds the adjusted non-water body threshold, the information missing pixels in the corresponding remote sensing image will be filled with non-water bodies;

[0033] ③ If the proportions of both do not reach their respective adjusted thresholds, the pixel will be kept as an information-missing pixel and will not be filled.

[0034] Furthermore, the step 4 is specifically as follows:

[0035] The river network product data is vector data, and the digital elevation model data is raster data. The river network product data and the digital elevation model data are preprocessed. The preprocessing process includes converting the river network product vector data into raster data, cropping, resampling and projection conversion of the converted river network product data and digital elevation model data, and unifying their spatial resolution and coordinate system with the remote sensing image set of preliminary water body filling to ensure that all data are accurately aligned in spatial position; the preprocessed river network product data, digital elevation model data and the remote sensing image set of preliminary water body filling are combined to form a training data set.

[0036] Furthermore, the hybrid model structure constructed in step 5 is specifically as follows:

[0037] The CNN model structure includes an input layer, a convolution layer, and a pooling layer. Data is input from the input layer, and multimodal features are extracted through convolution and pooling. The convolution layer uses 64 3×3 convolution kernels. The convolution operation is expressed as:

[0038] f conv (x) = RELU(Conv(x,W conv )+b conv )

[0039] Where x is the input data, W conv is the convolution kernel weight, b conv is the bias term, RELU is the activation function;

[0040] The MaxPooling layer is used to reduce the spatial dimension. The pooling kernel size used is 2×2 and the stride is 2. The output after pooling is expressed as:

[0041] f pool (x)=MaxPooling(f conv (x))

[0042] The spatial features output by the CNN model structure are flattened and input into the LSTM model structure for temporal feature extraction. The LSTM model structure includes an LSTM layer with 128 cell units. Each cell unit is divided into a forget gate, an input gate, and an output gate. The expressions of the control coefficients of the forget gate, input gate, and output gate are respectively:

[0043] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0044] i t =σ(W i ·[ht-1 ,x t ]+b i )

[0045] O t =σ(W o ·[h t-1 ,x t ]+b o )

[0046] Among them, f t is the control coefficient of the forget gate, W f is the weight matrix of the forget gate, h t-1 is the hidden layer state information at the previous moment, x t is the input value at the current moment, b f is the bias top of the forget gate, σ is the activation function Sigmoid, i t is the control coefficient of the input gate, W i is the weight matrix of the input gate, b i is the bias top of the input gate, W o is the weight matrix of the output gate, b o is the bias top of the output gate;

[0047] The forget gate and the input gate are combined to obtain the cell state at the current moment. The expression of the cell state at the current moment is as follows:

[0048] C t =f t *C t-1 +i t *tanh(W C ·[h t-1 ,x t ]+b C )

[0049] Among them, C t is the cell state at the current moment, C t-1 is the cell state at the previous moment, W c is the weight matrix of the cell state at the current moment, tanh() is the activation function, b C is the bias top of the cell state at the current moment;

[0050] Based on the current cell state and combined with the output gate, the current hidden layer state is obtained. The expression of the current hidden layer state is as follows:

[0051] h t =O t *tanh(C t )

[0052] The output of the LSTM model structure is connected to the fully connected layer, and the prediction results are output through the fully connected layer combination. The combination process is expressed as:

[0053] z=σ(W fc h+b fc )

[0054] W fc and b fc is the weight and bias process of the fully connected layer, h is the output of the LSTM layer, and σ is the activation function.

[0055] Furthermore, the hybrid model structure constructed in step 5 is trained to obtain a trained model. The training process is as follows:

[0056] Based on the training data set, combined with the minimization of the mean square error (MSE) loss function, the back propagation algorithm and the Adam optimizer are used to train the hybrid model structure. The model parameters are updated through multiple iterations until the error converges to the preset threshold. The training is completed and the trained model, i.e., the water body filling hybrid model, is obtained.

[0057] The loss function is expressed as follows:

[0058]

[0059] y i is the actual value, is the model prediction value, and n is the number of samples.

[0060] Furthermore, the water filling hybrid model is applied to finely fill the remote sensing image set with preliminary water filling. The specific process is as follows:

[0061] S1. Perform preliminary prediction: Apply the water filling hybrid model to the remote sensing image set of preliminary water filling to generate preliminary prediction results;

[0062] S2. Perform random missingness assessment: Randomly select some non-missing pixels from the preliminary prediction results, set them as missing values, simulate random missing pixels, and reapply the water body filling hybrid model to fill them; compare the filling results with the corresponding preliminary water body filling remote sensing image, calculate the error, and evaluate the filling effect of the model;

[0063] S3. Iterative repair: Based on the error between the preliminary prediction results and the remote sensing image with preliminary water filling, adjust the model input and retrain the model to perform iterative repair. The formula is:

[0064]

[0065] where y new is the updated prediction result, y initialis the initial predicted value, y observed is the observation value of the remote sensing image for preliminary water filling, α is the learning rate, is the predicted value missing at random;

[0066] S4. Verification of spatiotemporal consistency: After each iterative repair, the spatiotemporal consistency of the filling results is evaluated to ensure the continuity and rationality of the filled data in time and space dimensions, and ultimately obtain a set of remote sensing images with no missing water body information in each target month within the target historical year of the target area.

[0067] Furthermore, the remote sensing images without missing water body information in each target month in the target historical year of the target area were compared with the Sentinel satellite images of water body information in the target area and the hydrological measured data of the same period, and the mean square error (MSE) and correlation coefficient (R 2 ) to assess the accuracy of water body filling and the overall water body data availability to verify the accuracy and reliability of the filling data.

[0068] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0069] The remote sensing water body dataset filling method provided by the present invention, based on multi-source knowledge and deep learning models, effectively overcomes the problems of insufficient filling accuracy, poor spatial consistency and poor filling effect under complex terrain in existing methods; by introducing seasonal precipitation characteristics, river network product data and DEM data, the accuracy and applicability of the filling model are enhanced; the high-precision remote sensing image set with no missing water body information finally generated has broad application prospects in fields such as water resources management, environmental monitoring and remote sensing data analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is a flow chart of the method of the present invention;

[0071] Figure 2 A diagram of a hybrid model architecture in which a CNN model structure and an LSTM model structure are connected in series.

[0072] Figure 3 This is a schematic diagram of the water filling process in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0073] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0074] like Figure 1 As shown, the present invention provides a remote sensing water body dataset filling method based on multi-source knowledge and deep learning model, which includes the following steps:

[0075] Step 1: Obtain remote sensing images for water body identification in each target month of each target historical year in the target area;

[0076] Step 2: Obtain precipitation data for each target month in each target historical year in the target area, and calculate the seasonal precipitation characteristics of the target area;

[0077] Step 3: Based on the seasonal precipitation characteristics of the target area, the thresholds of water bodies and non-water bodies in the target area are dynamically adjusted using a dynamic threshold method. By counting the proportion of water body / non-water body pixels in each remote sensing image obtained in step 1, the filling type of information-missing pixels is determined. Preliminary water body filling is performed on each remote sensing image obtained in step 1 to obtain a remote sensing image set with preliminary water body filling.

[0078] Step 4: Preprocess the knowledge product data within the target historical year period of the target area. The knowledge product data includes the river network product data and digital elevation model data of the target area. The preprocessed river network product data and digital elevation model data are combined with the remote sensing image set of the preliminary water body filling to form a training data set.

[0079] Step 5: Construct a hybrid model structure that connects a convolutional neural network (CNN) model structure and a long short-term memory (LSTM) model structure in series: data is input from the input layer of the CNN model structure, the CNN model structure and the LSTM model structure are connected in series, the output of the LSTM model structure is connected to a fully connected layer, the fully connected layer is connected to the output layer, and the data processing result is output by the output layer;

[0080] Step 6: Based on the training data set, the remote sensing images of preliminary water body filling in each target month in each target historical year, the preprocessed river network product data and the digital elevation model data are used as input, and the water body distribution position images in each target month in each target historical year are used as output to train the hybrid model structure to obtain a trained model, i.e., a water body filling hybrid model;

[0081] Step 7. Use the water body filling hybrid model to predict the water body distribution position of each remote sensing image in the remote sensing image set with preliminary water body filling. Through random missing evaluation calculation error and iterative repair, fine filling is achieved to obtain remote sensing images with no water body information missing in each target month in the target historical year of the target area.

[0082] Furthermore, the step 1 is specifically as follows:

[0083] The remote sensing images obtained through the public remote sensing data platform include water body pixels, non-water body pixels and information-missing pixels; among them, water body pixels are marked as water bodies, and non-water body pixels such as houses and vegetation are marked as non-water bodies; due to factors such as cloud obstruction and atmospheric interference, some pixel data may be missing, and these pixels are marked as information-missing pixels.

[0084] Furthermore, the step 2 is specifically as follows:

[0085] Based on the precipitation data for each target month in the target historical year of the target area, the precipitation anomaly percentage is calculated to represent the seasonal precipitation characteristics of the target area. The calculation formula is as follows:

[0086]

[0087] Among them, i represents each target historical year, j represents each target month, D i,j is the precipitation anomaly percentage for each target month in each target historical year, P i,j is the actual precipitation in each target month in each target historical year, P avg,j It is the long-term average precipitation corresponding to each target month in the target historical year period.

[0088] Furthermore, the step three is specifically as follows:

[0089] Based on the seasonal precipitation characteristics of the target area, a dynamic threshold method is used to preliminarily fill in the missing pixels in each remote sensing image obtained in step 1. The dynamic threshold method dynamically updates the thresholds of water bodies and non-water bodies based on the climate characteristics and topographic conditions of the target area and the seasonal precipitation characteristic values of the target area. The dynamic threshold method includes two stages: dynamic threshold setting and dynamic threshold judgment:

[0090] S1. The process of setting the dynamic threshold is as follows:

[0091] ①First, based on the climate characteristics of the target area and the different seasons divided into the target area, preset preliminary benchmark water body thresholds and benchmark non-water body thresholds for different seasons;

[0092] ② On this basis, the benchmark water body threshold and non-water body threshold are dynamically adjusted according to the seasonal precipitation characteristic values of the target area to obtain the adjusted water body threshold and adjusted non-water body threshold for each target month in each target historical year. The adjustment method is:

[0093] Adjusted water body threshold j =Baseline water body threshold j -D i,j / 10,

[0094] Adjusted non-water body threshold j = Benchmark non-water body threshold j +D i,j / 10;

[0095] Among them, D i,j The range of is limited to [-1,1] to prevent over-adjustment;

[0096] S2. In the dynamic threshold judgment stage, all pixels in each remote sensing image are counted and the proportion of water and non-water bodies is calculated. The filling rules are as follows:

[0097] ① If the water body ratio exceeds the adjusted water body threshold, the information missing pixels in the corresponding remote sensing image will be filled with water bodies;

[0098] ② If the proportion of non-water bodies exceeds the adjusted non-water body threshold, the information missing pixels in the corresponding remote sensing image will be filled with non-water bodies;

[0099] ③ If the proportions of both do not reach their respective adjusted thresholds, the pixel will be kept as an information-missing pixel and will not be filled.

[0100] Furthermore, the step 4 is specifically as follows:

[0101] The river network product data is vector data, and the digital elevation model data is raster data. The river network product data and the digital elevation model data are preprocessed. The preprocessing process includes converting the river network product vector data into raster data, cropping, resampling and projection conversion of the converted river network product data and digital elevation model data, and unifying their spatial resolution and coordinate system with the remote sensing image set of preliminary water body filling to ensure that all data are accurately aligned in spatial position; the preprocessed river network product data, digital elevation model data and the remote sensing image set of preliminary water body filling are combined to form a training data set.

[0102] Furthermore, the hybrid model structure constructed in step 5 is specifically as follows:

[0103] The CNN model structure includes an input layer, a convolution layer, and a pooling layer. Data is input from the input layer, and multimodal features are extracted through convolution and pooling. The convolution layer uses 64 3×3 convolution kernels. The convolution operation is expressed as:

[0104] f conv (x) = RELU(Conv(x,W conv )+b conv )

[0105] Where x is the input data, W conv is the convolution kernel weight, b conv is the bias term, RELU is the activation function;

[0106] The MaxPooling layer is used to reduce the spatial dimension. The pooling kernel size used is 2×2 and the stride is 2. The output after pooling is expressed as:

[0107] f pool (x)=MaxPooling(f conv (x))

[0108] The spatial features output by the CNN model structure are flattened and input into the LSTM model structure for temporal feature extraction. The LSTM model structure includes an LSTM layer with 128 cell units. Each cell unit is divided into a forget gate, an input gate, and an output gate. The expressions of the control coefficients of the forget gate, input gate, and output gate are respectively:

[0109] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0110] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0111] O t =σ(W o ·[h t-1 ,x t ]+b o )

[0112] Among them, f t is the control coefficient of the forget gate, W f is the weight matrix of the forget gate, h t-1 is the hidden layer state information at the previous moment, x t is the input value at the current moment, b f is the bias top of the forget gate, σ is the activation function Sigmoid, i t is the control coefficient of the input gate, W i is the weight matrix of the input gate, b i is the bias top of the input gate, W o is the weight matrix of the output gate, b o is the bias top of the output gate;

[0113] The forget gate and the input gate are combined to obtain the cell state at the current moment. The expression of the cell state at the current moment is as follows:

[0114] C t =ft *C t-1 +i t *tanh(W C ·[h t-1 ,x t ]+b C )

[0115] Among them, C t is the cell state at the current moment, C t-1 is the cell state at the previous moment, W c is the weight matrix of the cell state at the current moment, tanh() is the activation function, b C is the bias top of the cell state at the current moment;

[0116] Based on the current cell state and combined with the output gate, the current hidden layer state is obtained. The expression of the current hidden layer state is as follows:

[0117] h t =O t *tanh(C t )

[0118] The output of the LSTM model structure is connected to the fully connected layer, and the prediction results are output through the fully connected layer combination. The combination process is expressed as:

[0119] z=σ(W fc h+b fc )

[0120] W fc and b fc is the weight and bias process of the fully connected layer, h is the output of the LSTM layer, and σ is the activation function.

[0121] Furthermore, the hybrid model structure constructed in step 5 is trained to obtain a trained model. The training process is as follows:

[0122] Based on the training data set, combined with the minimization of the mean square error (MSE) loss function, the back propagation algorithm and the Adam optimizer are used to train the hybrid model structure. The model parameters are updated through multiple iterations until the error converges to the preset threshold. The training is completed and the trained model, i.e., the water body filling hybrid model, is obtained.

[0123] The loss function is expressed as follows:

[0124]

[0125] y i is the actual value, is the model prediction value, and n is the number of samples.

[0126] Furthermore, the water filling hybrid model is applied to finely fill the remote sensing image set with preliminary water filling. The specific process is as follows:

[0127] S1. Perform preliminary prediction: Apply the water filling hybrid model to the remote sensing image set of preliminary water filling to generate preliminary prediction results;

[0128] S2. Perform random missingness assessment: Randomly select some non-missing pixels from the preliminary prediction results, set them as missing values, simulate random missing pixels, and reapply the water body filling hybrid model to fill them; compare the filling results with the corresponding preliminary water body filling remote sensing image, calculate the error, and evaluate the filling effect of the model;

[0129] S3. Iterative repair: Based on the error between the preliminary prediction results and the remote sensing image with preliminary water filling, adjust the model input and retrain the model to perform iterative repair. The formula is:

[0130]

[0131] where y new is the updated prediction result, y initial is the initial predicted value, y observed is the observation value of the remote sensing image for preliminary water filling, α is the learning rate, is the predicted value missing at random;

[0132] S4. Verification of spatiotemporal consistency: After each iterative repair, the spatiotemporal consistency of the filling results is evaluated to ensure the continuity and rationality of the filled data in time and space dimensions, and ultimately obtain a set of remote sensing images with no missing water body information in each target month within the target historical year of the target area.

[0133] Furthermore, the remote sensing images without missing water body information in each target month in the target historical year of the target area were compared with the Sentinel satellite images of water body information in the target area and the hydrological measured data of the same period, and the mean square error (MSE) and correlation coefficient (R 2 ) to assess the accuracy of water body filling and the overall water body data availability to verify the accuracy and reliability of the filling data.

[0134] Example

[0135] The existing target area has remote sensing images for water body identification with a spatial resolution of 30m from 1990 to 2021, precipitation data for each month from 1990 to 2021, river network product data and digital elevation model (DEM) data for the target area in the corresponding years; according to the method of the present invention, the water body data filling process of the area is as follows: Figure 3 As shown, specifically:

[0136] Step 1: Obtain remote sensing images of water bodies in the target area for each month from 1990 to 2021 through a publicly available remote sensing data platform. These images are based on long-term satellite observations of the Earth's surface and are processed using a professional classification algorithm to provide the monthly distribution of water and non-water areas. Water pixels are marked as water, while non-water areas such as houses and vegetation are marked as non-water. Due to factors such as cloud cover and atmospheric interference, some pixel data may be missing and are marked as information-missing pixels.

[0137] Step 2: Based on the monthly precipitation data of the target area from 1990 to 2021, the precipitation anomaly percentage is calculated to capture the seasonal precipitation characteristics of the target area;

[0138] Step 3: Based on the seasonal precipitation characteristics of the target area, a dynamic threshold method is used to perform preliminary water filling on the information-missing pixels in each remote sensing image. This obtains a preliminary water-filled remote sensing image set, laying the foundation for further deep learning model processing.

[0139] Step 4. The knowledge product data includes river network product data and digital elevation model (DEM) data of the target area. The river network product data is vector data, and the digital elevation model data is raster data. The river network product data and the digital elevation model data are preprocessed. The preprocessing process includes converting the river network product vector data into raster data, clipping, resampling and projecting the converted river network product data and digital elevation model data, and unifying their spatial resolution and coordinate system with the remote sensing image set of the preliminary water body filling to ensure that all data are accurately aligned in spatial position; the preprocessed river network product data, digital elevation model data and the remote sensing image set of the preliminary water body filling are combined to form a training data set, which provides input data for the subsequent hybrid model structure of water body filling, ensuring that the hybrid model structure can fully utilize the spatial and geographic feature information of the target area;

[0140] Step 5. Construct a hybrid model structure in which a CNN model structure and an LSTM model structure are connected in series to capture and simulate the complex spatiotemporal characteristics of water body distribution; the CNN model structure includes an input layer, a convolution layer, and a maximum pooling layer. The data is input from the input layer of the CNN model structure and output to the LSTM model structure after convolution and pooling. The LSTM model structure includes an LSTM layer, which contains 128 cell units, each cell unit consists of a forget gate, an input gate, and an output gate. The output of the LSTM model structure is connected to a fully connected layer, and the fully connected layer is connected to the output layer; the hybrid model structure performs convolution processing on the input preliminary filled water body dataset, river network product data, and digital elevation model (DEM) data through the CNN model structure, extracts its spatial features in a multimodal manner, and reduces the spatial dimension of the data through the maximum pooling layer; then, the spatial features output by the CNN model structure are flattened and input into the LSTM model structure. The LSTM model structure captures the dynamic changes and long-term dependencies of the water body data through its time series processing capability; finally, the output is combined through the fully connected layer;

[0141] Step 6: Based on the training data set, the hybrid model structure is trained using the back propagation algorithm and the Adam optimizer in combination with the minimization of the mean square error (MSE) loss function. The model parameters are updated through multiple iterations until the error converges to the preset threshold. The training is completed and the trained model, i.e., the water filling hybrid model, is obtained.

[0142] Step 7. Use the water body filling hybrid model to fine-fill the remote sensing image set with preliminary water body filling: first, apply the water body filling hybrid model to the remote sensing image set with preliminary water body filling to generate preliminary prediction results; then, randomly select some non-missing pixels from the preliminary prediction results, reapply the water body filling hybrid model to fill, simulate random missing situations, and compare with the preliminary water body filled remote sensing images to evaluate errors; finally, through iterative repair and spatiotemporal consistency verification, ensure the accuracy and continuity of the filled water body data; finally, obtain remote sensing images with no missing water body information in each month of 1990-2021 in the target area.

[0143] Finally, by comparing the remote sensing images of the target area without missing water body information in each month from 1990 to 2021 with the Sentinel satellite images of water body information in the target area from 1990 to 2021 and the hydrological measured data of the same period, the error value was calculated and the accuracy of the filling result was evaluated; the evaluation results showed that the mean square error (MSE) and correlation coefficient (R 2 ) are all within the expected range, verifying the accuracy and reliability of the imputed data.

[0144] The remote sensing image set finally restored by this method can provide high-precision support in practical applications such as water resources management and environmental monitoring, demonstrating its effectiveness and application value in actual scenarios.

Claims

1. A remote sensing water dataset filling method based on multi-source knowledge and deep learning models, characterized by: The method comprises the following steps: Step 1: Obtain remote sensing images for water body identification in each target month of each target historical year in the target area; Step 2: Obtain precipitation data for each target month in each target historical year in the target area, and calculate the seasonal precipitation characteristics of the target area; Step 3: Based on the seasonal precipitation characteristics of the target area, the thresholds of water bodies and non-water bodies in the target area are dynamically adjusted using a dynamic threshold method. By counting the proportion of water body / non-water body pixels in each remote sensing image obtained in step 1, the filling type of information-missing pixels is determined. Preliminary water body filling is performed on each remote sensing image obtained in step 1 to obtain a remote sensing image set with preliminary water body filling. Step 4: Preprocess the knowledge product data within the target historical year period of the target area. The knowledge product data includes the river network product data and digital elevation model data of the target area. The preprocessed river network product data and digital elevation model data are combined with the remote sensing image set of the preliminary water body filling to form a training data set. Step 5: Construct a hybrid model structure that connects a convolutional neural network (CNN) model structure and a long short-term memory (LSTM) model structure in series: data is input from the input layer of the CNN model structure, the CNN model structure and the LSTM model structure are connected in series, the output of the LSTM model structure is connected to a fully connected layer, the fully connected layer is connected to the output layer, and the data processing result is output by the output layer; Step 6: Based on the training data set, the remote sensing images of preliminary water body filling in each target month in each target historical year, the preprocessed river network product data and the digital elevation model data are used as input, and the water body distribution position images in each target month in each target historical year are used as output to train the hybrid model structure to obtain a trained model, i.e., a water body filling hybrid model; Step 7. Use the water body filling hybrid model to predict the water body distribution position of each remote sensing image in the remote sensing image set with preliminary water body filling. Through random missing evaluation calculation error and iterative repair, fine filling is achieved to obtain remote sensing images with no water body information missing in each target month in the target historical year of the target area.

2. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 1 is characterized by: Each remote sensing image obtained in step 1 includes water body pixels, non-water body pixels and information missing pixels.

3. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 2 is characterized by: The step 2 is specifically as follows: Based on the precipitation data for each target month in the target historical year of the target area, the precipitation anomaly percentage is calculated to represent the seasonal precipitation characteristics of the target area. The calculation formula is as follows: Among them, i represents each target historical year, j represents each target month, D i,j is the precipitation anomaly percentage for each target month in each target historical year, P i,j is the actual precipitation in each target month in each target historical year, P avg,j It is the long-term average precipitation corresponding to each target month in the target historical year period.

4. The method for filling a remote sensing water body dataset based on multi-source knowledge and deep learning models according to claim 3 is characterized by: The step three is specifically as follows: Based on the seasonal precipitation characteristics of the target area, a dynamic threshold method is used to preliminarily fill in the missing pixels in each remote sensing image obtained in step 1. The dynamic threshold method dynamically updates the thresholds of water bodies and non-water bodies based on the climate characteristics and topographic conditions of the target area and the seasonal precipitation characteristic values of the target area. The dynamic threshold method includes two stages: dynamic threshold setting and dynamic threshold judgment: S1. The process of setting the dynamic threshold is as follows: ① First, based on the climate characteristics of the target area and the different seasons divided into the target area, preset preliminary benchmark water body thresholds and benchmark non-water body thresholds for different seasons; ② On this basis, the benchmark water body threshold and non-water body threshold are dynamically adjusted according to the seasonal precipitation characteristic values of the target area to obtain the adjusted water body threshold and adjusted non-water body threshold for each target month in each target historical year. The adjustment method is: Adjusted water body threshold j =Baseline water body threshold j -D i,j / 10, Adjusted non-water body threshold j = Benchmark non-water body threshold j +D i,j / 10; Among them, D i,j The range of is limited to [-1,1]; S2. In the dynamic threshold judgment stage, all pixels in each remote sensing image are counted and the proportion of water and non-water bodies is calculated. The filling rules are as follows: ① If the water body ratio exceeds the adjusted water body threshold, the information missing pixels in the corresponding remote sensing image will be filled with water bodies; ② If the proportion of non-water bodies exceeds the adjusted non-water body threshold, the information missing pixels in the corresponding remote sensing image will be filled with non-water bodies; ③ If the proportions of both do not reach their respective adjusted thresholds, the pixel will be kept as an information-missing pixel and will not be filled.

5. The method for filling a remote sensing water body dataset based on multi-source knowledge and deep learning model according to claim 4 is characterized in that: The step 4 is specifically as follows: The river network product data is vector data, and the digital elevation model data is raster data. The river network product data and the digital elevation model data are preprocessed. The preprocessing process includes converting the river network product vector data into raster data, clipping, resampling and projecting the converted river network product data and digital elevation model data, and unifying their spatial resolution and coordinate system with the remote sensing image set for preliminary water body filling to ensure accurate alignment of all data in spatial position. The preprocessed river network product data, digital elevation model data and remote sensing image set with preliminary water body filling are combined to form the training dataset.

6. The remote sensing water body dataset filling method based on multi-source knowledge and deep learning model according to claim 5 is characterized by: The hybrid model structure constructed in step 5 is specifically as follows: The CNN model structure includes an input layer, a convolution layer, and a pooling layer. Data is input from the input layer, and multimodal features are extracted through convolution and pooling. The convolution layer uses 64 3×3 convolution kernels. The convolution operation is expressed as: f conv (x)=RELU(Conv(x,W conv )+b conv ) Where x is the input data, W conv is the convolution kernel weight, b conv is the bias term, RELU is the activation function; The maximum pooling layer is used to reduce the spatial dimension. The pooling kernel size used is 2×2 and the stride is 2. The output after pooling is expressed as: f pool (x)=MaxPooling(f conv (x)) The spatial features output by the CNN model structure are flattened and input into the LSTM model structure for temporal feature extraction. The LSTM model structure includes an LSTM layer with 128 cell units. Each cell unit is divided into a forget gate, an input gate, and an output gate. The expressions of the control coefficients of the forget gate, input gate, and output gate are respectively: f t =σ(W f ·[h t-1 ,x t ]+b f ) i t =σ(W i ·[h t-1 ,x t ]+b i ) the t =σ(W o ·[h t-1 ,x t ]+b o ) Among them, f t is the control coefficient of the forget gate, W f is the weight matrix of the forget gate, h t-1 is the implicit layer state information of the previous moment, x t is the input value at the current moment, b f is the bias top of the forget gate, σ is the activation function Sigmoid, i t is the control coefficient of the input gate, W i is the weight matrix of the input gate, b i is the bias top of the input gate, W o is the weight matrix of the output gate, b o is the bias top of the output gate; The forget gate and the input gate are combined to obtain the cell state at the current moment. The expression of the cell state at the current moment is as follows: C t =f t *C t-1 +i t *tanh(W C ·[h t-1 ,x t ]+b C ) Among them, C t is the cell state at the current moment, C t-1 is the cell state at the previous moment, W c is the weight matrix of the cell state at the current moment, tanh() is the activation function, b C is the bias top of the cell state at the current moment; Based on the current cell state and combined with the output gate, the current hidden layer state is obtained. The expression of the current hidden layer state is as follows: h t =O t *tanh(C t ) The output of the LSTM model structure is connected to the fully connected layer, and the prediction results are output through the fully connected layer combination. The combination process is expressed as: z=σ(W fc ·h+b fc ) W fc and b fc is the weight and bias process of the fully connected layer, h is the output of the LSTM layer, and σ is the activation function.

7. The method for filling a remote sensing water body dataset based on multi-source knowledge and deep learning models according to claim 6 is characterized by: The hybrid model structure constructed in step 5 is trained to obtain a trained model. The training process is as follows: Based on the training data set, combined with the minimization of the mean square error (MSE) loss function, the back propagation algorithm and the Adam optimizer are used to train the hybrid model structure. The model parameters are updated through multiple iterations until the error converges to the preset threshold. The training is completed and the trained model, i.e., the water filling hybrid model, is obtained. The expression of the loss function is as follows: y i is the actual value, is the model prediction value, and n is the number of samples.

8. The method for filling a remote sensing water body dataset based on multi-source knowledge and deep learning models according to claim 7 is characterized in that: Apply the water filling hybrid model to finely fill the remote sensing image set that has been initially filled with water bodies. The specific process is as follows: S1. Perform preliminary prediction: Apply the water filling hybrid model to the remote sensing image set of preliminary water filling to generate preliminary prediction results; S2. Perform random missing value assessment: randomly select some non-missing pixels from the preliminary prediction results, set them as missing values, and re-apply the water body filling hybrid model to fill them; compare the filling results with the corresponding preliminary water body filling remote sensing image, calculate the error, and evaluate the filling effect of the model; S3. Iterative repair: Based on the error between the preliminary prediction results and the remote sensing image with preliminary water filling, adjust the model input and retrain the model, and perform multiple iterative repairs. The formula is: where y new is the updated prediction result, y initial is the initial predicted value, y observed is the observation value of the remote sensing image for preliminary water filling, α is the learning rate, is the predicted value missing at random; S4. Verification of spatiotemporal consistency: After each iterative repair, evaluate the spatiotemporal consistency of the filling results. Ensure the continuity and rationality of the filled data in time and space dimensions, and ultimately obtain a set of remote sensing images with no missing water body information in each target month within the target historical year of the target area.

9. The method for filling a remote sensing water body dataset based on multi-source knowledge and deep learning models according to claim 8, characterized in that: The remote sensing images without missing water body information in each target month in the target historical year of the target area are compared with the sentinel satellite images of water body information in the target area and the hydrological measured data of the same period. The mean square error (MSE) and correlation coefficient (R) are used to analyze the water body information. 2 The evaluation indicators include water body filling accuracy and overall water body data availability.

Images