Method for inverting water quality lacking remote sensing image by combining migration and space-time deep learning

Abstract

The invention relates to a method for inverting water quality without a remote sensing image by combining migration and space-time deep learning, which comprises the following steps: firstly, acquiring a satellite image and ground observation data of an area A, and automatically extracting a water area based on an SAM segmentation large model; associating satellite image data with ground observation data through spatio-temporal information, and establishing a water quality inversion model of the region A by using a deep neural network; acquiring a satellite image and ground observation data of the region B, segmenting a cloud occlusion region by using an SAM segmentation large model, and performing fine tuning by using a small sample of the region B through transfer learning to obtain an inversion model with satellite data of the region B and a result; and finally, combining the inversion result with a time sequence interpolation, and establishing space-time model correction by using a space-time deep learning network GTWNN, thereby estimating the inversion result which does not have satellite data but has ground observation station information pixel by pixel. The method has automation, high efficiency and universality under the conditions that the cloud amount is large and no unmanned aerial vehicle is used, and the water quality state can be continuously monitored at low cost.

Description

Technical Field

[0001] The present invention belongs to the field of environmental remote sensing and machine learning, and relates to a method and system for inverting water quality of missing remote sensing images, and specifically to a method and system for inverting water quality of missing remote sensing images through joint migration and spatiotemporal deep learning. Background Art

[0002] Environmental remote sensing plays a role in many fields, such as land and water resource monitoring, oil and mineral detection, etc. With the rapid development of artificial intelligence, machine learning has also penetrated into the geographical field, playing an important role in image recognition and image processing. In terms of water resources, as of 2022, the total amount of water resources in my country will reach 266 million cubic meters, which is huge. It has various natural water types such as rivers, lakes, etc. Under the needs of modern society, the number of artificial waters such as reservoirs and aquaculture ponds has also increased, and the demand for supervision is large. This patent provides a new idea for water quality inversion through innovative methods of environmental remote sensing and machine learning.

[0003] The existing water quality inversion scheme has the following problems:

[0004] 1. In the past, there have been precedents of using drones to obtain water images for water quality inversion, but there is still a risk of missing aerial photos. Drones are prone to getting lost due to unstable signals at high altitudes, weak endurance, and poor safety, resulting in missing aerial photos.

[0005] 2. In the process of water quality inversion, satellite images are often missing, which makes it more difficult to map water quality based on satellite data. The possible reasons for image missing are as follows:

[0006] (1) Due to internal limitations such as orbital period, number and field of view width, there is a lack of complete satellite data of the required time phase over the target waters;

[0007] (2) External interference factors such as cloud cover in the Earth’s atmosphere often result in a small number of satellite images and low quality;

[0008] (3) The strip-like bad tracks that appear in Landsat7 satellite images will cause the loss of some spectral data.

[0009] 3. In addition, the regional differences of water areas determine that each water quality inversion model is unique. The constructed water quality inversion model is only applicable to the empirical model of the sampled water area and may not be used in another water area due to its low accuracy. Summary of the invention

[0010] The purpose of the present invention is to solve the technical problem of efficient, accurate, stable and universal water quality inversion in the absence of remote sensing images, and to propose a method and system for inverting water quality in missing remote sensing images through joint migration and spatiotemporal deep learning.

[0011] The technical solution adopted by the method of the present invention is: a method for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning, comprising the following steps:

[0012] Step 1: Obtain satellite image data and ground observation data of area A, extract the water area in the satellite image and the spectral data of the ground observation data;

[0013] Step 2: Establish a water quality inversion model for region A by associating satellite image data with ground-based spectral data through temporal and spatial information;

[0014] Step 3: Obtain satellite image data and ground observation data of area B, perform cloud detection on the satellite image data, and segment the cloud-blocked area; and use transfer learning to fine-tune using a small number of samples from area B to obtain the inversion model and results of area B with satellite data;

[0015] Step 4: Based on the obtained inversion results and combined with time series interpolation, a spatiotemporal model correction is established to estimate the inversion results of satellite data pixel by pixel but with ground observation station information.

[0016] Preferably, in step 1, the water area in the satellite image is automatically extracted using the SAM segmentation model; the specific implementation includes the following steps:

[0017] Step 1.1: Obtain remote sensing images of area A and perform image preprocessing;

[0018] Step 1.2: Use SAM to segment the large model and input the image for feature extraction;

[0019] Step 1.3: Input the prompt into the SAM segmentation model, and then convert the image encoding and prompter encoding obtained by the SAM segmentation model into embedded representations respectively, and map them to the shared embedding space for alignment through contrastive learning to achieve the effect of splicing; then generate the final output through the decoding network for predicting the segmentation mask;

[0020] Step 1.4: Automatically extract the water area through the segmentation mask to obtain the segmented image of the water area in area A.

[0021] Preferably, the SAM segmentation large model is a trained large model; during the training process, the extraction results are evaluated using precision, recall rate and Kappa coefficient; according to the evaluation results, the SAM segmentation large model is adjusted until a result that meets the threshold is obtained;

[0022] The SAM segmentation large model introduces a domain adaptation strategy in the model architecture and training process, uses a method that only contains valid hints to generate the final prediction mask, decouples knowledge distillation to refine the lightweight image encoder, and introduces CLIP to assign labels to the SAM prediction mask, thereby improving the segmentation efficiency of the SAM segmentation large model in remote sensing images.

[0023] Preferably, in step 2, a water quality inversion model of area A is established using a progressive deep neural network; the specific implementation includes the following sub-steps:

[0024] Step 2.1: Use the correlation vector machine (RVM) to calculate the correlation coefficient between the band and the water quality parameter; take the m bands with the largest correlation coefficient as the final independent variable of one of the indicators of the water quality of the target area water environment; where m is a preset value;

[0025] Step 2.2: Use progressive deep neural network to construct a water quality inversion model for the water quality index WQI and the determined m bands;

[0026] (WQI) I =f(b r , b s …b z );

[0027] Among them, (WQI) I represents the water quality parameters of area A, I represents the image Image, b r , b s …b z Indicates the determined m bands;

[0028] Step 2.3: training the water quality inversion model to obtain a trained water quality inversion model;

[0029] During the training process, select the determination coefficient R 2 and root mean square error RMSE as evaluation criteria to select the optimal water quality inversion model, R 2 The value should be close to 1 and the RMSE should be as small as possible. During the training process, the water quality inversion model parameters are optimized to optimize the progressive deep neural network, so as to obtain the trained water quality inversion model of area A.

[0030] Preferably, the progressive deep neural network uses a proxy model to predict the performance of different network structures; and uses an early stopping strategy, specifically setting a tolerance, and stops training when the performance of the model on the validation set no longer improves; adopts an adaptive strategy such as adaptive learning rate adjustment, that is, using the Adam adaptive optimizer to automatically adjust the learning rate of each parameter according to the historical information of the gradient, dynamically reducing or increasing the learning rate; and introduces a regularization method, wherein the forward propagation calculation formula of the neural network with DropOut added is described as r (l) It obeys Bernoulli distribution, that is:

[0031] r (l) ~Bernoulli(p);

[0032] In the above formula, l represents the lth layer of the neural network, p is the probability of retaining the neuron, if r (l) =1, indicating that the neuron is retained; if r (l) =0 means that the neuron is discarded;

[0033] Then calculate the output vector:

[0034]

[0035] In the above formula, y (l) represents the input vector in the neural network, Represents the output vector in the neural network, and “*” represents the dot product; then recalculate the input vector of the l+1 layer:

[0036]

[0037] In the above formula, is the weighted output of the neurons in layer l, represents the weight, represents deviation; at the same time, It is also the input of the l+1th layer, and finally the output result of the l+1th layer is calculated as shown in the following formula:

[0038]

[0039] In the above formula, f represents the activation function, and As the input of the lower layer; use batch normalization to speed up network training and convergence; the idea is that the input batch of data is B = {x 1…m}; First calculate the mean μ of B B and variance

[0040]

[0041] For {x 1…m} is normalized, the formula is as follows:

[0042]

[0043] In the above formula, ∈ is random noise; for each output in B Normalize and restore the normalized features. The formula is as follows:

[0044]

[0045] Where γ={γ i},β={β i} is a trainable parameter:

[0046] Through the above technology, a more efficient progressive deep neural network is built to improve the accuracy of water body inversion.

[0047] Preferably, the specific implementation of step 3 includes the following sub-steps:

[0048] Step 3.1: Obtain the existing satellite data and water surface monitoring data of area B, use the SAM segmentation model to perform cloud detection, and segment the cloud-blocked area;

[0049] Step 3.2: For the images of the water area not blocked by clouds, based on the water quality inversion model of area A, the satellite data and water surface monitoring data obtained in step 3.1 are used as samples to train the water quality inversion model of area A;

[0050] During the training process, select the determination coefficient R 2 and root mean square error RMSE as evaluation criteria to fine-tune the water quality inversion model;

[0051] Step 3.3: Evaluate the effect of the fine-tuned water quality inversion model on the validation set, optimize the water quality inversion model, and obtain the inversion model and inversion results of area B under the condition of satellite data through transfer learning.

[0052] Preferably, the specific implementation of step 4 includes the following sub-steps:

[0053] Step 4.1: Use the Lagrange interpolation method in time series interpolation to estimate the water quality parameters (WQI) at the intermediate unknown time points p (t) and correct the estimated value using the field measured water quality data of the day;

[0054]

[0055] Where, ω(t)=(t-t1)(t-t2)…(tt n ), (WQI) p is the water quality prediction result obtained by interpolation, t1…t nis the time series, t i is the time when the water quality parameters are known;

[0056] Step 4.2: Use the spatiotemporal deep learning network GTWNN to perform spatiotemporal modeling and predict the water quality parameters (WQI) at point j MIj ,

[0057]

[0058] Where, the output value a of the i-th neuron in the spatiotemporal deep learning network GTWNN pattern layer is i , where the spatiotemporal weight w ji is the quantity to be determined, c is the bias term; (WQI) pi is the output water quality parameter (WQI) of the i-th neuron p (t);

[0059] Space-time weight (ds jk ) 2 =9u k -u j ) 2 +(v k -v j ) 2 , (dt ik ) 2 =(t k -t j ) 2 ,(u k , v k ) represents a coordinate point, t k Indicates a certain moment, (u k , v k , t k ) is the space-time position at a certain moment and a certain spatial coordinate, h ST is the parameter of space-time bandwidth, λ is the scale factor that balances the influence of spatial distance and temporal distance;

[0060] The output value a of the i-th neuron in the spatiotemporal weighted neural network GTWNN pattern layer i =exp(-(||x i -x j ||·b) 2 ); x i is the input vector of grid cell i, x j is the input vector of grid cell j, ∥·∥ represents the Euclidean distance, and b is a hyperparameter;

[0061] The i-th neuron in the pattern layer of the spatiotemporal deep learning network GTWNN and the S in the summation layer w The relationship between neurons is y i is the water quality parameter after correction for grid unit i; the i-th neuron in the pattern layer of the spatiotemporal weighted neural network GTWNN and S in the summation layer S The relationship between neurons is The NN weight between the pattern layer neuron and the summing unit SS is 1;

[0062] According to a i 、w ji , and obtain the water quality parameter (WQI) corrected by the spatiotemporal deep learning network GTWNN MIj , that is, the water quality inversion results of the remote sensing image missing phase and cloud-blocked water area are obtained.

[0063] The technical solution adopted by the system of the present invention is: a system for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning, including the following modules:

[0064] Module 1 is used to obtain satellite image data and ground observation data of area A, extract the water area in the satellite image, and the spectral data of the ground observation data;

[0065] Module 2 is used to establish a water quality inversion model for region A by associating satellite image data with ground observation spectral data through temporal and spatial information;

[0066] Module 3 is used to obtain satellite image data and ground observation data of area B, perform cloud detection on satellite image data, and segment the cloud-blocked area; and through transfer learning, fine-tune using a small number of samples of area B to obtain the inversion model and results of area B with satellite data;

[0067] Module 4 is used to combine the obtained inversion results with time series interpolation and establish a spatiotemporal model correction, so as to estimate the inversion results without satellite data but with ground observation station information pixel by pixel.

[0068] Compared with the prior art, the beneficial effects of the present invention include:

[0069] (1) The present invention uses the SAM segmentation model to automatically extract water images and segment the cloud-blocked areas, reducing the workload of manual mapping.

[0070] (2) The present invention is universal and can obtain the inversion model of a new area by transferring the learning sample adjustment model. The one-to-many form expands the scope of the research area and is more efficient, reducing the preliminary work.

[0071] (3) The present invention adopts deep neural network technology to establish complex nonlinear inversion models.

[0072] (4) The present invention uses fine-tuning and spatiotemporal interpolation to perform model correction, which can improve the accuracy of water quality inversion.

[0073] (5) The present invention utilizes spatiotemporal deep learning GTWNN, and can still deduce water quality parameters based on existing satellite image data when remote sensing satellite images are missing or blocked by clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The technical solution of the present invention is further described below using embodiments and specific implementation methods. In addition, in the process of describing the technical solution, some drawings are also used. For those skilled in the art, other drawings and the intention of the present invention can also be obtained based on these drawings without paying creative work.

[0075] Figure 1 is a flow chart of an embodiment of the present invention.

[0076] Figure 2 It is a schematic diagram of the SAM segmentation model in the example of the present invention.

[0077] Figure 3 Schematic diagram of a progressive deep neural network in an embodiment of the present invention.

[0078] Figure 4 Schematic diagram of spatiotemporal modeling using GTWNN in an embodiment of the present invention.

[0079] Figure 5 This is an example diagram of the water quality inversion result of the total nitrogen value of the Danjiangkou Reservoir based on remote sensing images in an embodiment of the present invention;

[0080] Figure 6 This is an example diagram of the total nitrogen water quality inversion results of the Danjiangkou Reservoir from September 3, 2020 to September 17, 2020 in an embodiment of the present invention. DETAILED DESCRIPTION

[0081] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0082] Please see Figure 1 , this embodiment provides a method for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning, including the following steps:

[0083] Step 1: Obtain satellite image data and ground observation data of area A, extract the water area in the satellite image and the spectral data of the ground observation data;

[0084] In one implementation, in step 1, the water area in the satellite image is automatically extracted using the SAM segmentation model; the specific implementation includes the following steps:

[0085] Step 1.1: Obtain remote sensing images of area A and perform image preprocessing;

[0086] Step 1.2: Use SAM to segment the large model and input the image for feature extraction;

[0087] Step 1.3: Input the prompt into the SAM segmentation model, and then convert the image encoding and prompter encoding obtained by the SAM segmentation model into embedded representations respectively, and map them to the shared embedding space for alignment through contrastive learning to achieve the effect of splicing. During the training process, the contrastive loss function is used to maximize the similarity between related samples and minimize the similarity between unrelated samples. Then the final output is generated through the decoding network for predicting the segmentation mask;

[0088] Step 1.4: Automatically extract the water area through the segmentation mask to obtain the segmented image of the water area in area A.

[0089] Please see Figure 2 In one embodiment, the SAM segmentation model is an improved version to improve segmentation efficiency and performance. The model architecture and training process are optimized, a domain adaptation strategy is introduced, a method that only contains valid hints is used to generate the final mask, knowledge distillation is decoupled to refine the lightweight image encoder, and CLIP is introduced to assign labels to the predicted mask of SAM, thereby improving the segmentation efficiency of SAM in remote sensing images.

[0090] In one embodiment, the SAM segmentation large model is a trained large model; during the training process, the extraction results are evaluated using precision, recall rate and Kappa coefficient; based on the evaluation results, the SAM segmentation large model is adjusted until a result that meets the threshold is obtained.

[0091] Step 2: Establish a water quality inversion model for region A by associating satellite image data with ground-based spectral data through temporal and spatial information;

[0092] In one embodiment, a water quality inversion model of region A is established using a progressive deep neural network; the specific implementation includes the following sub-steps:

[0093] Step 2.1: Use the relevance vector machine (RVM) to calculate the correlation coefficient between the band and the water quality parameter; take the m bands with the largest correlation coefficient as the final independent variable of one of the indicators of the water quality of the target area water environment; where m is a preset value;

[0094] In one embodiment, RVM is a trained model. After training, the correlation weight of each variable can be obtained to represent its explanatory power for the target, and the key bands most relevant to the water quality index are selected according to the weight. The m bands with larger correlation coefficients are taken as the final independent variables of one of the water quality indicators of the target area water environment. The water quality index (WQI) is shown in Table 1 below.

[0095] Table 1 Water quality indicators

[0096]

[0097] Step 2.2: Use progressive deep neural network to construct a water quality inversion model for the water quality index WQI and the determined m bands;

[0098] (WQI) I =f(b r , b s …b z );

[0099] Among them, (WQI) I represents the water quality parameters of area A, I represents the image Image, b r , b s …b z Indicates the determined m bands;

[0100] In one embodiment, machine learning can be used to construct a water quality inversion model for area A, such as decision tree regression (DTR), random forest regression (RFR), support vector machine regression (SVR), Bayesian regression (BR), Lasso regression (LR), gradient boosting regression tree (GBRT), etc.

[0101] Please see Figure 3In one embodiment, the progressive deep neural network is a trained network; during the training process, the progressive deep neural network uses a proxy model to predict the performance of different network structures; and uses an early stopping strategy (Early Stopping), specifically setting a tolerance, when the performance of the model on the validation set no longer improves, stops training; adopts an adaptive strategy such as adaptive learning rate adjustment, that is, using the Adam adaptive optimizer, automatically adjusts the learning rate of each parameter according to the historical information of the gradient, and dynamically reduces or increases the learning rate; and introduces a regularization method, wherein the forward propagation calculation formula of the neural network with DropOut added is described as r (l) It obeys Bernoulli distribution, that is:

[0102] r (l) ~Bernoulli(p);

[0103] In the above formula, l represents the lth layer of the neural network, p is the probability of retaining the neuron, if r (l) =1, indicating that the neuron is retained; if r (l) =0 means that the neuron is discarded;

[0104] Then calculate the output vector:

[0105]

[0106] In the above formula, y (l) represents the input vector in the neural network, Represents the output vector in the neural network, and “*” represents the dot product; then recalculate the input vector of the l+1 layer:

[0107]

[0108] In the above formula, is the weighted output of the neurons in layer l, represents the weight, represents deviation; at the same time, It is also the input of the l+1th layer, and finally the output result of the l+1th layer is calculated as shown in the following formula:

[0109]

[0110] In the above formula, f represents the activation function, and As the input of the lower layer; use batch normalization to speed up network training and convergence; the idea is that the input batch of data is B = {x 1…m}; First calculate the mean μ of B B and variance

[0111]

[0112] For {x 1…m} is normalized, the formula is as follows:

[0113]

[0114] In the above formula, ∈ is random noise; for each output in B Normalize and restore the normalized features. The formula is as follows:

[0115]

[0116] Where γ={γ i},β={β i} is a trainable parameter:

[0117] Through the above technology, a more efficient progressive deep neural network is built to improve the accuracy of water body inversion.

[0118] Step 2.3: training the water quality inversion model to obtain a trained water quality inversion model;

[0119] During the training process, select the determination coefficient R 2 and root mean square error (RMSE) as evaluation criteria to select the optimal water quality inversion model, R 2 The value should be close to 1 and the RMSE should be as small as possible. During the training process, the water quality inversion model parameters are optimized to optimize the progressive deep neural network, so as to obtain the trained water quality inversion model of area A.

[0120] Step 3: Obtain satellite image data and ground observation data of area B, perform cloud detection on the satellite image data, and segment the cloud-blocked area; and use transfer learning to fine-tune using a small number of samples from area B to obtain the inversion model and results of area B with satellite data;

[0121] In one implementation, the specific implementation of step 3 includes the following sub-steps:

[0122] Step 3.1: Obtain the existing satellite data and water surface monitoring data of area B, use the SAM segmentation model to perform cloud detection, and segment the cloud-blocked area;

[0123] Step 3.2: For the images of the water area not blocked by clouds, based on the water quality inversion model of area A, the satellite data and water surface monitoring data obtained in step 3.1 are used as samples to train the water quality inversion model of area A;

[0124] During the training process, select the determination coefficient R 2and root mean square error RMSE as evaluation criteria to fine-tune the water quality inversion model;

[0125] Step 3.3: Evaluate the effect of the fine-tuned water quality inversion model on the validation set, optimize the water quality inversion model, and obtain the inversion model and inversion results of area B under the condition of satellite data through transfer learning.

[0126] Step 4: Based on the obtained inversion results and combined with time series interpolation, a spatiotemporal model correction is established to estimate the inversion results of satellite data pixel by pixel but with ground observation station information.

[0127] Please see Figure 3 In one implementation, the specific implementation of step 4 includes the following sub-steps:

[0128] Step 4.1: Use the Lagrange interpolation method in time series interpolation to estimate the water quality parameters (WQI) at the intermediate unknown time points p (t) and correct the estimated value using the field measured water quality data of the day;

[0129]

[0130] Where, ω(t)=(t-t1)(t-t2)…(tt n ), (WQI) p is the water quality prediction result obtained by interpolation, t1…t n is the time series, t i is the time when the water quality parameters are known;

[0131] Step 4.2: Use the spatiotemporal deep learning network GTWNN to perform spatiotemporal modeling and predict the water quality parameters (WQI) at point j MIj ,

[0132]

[0133] Where, the output value a of the i-th neuron in the spatiotemporal deep learning network GTWNN pattern layer is i , where the spatiotemporal weight w ji is the quantity to be determined, c is the bias term; (WQI) pi is the output water quality parameter (WQI) of the i-th neuron p (t);

[0134] Space-time weight (ds jk ) 2 =(u k -u j ) 2 +(v k -v j ) 2, (dt ik ) 2 =(t k -t j ) 2 ,(u k , v k ) represents a coordinate point, t k Indicates a certain moment, (u k , v k , t k ) is the space-time position at a certain moment and a certain spatial coordinate, h ST is the parameter of space-time bandwidth, λ is the scale factor that balances the influence of spatial distance and temporal distance;

[0135] The output value a of the i-th neuron in the spatiotemporal weighted neural network GTWNN pattern layer i =exp(-(||x i -x j ||·b) 2 ); x i is the input vector of grid cell i, x j is the input vector of grid cell j, ∥·∥ represents the Euclidean distance, and b is a hyperparameter;

[0136] The i-th neuron in the pattern layer of the spatiotemporal deep learning network GTWNN and the S in the summation layer w The relationship between neurons is y i is the water quality parameter after correction for grid unit i; the i-th neuron in the pattern layer of the spatiotemporal weighted neural network GTWNN and S in the summation layer S The relationship between neurons is The NN weight between the pattern layer neuron and the summing unit SS is 1;

[0137] According to a i 、w ji , and obtain the water quality parameter (WQI) corrected by the spatiotemporal deep learning network GTWNN MIj , that is, the water quality inversion results of the remote sensing image missing phase and cloud-blocked water area are obtained.

[0138] This embodiment also provides a system for inverting water quality in missing remote sensing images through joint migration and spatiotemporal deep learning, which is used to implement the method described; the system includes the following modules:

[0139] Module 1 is used to obtain satellite image data and ground observation data of area A, extract the water area in the satellite image, and the spectral data of the ground observation data;

[0140] Module 2 is used to establish a water quality inversion model for region A by associating satellite image data with ground observation spectral data through temporal and spatial information;

[0141] Module 3 is used to obtain satellite image data and ground observation data of area B, perform cloud detection on satellite image data, and segment the cloud-blocked area; and through transfer learning, fine-tune using a small number of samples of area B to obtain the inversion model and results of area B with satellite data;

[0142] Module 4 is used to combine the obtained inversion results with time series interpolation and establish a spatiotemporal model correction, so as to estimate the inversion results without satellite data but with ground observation station information pixel by pixel.

[0143] The present invention is further described below through specific experiments.

[0144] This experiment uses the Landsat 8 remote sensing images of the Danjiangkou Reservoir area on September 2, 2020 and September 18, 2020 as a verification. After using the SAM large model to segment the water area, combined with ground station data, taking total nitrogen (Total Nitrogen) as an example, a regional water quality inversion model was constructed to obtain the remote sensing water quality inversion results of the Danjiangkou Reservoir area (see the attached results). Figure 5 ); Lagrange interpolation was then used to obtain the total nitrogen water quality parameter inversion results of the Danjiangkou Reservoir from September 3, 2020 to September 17, 2020 when there was no satellite remote sensing image; the data was corrected using a spatiotemporal deep learning network to obtain the corrected total nitrogen value of the Danjiangkou waters in the time series (see the attached results for the results). Figure 6 ).

[0145] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0146] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A method for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning, characterized in that: The following steps are involved: Step 1: Obtain satellite image data and ground observation data of area A, extract the water area in the satellite image and the spectral data of the ground observation data; Step 2: Establish a water quality inversion model for region A by associating satellite image data with ground-based spectral data through temporal and spatial information; Step 3: Obtain satellite image data and ground observation data of area B, perform cloud detection on the satellite image data, and segment the cloud-blocked area; And through transfer learning, a small number of samples in region B are used for fine-tuning to obtain the inversion model and results of region B with satellite data; Step 4: Based on the obtained inversion results and combined with time series interpolation, a spatiotemporal model correction is established to estimate the inversion results of satellite data pixel by pixel but with ground observation station information.

2. The method for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning according to claim 1, characterized in that: In step 1, the SAM segmentation model is used to automatically extract the water area in the satellite image; the specific implementation includes the following steps: Step 1.1: Obtain remote sensing images of area A and perform image preprocessing; Step 1.2: Use SAM to segment the large model and input the image for feature extraction; Step 1.3: Input the prompt into the SAM segmentation model, and then convert the image encoding and prompter encoding obtained by the SAM segmentation model into embedded representations respectively, and map them to the shared embedding space for alignment through contrastive learning to achieve the effect of splicing; then generate the final output through the decoding network for predicting the segmentation mask; Step 1.4: Automatically extract the water area through the segmentation mask to obtain the segmented image of the water area in area A.

3. The method for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning according to claim 2, characterized in that: The SAM segmentation model is a trained large model; During the training process, the extraction results are evaluated using precision, recall rate, and Kappa coefficient; based on the evaluation results, the SAM segmentation model is adjusted until a result that meets the threshold is obtained; The SAM segmentation large model introduces a domain adaptation strategy in the model architecture and training process, uses a method that only contains valid hints to generate the final prediction mask, decouples knowledge distillation to refine the lightweight image encoder, and introduces CLIP to assign labels to the SAM prediction mask, thereby improving the segmentation efficiency of the SAM segmentation large model in remote sensing images.

4. The method for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning according to claim 1, characterized in that: In step 2, a water quality inversion model for area A is established using a progressive deep neural network; the specific implementation includes the following sub-steps: Step 2.1: Use the correlation vector machine (RVM) to calculate the correlation coefficient between the band and the water quality parameter; take the m bands with the largest correlation coefficient as the final independent variable of one of the indicators of the water quality of the target area water environment; where m is a preset value; Step 2.2: Use progressive deep neural network to construct a water quality inversion model for the water quality index WQI and the determined m bands; (WQI) I =f(b r ,b s …b z ); Among them, (WQI) I represents the water quality parameters of area A, I represents the image Image, b r , b s …b z Indicates the determined m bands; Step 2.3: training the water quality inversion model to obtain a trained water quality inversion model; During the training process, select the determination coefficient R 2 and root mean square error RMSE as evaluation criteria to select the optimal water quality inversion model, R 2 The value should be close to 1 and the RMSE should be as small as possible. During the training process, the water quality inversion model parameters are optimized to optimize the progressive deep neural network, so as to obtain the trained water quality inversion model of area A.

5. The method for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning according to claim 4, characterized in that: The progressive deep neural network uses a proxy model to predict the performance of different network structures; uses an early stopping strategy, specifically setting a tolerance, and stops training when the performance of the model on the validation set no longer improves; adopts an adaptive strategy such as adaptive learning rate adjustment, that is, using the Adam adaptive optimizer to automatically adjust the learning rate of each parameter according to the historical information of the gradient, dynamically reducing or increasing the learning rate; and introduces a regularization method, in which the forward propagation calculation formula of the neural network with DropOut added is described as r (l) It obeys Bernoulli distribution, that is: r (l) ~Bernoulli(p); In the above formula, l represents the lth layer of the neural network, p is the probability of retaining the neuron, if r (l) =1, indicating that the neuron is retained; if r (l) =0 means that the neuron is discarded; Then calculate the output vector: In the above formula, y (l) represents the input vector in the neural network, Represents the output vector in the neural network, "*" represents the dot product; recalculate the input vector of the l+1 layer: In the above formula, is the weighted output of the neurons in layer l, represents the weight, represents deviation; at the same time, It is also the input of the l+1th layer, and finally the output result of the l+1th layer is calculated as shown in the following formula: In the above formula, f represents the activation function, and As input to the lower layer; Use batch normalization to speed up network training and convergence; the idea is to input a batch of data B = {x 1…m }; First calculate the mean μ of B B and variance For {x 1…m } is normalized, the formula is as follows: In the above formula, ∈ is random noise; for each output in B Normalize and restore the normalized features. The formula is as follows: Where γ={γ i },β={β i } is a trainable parameter: Through the above technology, a more efficient progressive deep neural network is built to improve the accuracy of water body inversion.

6. The method for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning according to claim 1, characterized in that: The specific implementation of step 3 includes the following sub-steps: Step 3.1: Obtain the existing satellite data and water surface monitoring data of area B, use the SAM segmentation model to perform cloud detection, and segment the cloud-blocked area; Step 3.2: For the images of the water area not blocked by clouds, based on the water quality inversion model of area A, the satellite data and water surface monitoring data obtained in step 3.1 are used as samples to train the water quality inversion model of area A; During the training process, select the determination coefficient R 2 and root mean square error RMSE as evaluation criteria to fine-tune the water quality inversion model; Step 3.3: Evaluate the effect of the fine-tuned water quality inversion model on the validation set, optimize the water quality inversion model, and obtain the inversion model and inversion results of area B under the condition of satellite data through transfer learning.

7. The method for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning according to any one of claims 1 to 6, characterized in that: The specific implementation of step 4 includes the following sub-steps: Step 4.1: Use the Lagrange interpolation method in time series interpolation to estimate the water quality parameters (WQI) at the intermediate unknown time points p (t) and correct the estimated value using the field measured water quality data of the day; Where, ω(t)=(t-t1)(t-t2)…(tt n ), (WQI) p is the water quality prediction result obtained by interpolation, t1…t n is the time series, t i is the time when the water quality parameters are known; Step 4.2: Use the spatiotemporal deep learning network GTWNN to perform spatiotemporal modeling and predict the water quality parameters (WQI) at point j MIj , Where, the output value a of the i-th neuron in the spatiotemporal deep learning network GTWNN pattern layer is i , where the spatiotemporal weight w ji is the quantity to be determined, c is the bias term; (WQI) pi is the output water quality parameter (WQI) of the i-th neuron p (t); Space-time weight (ds jk ) 2 =(u k -u j ) 2 +(v k -V j ) 2 , (dt ik ) 2 =(t k -t j ) 2 ,(u k , v k ) represents a coordinate point, t k Indicates a certain moment, (u k , v k , t k ) is the space-time position at a certain moment and a certain spatial coordinate, h ST is the parameter of space-time bandwidth, λ is the scale factor that balances the influence of spatial distance and temporal distance; The output value a of the i-th neuron in the spatiotemporal weighted neural network GTWNN pattern layer i =exp(-(||x i -x j ||·b) 2 ); x i is the input vector of grid cell i, x j is the input vector of grid cell j, ||·|| represents the Euclidean distance, and b is a hyperparameter; The i-th neuron in the pattern layer of the spatiotemporal deep learning network GTWNN and the S in the summation layer w The relationship between neurons is y i is the water quality parameter after correction for grid unit i; the i-th neuron in the pattern layer of the spatiotemporal weighted neural network GTWNN and S in the summation layer S The relationship between neurons is The NN weight between the pattern layer neuron and the summing unit SS is 1; According to a i 、w ji , and obtain the water quality parameter (WQI) corrected by the spatiotemporal deep learning network GTWNN MIj , that is, the water quality inversion results of the remote sensing image missing phase and cloud-blocked water area are obtained.

8. A system for inverting missing remote sensing image water quality by joint migration and spatiotemporal deep learning, used to implement the method described in any one of claims 1 to 7; characterized in that: Includes the following modules: Module 1 is used to obtain satellite image data and ground observation data of area A, extract the water area in the satellite image, and the spectral data of the ground observation data; Module 2 is used to establish a water quality inversion model for region A by associating satellite image data with ground observation spectral data through temporal and spatial information; Module 3 is used to obtain satellite image data and ground observation data of area B, perform cloud detection on the satellite image data, and segment the cloud-blocked area; And through transfer learning, a small number of samples in region B are used for fine-tuning to obtain the inversion model and results of region B with satellite data; Module 4 is used to combine the obtained inversion results with time series interpolation and establish a spatiotemporal model correction, so as to estimate the inversion results without satellite data but with ground observation station information pixel by pixel.

Images