Small-sample flood forecasting method based on adversarial and pseudo-label learning
Through the methods of adversarial learning and pseudo-label learning, the feature domain differences between pre-training and downstream data sets are narrowed, and pseudo-labels are generated, which solves the accuracy of small sample flood forecasting and improves the flood forecasting effect in areas with fewer historical hydrological data. .
Patent Information
- Application Number
- CN202211082906.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-06
- Publication Date
- 2025-01-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In areas with fewer historical hydrological data, learning-based data-driven models require a large amount of data to support training, resulting in poor flood forecasting of small samples and difficult to achieve accurate flood prediction.
Adversarial learning and pseudo-label learning methods are adopted to train through pre-training feature encoder and prediction heads, using mean square error to generate pseudo-labels, narrow the feature domain differences between pre-trained data sets and downstream data sets, and realize semi-supervised flooding forecast.
Improved flood forecasting accuracy in areas with fewer historical hydrological data, alleviated the problem of fewer labels in downstream data sets, and provided a solution for small sample learning.
Smart Images
Figure CN115358343B_ABST
Abstract
Description
Technical Field
[0001] The invention discloses a small sample flood forecasting method based on adversarial and pseudo-label learning, relates to a flood forecasting method in a semi-supervised manner, and in particular relates to flood forecasting using adversarial learning and pseudo-label learning techniques, and belongs to the field of computer technology. Background Art
[0002] Flood is one of the major natural disasters that endangers human safety and economic development. Timely and accurate prediction of basin runoff is of great significance for flood prevention and disaster reduction, saving lives, and reducing losses. At present, there are two main ways to predict floods: physical models and learning-based data-driven models. Among them, physical models rely on complex hydrological data and a large amount of calculations to predict floods. However, its basis is hydrological knowledge, so its promotion and scalability are greatly restricted. As for the learning-based data-driven model, it is mainly based on the relationship between input data and observed objects, and does not contain any artificially set physical parameters. Compared with physical models, learning-based data-driven models can be better applied to various rivers. In recent years, with the continuous development of machine learning technology, learning-based models have become more and more popular and their performance has become better and better.
[0003] However, learning-based data-driven models usually require a lot of data to support model training. Globally, there are many developing countries with little historical hydrological data on rivers, especially small-scale rivers. Therefore, how to use limited hydrological data for flood prediction is of great research significance. So far, most methods have improved performance by optimizing the network structure. However, due to the limitation of data volume, the model effect still needs to be improved. Therefore, realizing small sample flood forecasting through neural network models is a significant but extremely challenging task.
[0004] In order to solve this problem, the present invention proposes a small sample flood forecasting method based on adversarial and pseudo-label learning. First, a large number of labeled data sets are used to pre-train a feature encoder, and then the adversarial learning method is used to learn the feature encoder of the target area under semi-supervised conditions. This improves the accuracy of flood forecasting in areas with less historical hydrological data. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide a small sample flood forecasting method based on adversarial and pseudo-label learning, thereby providing a solution to the semi-supervised flood forecasting problem.
[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0007] The small sample flood forecasting method based on adversarial and pseudo-label learning includes the following steps:
[0008] Step 1: Build a feature extractor E pre and a prediction head, and concatenate the two as a pre-trained model for learning from a large-scale labeled pre-training flood dataset D pre Learn prior knowledge;
[0009] Step 2: Use mean square error (MSE) to train the pre-trained model constructed in step 1;
[0010] Step 3: The feature extractor E trained in step 2 pre The parameters of the prediction head are frozen and a feature extractor E is constructed. down , used to extract the downstream dataset D down Features of labeled samples in
[0011] Step 4: Train the feature extractor E through adversarial learning down , so that E down The output features and E pre Output feature alignment to narrow the difference in feature domain between pre-training dataset and downstream dataset;
[0012] Step 5: Use step 4 to counter the learned E down It is spliced with the prediction head to generate pseudo labels for the unlabeled samples in the downstream dataset, and the newly generated labeled samples are merged with the original labeled samples to obtain a new downstream dataset D new ;
[0013] Step 6: Use the pre-training dataset and D obtained in step 5 new Train the feature extractor E again through adversarial learning down ;
[0014] Step 7: Retrain the E down Spliced with head to achieve flood prediction in the downstream target domain.
[0015] As a preferred embodiment of the present invention, the specific process of step 1 is as follows:
[0016] 1-1. Build a time-domain convolutional neural network as a feature extractor E pre , used to extract features from labeled pre-training data;
[0017] 1-2. Build a multi-layer perceptron as a nonlinear prediction head to map the features extracted by the feature extractor to the label space in the flood dataset;
[0018] 1-3. Feature extractor E preConcatenated with the prediction head as a pre-training model.
[0019] As a preferred embodiment of the present invention, the specific process of step 2 is as follows:
[0020] 2-1. Given a pre-training dataset of duration N Where X i is a vector of length d, representing the rainfall data of various places in the basin at the i-th time step, Y i represents the runoff data at the i-th time step, and each time step differs by 1 hour;
[0021] 2-2. Use the rainfall data of the past 6 hours and the current runoff data to predict the runoff in the next hour, i.e. feature extractor E pre The input is and Y t-1 , the output of the pre-trained model is expressed as follows:
[0022]
[0023] in concat(·) represents tensor concatenation operation;
[0024] 2-3. Use mean square error (MSE) to pre-train the model, and the loss function is as follows:
[0025]
[0026] where Y t and represent the true value and the predicted value respectively.
[0027] As a preferred solution of the present invention, the specific process of step 3 is:
[0028] 3-1. Freeze feature extractor E pre and the network weights of the prediction head;
[0029] 3-2. Construct a time-domain convolutional neural network as a feature extractor E down , used to extract features from labeled downstream data.
[0030] As a preferred solution of the present invention, the specific process of step 4 is:
[0031] 4-1. Given a small sample downstream dataset D with a duration of M down , which contains K labeled samples With MK unlabeled samples
[0032] 4-2. Use Epre and E down For the pre-training dataset D pre and downstream dataset D down Extract features from labeled samples in the dataset;
[0033] 4-3. Build the discriminator D and transform E pre The extracted features are taken as the true value, E down As a generator, E is trained by adversarial learning. down and D, the loss function is as follows:
[0034]
[0035]
[0036] in and As described in step 2-2, they represent downstream data and pre-training data, k∈[1,2,…,N], P is the gradient penalty term, ω GP is the weight of the gradient penalty term.
[0037] As a preferred solution of the present invention, the specific process of step 5 is:
[0038] 5-1. Use step 4 against the learned E down Concatenate with the prediction head frozen in step 3, and generate pseudo labels for the next MK unlabeled downstream data based on the labeled data in the first K time steps
[0039] 5-2. Combine pseudo labels with D down Merge to get a new labeled downstream dataset D new .
[0040] As a preferred solution of the present invention, the specific process of step 6 is:
[0041] 6-1. Given a dataset D with a duration of M new , which contains K labeled samples and MK pseudo-labeled samples
[0042] 6-2. Use E pre and E down For the pre-training dataset D pre and dataset D new Extract features from samples in the
[0043] 6-3. Use the discriminator D trained in step 4 to transform E pre The extracted features are taken as the true value, Edown As a generator, E is trained by adversarial learning. down and D, the loss function is as follows:
[0044]
[0045]
[0046] in and As described in step 2-2, they represent downstream data and pre-training data, k∈[1,2,…,N], P is the gradient penalty term, ω GP is the weight of the gradient penalty term.
[0047] As a preferred solution of the present invention, the specific process of step 7 is:
[0048] The feature extractor E trained in step 6 down It is concatenated with the prediction head and used for flood prediction in the downstream target domain. The inference formula is as follows:
[0049]
[0050] in concat(·) represents a tensor concatenation operation.
[0051] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:
[0052] 1. The present invention reduces the difference in feature domain between the pre-training dataset and the downstream dataset through adversarial learning, thus achieving the effect of domain adaptation.
[0053] 2. The present invention uses pseudo-label learning to alleviate the problem of fewer labels in the downstream dataset, realizes small sample learning in the target domain, and provides a solution for flood forecasting in basins with less historical hydrological data. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is an algorithm flow chart of the small sample flood forecasting method based on adversarial and pseudo-label learning of the present invention.
[0055] Figure 2 Schematic diagram of the small sample flood forecasting method based on adversarial and pseudo-label learning of the present invention. DETAILED DESCRIPTION
[0056] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, examples of which are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.
[0057] Currently, there are two main approaches to flood prediction: physical models and learning-based data-driven models. Physical models rely on complex hydrological parameters and intensive calculations to predict floods. However, since they are mainly based on hydrological knowledge, this limits their generalization and scalability. Learning-based data-driven models are mainly based on learning the relationship between input data and observed targets, and do not explicitly involve any human-defined physical parameters. However, learning-based data-driven models usually require a large amount of training data, and there are many rivers around the world with little historical hydrological data. Therefore, it is very important to use limited hydrological data to predict floods in these areas.
[0058] To address this problem, the present invention proposes a small sample flood forecasting method based on adversarial and pseudo-label learning. Through adversarial learning, the feature extractor of the downstream target domain distills knowledge from the pre-trained extractor. And through pseudo-label learning, the problem of fewer labeled samples in the downstream dataset is alleviated, providing a solution for small sample flood forecasting.
[0059] Combination Figure 1 As shown, the small sample flood forecasting method based on adversarial and pseudo-label learning of the present invention comprises the following steps:
[0060] 1. Build a feature extractor E pre and a prediction head, and concatenate the two as a pre-trained model for learning from a large-scale labeled pre-training flood dataset D pre The specific process is as follows:
[0061] (1) Construct a time-domain convolutional neural network as the feature extractor E pre , used to extract features from labeled pre-training data;
[0062] (2) Construct a multi-layer perceptron as a nonlinear prediction head to map the features extracted by the feature extractor to the label space in the flood dataset;
[0063] (3) The feature extractor E pre Concatenated with the prediction head as a pre-training model.
[0064] 2. Use the mean square error (MSE) to train the pre-trained model constructed in step 1. The specific process is as follows:
[0065] (1) Given a pre-training dataset of length N Where X i is a vector of length d, representing the rainfall data of various places in the basin at the i-th time step, Y irepresents the runoff data at the i-th time step, and each time step differs by 1 hour;
[0066] (2) Use the rainfall data of the past 6 hours and the current runoff data to predict the runoff in the next hour, i.e., feature extractor e pre The input is and Y t-1 , the output of the pre-trained model is expressed as follows:
[0067]
[0068] in concat(·) represents tensor concatenation operation;
[0069] (3) Use mean square error (MSE) to pre-train the model, and the loss function is as follows:
[0070]
[0071] where Y t and represent the true value and the predicted value respectively.
[0072] 3. The feature extractor E trained in step 2 pre The parameters of the prediction head are frozen and a feature extractor E is constructed. down , used to extract the downstream dataset D down There are features of labeled samples in , and the specific process is as follows:
[0073] (1) Freeze the feature extractor E pre and the network weights of the prediction head;
[0074] (2) Construct a time-domain convolutional neural network as a feature extractor E down , used to extract features from labeled downstream data.
[0075] 4. Train the feature extractor E through adversarial learning down , so that E down The output features and E pre The output features are aligned to narrow the difference in feature domain between the pre-training dataset and the downstream dataset. The specific process is as follows:
[0076] (1) Given a small sample downstream dataset D with a duration of M down , which contains K labeled samples With MK unlabeled samples
[0077] (2) Use E pre and E downFor the pre-training dataset D pre and downstream dataset D down Extract features from labeled samples in the dataset;
[0078] (3) Build the discriminator D and transform E pre The extracted features are taken as the true value, E down As a generator, E is trained by adversarial learning. down and D, the loss function is as follows:
[0079]
[0080]
[0081] in and As described in step 2-2, they represent downstream data and pre-training data, k∈[1,2,…,N], P is the gradient penalty term, ω GP is the weight of the gradient penalty term.
[0082] 5. Use step 4 against the learned E down It is spliced with the prediction head to generate pseudo labels for the unlabeled samples in the downstream dataset, and the newly generated labeled samples are merged with the original labeled samples to obtain a new downstream dataset D new , the specific process is as follows:
[0083] (1) Apply step 4 to the learned E down Concatenate with the prediction head frozen in step 3, and generate pseudo labels for the next MK unlabeled downstream data based on the labeled data in the first K time steps
[0084] (2) Combine the pseudo labels with D down Merge to get a new labeled downstream dataset D new .
[0085] 6. Use the pre-training dataset and D obtained in step 5 new Train the feature extractor E again through adversarial learning down , the specific process is as follows:
[0086] (1) Given a dataset D of duration M new , which contains K labeled samples and MK pseudo-labeled samples
[0087] (2) Use E pre and E down For the pre-training dataset D pre and dataset Dnew Extract features from samples in the
[0088] (3) Use the discriminator D trained in step 4 to transform E pre The extracted features are taken as the true value, E down As a generator, E is trained by adversarial learning. down and D, the loss function is as follows:
[0089]
[0090]
[0091] in and As described in step 2-2, they represent downstream data and pre-training data, k∈[1,2,…,N], P is the gradient penalty term, ω GP is the weight of the gradient penalty term.
[0092] 7. Use the feature extractor E trained in step 6 down It is concatenated with the prediction head and used for flood prediction in the downstream target domain. The inference formula is as follows:
[0093]
[0094] in concat(·) represents a tensor concatenation operation.
[0095] The above embodiments are only for illustrating the technical idea of the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.
Claims
1. A small sample flood forecasting method based on adversarial and pseudo-label learning, characterized in that: The steps include: Step 1: Build a feature extractor E pre and a prediction head, and concatenate the two as a pre-trained model for learning from a large-scale labeled pre-training flood dataset D pre Learn prior knowledge; Step 2: Use mean square error (MSE) to train the pre-trained model constructed in step 1. Step 3: The feature extractor E trained in step 2 pre The parameters of the prediction head are frozen and a feature extractor E is constructed. down , used to extract the downstream dataset D down Features of labeled samples in Step 4: Train the feature extractor E through adversarial learning down , so that E down The output features and E pre Output feature alignment to narrow the difference in feature domain between pre-training dataset and downstream dataset; Step 5: Use step 4 to counter the learned E down It is spliced with the prediction head to generate pseudo labels for the unlabeled samples in the downstream dataset, and the newly generated labeled samples are merged with the original labeled samples to obtain a new downstream dataset D new ; Step 6: Use the pre-training dataset and D obtained in step 5 new Train the feature extractor E again through adversarial learning down ; Step 7: Retrain the E down Connect with head to realize flood prediction in downstream target domain; The specific process of step 4 is as follows: 4-1. Given a small sample downstream dataset D with a duration of M down , which contains K labeled samples With MK unlabeled samples 4-2. Use E pre and E down For the pre-training dataset D pre and downstream dataset D down Extract features from labeled samples in the dataset; 4-3. Build the discriminator D and transform E pre The extracted features are taken as the true value, E down As a generator, E is trained by adversarial learning. down and D, the loss function is as follows: in and denote downstream data and pre-training data respectively, k∈[1,2,…,N], P is the gradient penalty term, ω GP is the weight of the gradient penalty term.
2. The small sample flood forecasting method based on adversarial and pseudo-label learning according to claim 1 is characterized in that: The specific process of step 1 is as follows: 1-1. Build a time-domain convolutional neural network as a feature extractor E pre , used to extract features from labeled pre-training data; 1-2. Build a multi-layer perceptron as a nonlinear prediction head to map the features extracted by the feature extractor to the label space in the flood dataset; 1-3. Feature extractor E pre Concatenated with the prediction head as a pre-training model.
3. The small sample flood forecasting method based on adversarial and pseudo-label learning according to claim 1 is characterized in that: The specific process of step 2 is: 2-1. Given a pre-training dataset of duration N Where X i is a vector of length d, representing the rainfall data of various places in the basin at the i-th time step, Y i represents the runoff data at the i-th time step, and each time step differs by 1 hour; 2-2. Use the rainfall data of the past 6 hours and the current runoff data to predict the runoff in the next hour, i.e. feature extractor E pre The input is and Y t-1 , the output of the pre-trained model is expressed as follows: in concat(·) represents tensor concatenation operation; 2-3. Use mean square error MSE to pre-train the model, and the loss function is as follows: where Y t and represent the true value and the predicted value respectively.
4. The small sample flood forecasting method based on adversarial and pseudo-label learning according to claim 1 is characterized in that: The specific process of step 3 is as follows: 3-1. Freeze feature extractor E pre and the network weights of the prediction head; 3-2. Construct a time-domain convolutional neural network as a feature extractor E down , used to extract features from labeled downstream data.
5. The small sample flood forecasting method based on adversarial and pseudo-label learning according to claim 1 is characterized in that: The specific process of step 5 is as follows: 5-1. Use step 4 against the learned E down Concatenate with the prediction head frozen in step 3, and generate pseudo labels for the next MK unlabeled downstream data based on the labeled data in the first K time steps 5-2. Combine pseudo labels with D down Merge to get a new labeled downstream dataset D new .
6. The small sample flood forecasting method based on adversarial and pseudo-label learning according to claim 1 is characterized in that: The specific process of step 6 is as follows: 6-1. Given a dataset D with a duration of M new , which contains K labeled samples and MK pseudo-labeled samples 6-2. Use E pre and E down For the pre-training dataset D pre and dataset D new Extract features from samples in the 6-3. Use the discriminator D trained in step 4 to transform E pre The extracted features are taken as the true value, E down As a generator, E is trained by adversarial learning. down and D, the loss function is as follows: in and denote downstream data and pre-training data respectively, k∈[1,2,…,N], P is the gradient penalty term, ω GP is the weight of the gradient penalty term.
7. The small sample flood forecasting method based on adversarial and pseudo-label learning according to claim 1 is characterized in that: The specific process of step 7 is as follows: The feature extractor E trained in step 6 down It is concatenated with the prediction head and used for flood prediction in the downstream target domain. The inference formula is as follows: in concat(·) represents a tensor concatenation operation.