An intelligent flow prediction method and system for information-deficient watersheds

By training the Transformer model in a source basin with sufficient data, combining multiple perceptrons and attention mechanisms, optimizing the traffic prediction results, the problem of inaccurate traffic prediction in the information-scarce basin is solved, and accurate prediction and disasters are achieved in areas with lack of observation data. Depletion.

CN118261285BActive Publication Date: 2024-11-15CHINA INST OF WATER RESOURCES & HYDROPOWER RES +1
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410280168.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-12
Publication Date
2024-11-15
Estimated Expiration
2044-03-12

AI Technical Summary

Technical Problem

In information-scarce watersheds, the lack of effective runoff observation means leads to inaccurate flow prediction, and traditional methods rely on a large amount of observation data to effectively predict in areas with scarce data.

Method used

Using the Transformer model as the basis, by collecting feature data and training in the source basin with sufficient data, the source basin model is constructed, and the prediction results are optimized based on the similarity between the source basin and the target basin, and multiple perceptrons, encoders and decoding are used. The modules such as the instrument improve the learning ability of hydrological feature timing information, generate dynamic and static vectors for projection, and combine self-attention layer and feedforward neural network for prediction.

Benefits of technology

Accurate flow forecasts for information-scarce watersheds are achieved, early warnings are taken to reduce disaster losses, and the impact of climate change on hydrological cycles is simulated, which improves the reliability of water resource management and climate change research.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118261285B_ABST
    Figure CN118261285B_ABST
Patent Text Reader

Abstract

The present invention discloses an intelligent flow prediction method and system for an information-deficient basin, belonging to the technical field of water resource management, comprising: taking a Transformer model as a basic model, collecting feature data in a source basin with sufficient data, training a basic model, and constructing a source basin model, wherein feature projection embedding is performed at a multi-perceptron MLP, and after encoder processing, the model improves the learning ability of the hydrological feature time series information in the attention module, and generates a prediction result of the source basin through a decoder; based on the source basin model, flow prediction is performed for a target basin with scarce information, and the prediction result is optimized according to the similarity between the source basin and the target basin, combined with the pre-training parameters of the source basin model; the present invention provides reliable data support for water resource management, and has extremely high academic and social value.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of water resource management, and in particular to an intelligent flow prediction method and system for information-deficient river basins. Background Art

[0002] With the continuous development of climate change and human activities, water resource management and natural disaster prevention have become increasingly important. In this context, runoff prediction, as a key technology at the intersection of hydrology and meteorology, is increasingly gaining widespread attention. Runoff prediction can help water resource planning and management, flood warning and prevention, disaster risk assessment, and climate change adaptation by simulating and predicting the generation, transmission, and distribution of surface runoff.

[0003] Flow forecasting is the foundation of hydrological science. It can predict future changes in flow within a river basin. Decision makers can better plan and manage water resources to ensure sustainable use of water resources and avoid excessive development or waste. In flood warning, the time of flood occurrence can be predicted in advance based on rainfall data and predicted flow, so that residents and decision makers have enough time to take necessary flood prevention measures and minimize disaster losses. Flow forecasting can also help climate change research by controlling rainfall patterns, snowmelt processes, etc., to understand the impact of climate change on the hydrological cycle.

[0004] Under realistic conditions, some river basins have insufficient data due to the scarcity of hydrological stations, geographical factors, and other reasons, and lack effective means of runoff observation. Therefore, we try to introduce artificial intelligence technology to predict the flow in areas with insufficient data through artificial intelligence algorithms, and effectively improve the prediction accuracy. The main auxiliary direction is: the runoff prediction problem in river basins with insufficient observation data. Summary of the invention

[0005] In order to solve the above problems, the purpose of the present invention is to provide an intelligent flow prediction technology suitable for information-deficient basins to make up for the shortcomings of traditional methods in these areas.

[0006] In order to achieve the above technical objectives, the present application provides an intelligent flow prediction method for information-deficient watersheds, comprising the following steps:

[0007] The Transformer model is used as the basic model. By collecting feature data in the source basin with sufficient data, the basic model is trained and the source basin model is constructed. Feature projection embedding is performed at the multi-perceptron MLP. After encoder processing, the model improves the learning ability of the hydrological feature time series information in the attention module, and generates the prediction results of the source basin through the decoder.

[0008] Based on the source basin model, flow prediction is performed for the target basin with scarce information, and the prediction results are optimized according to the similarity between the source basin and the target basin combined with the pre-trained parameters of the source basin model.

[0009] Preferably, in the process of constructing the source basin model, dynamic vectors and static vectors are generated according to the input hydrological characteristics and time series information and the changes in basin characteristics, wherein the dynamic vector refers to the characteristics that change over time, and the static vector refers to the basin characteristics that remain fixed for a long time.

[0010] Preferably, in the process of building the source domain model, the dynamic vector and the static vector are projected into a common latent space through a multi-layer perceptron MLP to create a combined hidden representation or embedding and sent to the encoder.

[0011] Preferably, in the process of constructing the source basin model, an encoder is constructed through a self-attention layer and a feed-forward neural network;

[0012] The decoder is constructed through the self-attention layer, the decoder-encoder attention layer and the feedforward neural network, where the decoder-encoder attention layer contains the information of the encoder and the information of the decoder.

[0013] Preferably, in the process of constructing the source basin model, a self-attention layer is generated according to the multi-head attention mechanism.

[0014] Preferably, in the process of generating the self-attention layer, the self-attention layer is generated by adding convolutional attention to the multi-head attention mechanism, wherein the convolutional attention represents the use of a convolutional layer with a kernel size of k to match relevant feature trends.

[0015] Preferably, in the process of obtaining the similarity between the source basin and the target basin, the similarity is calculated based on the first vector output by the self-attention layer of the source basin and the second vector output by the self-attention layer of the target basin.

[0016] Preferably, in the process of calculating the similarity, the similarity is calculated by cosine similarity.

[0017] Preferably, in the process of optimizing the prediction results, the predicted flow of the target basin is optimized through the calculation results of the similarity.

[0018] The present invention discloses an intelligent flow prediction system for information-deficient watersheds, comprising:

[0019] The model building module is used to use the Transformer model as the basic model, collect feature data in the source basin with sufficient data, train the basic model, and build the source basin model. Feature projection embedding is performed at the multi-perceptron MLP. After the encoder is processed, the model improves the learning ability of the hydrological feature time series information in the attention module, and generates the prediction result of the source basin through the decoder;

[0020] The flow prediction module is used to predict the flow of the target basin with scarce information based on the source basin model, and optimize the prediction results according to the similarity between the source basin and the target basin combined with the pre-trained parameters of the source basin model.

[0021] The present invention discloses the following technical effects:

[0022] The present invention trains models in areas with rich observation data, extracts the relationship between potential hydrological features and runoff, uses pre-trained models to perform flood prediction from data-rich areas to information-poor areas, and accurately predicts the runoff in the next 24 hours in areas with current sparse data.

[0023] In water resource management, the model prediction data of the present invention can be used to make advance estimates of runoff in areas lacking observation data, so that users can understand the flow changes in river sections. In flood prevention, advance predictions can be made to minimize disaster losses; in experiments, climate and rainfall conditions can be simulated to facilitate understanding of the impact of climate change on the hydrological cycle. Therefore, the academic and social value of the present invention is extremely high. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a schematic diagram of a model of the intelligent traffic prediction method described in the present invention;

[0026] Figure 2 is a schematic diagram of the structure of the model input layer of the present invention;

[0027] Figure 3 It is a schematic diagram of the structure of the model encoder and decoder of the present invention;

[0028] Figure 4 It is a schematic diagram of the structure of the multi-head attention mechanism described in the present invention;

[0029] Figure 5 This is a schematic diagram of the convolutional attention added to the multi-head attention mechanism described in the present invention. DETAILED DESCRIPTION

[0030] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0031] like Figure 1-5 As shown, the present invention aims at the traditional flow prediction method which usually relies on a large amount of observation data, while in the information-deficient basin, the lack of feature data may lead to inaccurate prediction; the main purpose is to propose a universal intelligent flow prediction technology, which collects parameters in the data-rich area to train the model and then migrates it to the information-deficient area for prediction; by designing each module of the model, especially the attention module, adding the calculation of feature similarity and the convolutional attention mechanism, the attention module can better identify the temporal trend of hydrological information.

[0032] The present invention consists of three submodules: source basin training, similarity calculation and target basin prediction. The model idea of ​​Transformer is adopted as a whole, and some modules are modified and optimized.

[0033] Source basin training refers to collecting feature data in places with sufficient data, training the basic model, and embedding the feature projection at the multi-perceptron MLP. After encoder processing, the model can improve its learning ability of hydrological feature time series information in the attention module, and then generate the prediction results of the source basin through the decoder.

[0034] The similarity calculation is to combine the attention vectors of the source basin and the target basin, calculate the similarity score of the source basin and the target basin through the domain classifier, and finally obtain the similarity between the source basin and the target basin.

[0035] The target domain prediction is to optimize the runoff prediction results of the target domain by combining the pre-trained parameters of the source watershed model with the calculation results of the attention module similarity.

[0036] This paper proposes an intelligent flow prediction technology suitable for information-deficient watersheds. The specific model structure is as follows: Figure 1 As shown:

[0037] Input layer (dynamic vector, static vector): input hydrological characteristics and time series information, converted into vectors;

[0038] MLP module: Multilayer Perceptron (MLP), a type of artificial neural network (ANN), which consists of multiple layers of neurons, where the neurons in each layer are connected to all neurons in the previous layer. The purpose of using the MLP module is to create a combined hidden representation or embedding for the dynamic vector and the static vector by projecting them into a common latent space;

[0039] Encoder module: The encoder is constructed by introducing the Transformer's multi-head attention and feedforward neural network, and the convolutional attention mechanism is introduced into the attention mechanism, so that the model can better learn the periodic trends in the time series. Inside the encoder is a complete Transformer network, which receives the embedding vector output by the MLP, converts the input data into a set of hidden representations, and extracts the features in the input data;

[0040] Decoder module: Introduce Transformer self-attention, decoder-encoder self-attention and feedforward neural network, input the output vector of the attention module into the decoder, and finally output the predicted runoff flow with the time step;

[0041] Domain classifier module: Calculate attention for the source domain and the target domain respectively to obtain their respective attention vectors. This vector is used to measure the correlation between the current state of the decoder and the hidden state of the encoder, so as to more focus on selecting certain parts of the source domain for decoding at the current time step. Based on the attention vector, calculate the similarity score, and finally obtain the similarity between the source domain and the target domain;

[0042] Result output module: weight calibration is performed based on the output prediction results of the target basin and the similarity between the source basin and the target basin to obtain accurate runoff prediction results of the target basin;

[0043] Input Layer:

[0044] The design of the model input layer is as follows Figure 2As shown in the figure: Before the model is input, the input vector is divided into dynamic vector and static vector according to whether the watershed characteristics have changed. Dynamic vector refers to characteristics that change over time, such as maximum temperature, minimum temperature, rainfall, etc.; static vector refers to watershed characteristics that remain fixed for a long time, such as soil type, initial soil moisture content, climate type, groundwater flow, etc. After the initial input of dynamic vector and static vector is completed, the two vectors are projected into a common latent space through the multi-layer perceptron MLP to create a combined hidden representation or embedding for them. This output representation can be input into the encoder.

[0045] Encoder and Decoder:

[0046] The structure of encoder and decoder is as follows Figure 3 As shown in the figure, the overall structure adopts the attention + feedforward neural network structure. The difference is that there is a decoder-encoder attention layer in the decoder. The input contains not only the encoder information, but also the decoder information. The multi-head attention mechanism is used in the self-attention module. Figure 4 Shown: a i represents the current input sequence, q i refers to the representation of the current position input; k i Represents the representation of each position in the input sequence. Similar to q, k is used to measure the degree of association between the current position and other positions in the input sequence, thereby determining the weight allocated in the attention; v i Represents the actual value of each position in the input sequence. When calculating attention, v at each position is weighted averaged, where the weight is determined by the degree of association between q and k. In this way, the model can focus on the relevant part of the input sequence at the current position. i , k i , v i On this basis, multi-head attention is mapped to different subspaces, and q, k, v are divided more finely, focusing on different aspects of the input sequence, improving the model's expressiveness and generalization capabilities. At the same time, convolutional attention is added to the multi-head attention mechanism. Figure 5 As shown: Figure 5 The left figure shows that the original Transformer multi-head attention mechanism uses dot product calculations, lacks feature learning of the previous and next time periods, resulting in the model only paying attention to the value of a certain position, and does not fully utilize the local time relationship. Figure 5 The right picture shows the improved convolution layer with kernel size k to match the relevant feature trends.

[0047] Overall, the encoder is responsible for converting the input sequence into a series of hidden representations, which contain the information of the input sequence; the decoder accepts the hidden representation output by the encoder and generates the target sequence.

[0048] Domain Classifier:

[0049] The domain classifier calculates the similarity between the vector output by the attention module of the source domain and the vector output by the attention module of the target domain. The similarity calculation in this part adopts the cosine similarity measurement method. The calculation formula is as follows:

[0050]

[0051] Among them, S represents the source basin, T represents the target basin, N(S) is the attention vector of the source basin, N(T) is the attention vector of the target basin, and the similarity score is calculated by dividing the inner product of vector S and vector T by the modulus of the vector. The final value range is between [0,1]. When the value is close to 1, it means that the attention features of the two regions are relatively similar; when the value is close to 0, it means that the attention features of the two regions are quite different.

[0052] Result output: The final output result is shown in the formula:

[0053] qcms T =qcms S *sim ST

[0054] Among them, qcms T represents the predicted flow of the target basin, qcms S represents the predicted flow of the model trained in the source basin in the target basin, sin ST Represents the similarity of hydrological features between the target basin and the source basin for training. By considering feature similarity, domain adaptation can be attempted. This helps make the model better adaptable to different environments.

[0055] The present invention aims at the problem that traditional flow prediction methods usually rely on a large amount of observation data, and in information-deficient basins, the lack of observation data may lead to inaccurate predictions. By training the model in the area with rich observation data, the relationship between potential hydrological characteristics and runoff is extracted, and the pre-trained model is used to perform flood prediction from the area with rich data to the area with scarce information, and then the runoff in the current data-sparse area in the next 24 hours is accurately predicted; when performing water resource management, the model prediction data of the present invention can be used to make advance estimates of the runoff in the area lacking observation data, so that users can understand the flow changes in the river section of the basin. In flood prevention, it can be predicted in advance to minimize the losses caused by disasters; in experiments, it can simulate climate and rainfall and other conditions to facilitate understanding of the impact of climate change on the hydrological cycle. Therefore, the academic value and social value of the present invention are extremely high.

[0056] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0057] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0058] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An intelligent flow prediction method for information-deficient watersheds, It is characterized in that The following steps are involved: The Transformer model is used as the basic model, and feature data is collected in the source basin with sufficient data to train the basic model and build a source basin model, wherein feature projection embedding is performed at the multi-perceptron MLP. After encoder processing, the model improves the learning ability of the hydrological feature time series information in the attention module, and generates the prediction result of the source basin through the decoder; Based on the source basin model, flow prediction is performed on a target basin with scarce information, and the prediction result is optimized according to the similarity between the source basin and the target basin in combination with the pre-trained parameters of the source basin model; In the process of constructing the source basin model, according to the input hydrological characteristics and time series information, dynamic vectors and static vectors are generated according to the changes in basin characteristics, wherein the dynamic vector refers to the characteristics that change over time, and the static vector refers to the basin characteristics that remain fixed for a long time; In the process of building the source domain model, the dynamic vector and the static vector are projected into a common latent space through a multi-layer perceptron MLP to create a combined hidden representation or embedding and sent to the encoder.

2. According to claim 1, a method for intelligent flow prediction in information-deficient watersheds, Features: In the process of constructing the source basin model, the encoder is constructed through a self-attention layer and a feed-forward neural network; A decoder is constructed through the self-attention layer, the decoding-encoding attention layer and the feedforward neural network, wherein the decoding-encoding attention layer contains information of the encoder and information of the decoder.

3. According to claim 2, a method for intelligent flow prediction in information-deficient watersheds, Features: In the process of constructing the source basin model, the self-attention layer is generated according to the multi-head attention mechanism.

4. According to claim 3, a method for intelligent flow prediction in information-deficient watersheds, Features: In the process of generating the self-attention layer, the self-attention layer is generated by adding convolutional attention to the multi-head attention mechanism, wherein the convolutional attention represents a convolutional layer with a kernel size of k to match relevant feature trends.

5. According to claim 4, a smart flow prediction method for information-deficient watersheds, Features: In the process of obtaining the similarity between the source basin and the target basin, the similarity is calculated based on the first vector output by the self-attention layer of the source basin and the second vector output by the self-attention layer of the target basin.

6. According to claim 5, a method for intelligent flow prediction in information-deficient watersheds, Features: In the process of similarity calculation, the similarity is calculated by cosine similarity.

7. According to claim 6, a method for intelligent flow prediction in information-deficient watersheds, Features: In the process of optimizing the prediction results, the predicted flow of the target basin is optimized through the calculation results of the similarity.

8. An intelligent flow prediction system for information-deficient basins, It is characterized in that include: A model building module is used to use the Transformer model as a basic model, collect feature data in a source basin with sufficient data, train the basic model, and build a source basin model, wherein feature projection embedding is performed at the multi-perceptron MLP, and after encoder processing, the model improves the learning ability of the hydrological feature time series information in the attention module, and generates the prediction result of the source basin through the decoder; A flow prediction module, used to predict the flow of a target basin with scarce information based on the source basin model, and optimize the prediction result according to the similarity between the source basin and the target basin in combination with the pre-trained parameters of the source basin model; In the process of constructing the source basin model, according to the input hydrological characteristics and time series information, dynamic vectors and static vectors are generated according to the changes in basin characteristics, wherein the dynamic vector refers to the characteristics that change over time, and the static vector refers to the basin characteristics that remain fixed for a long time; In the process of building the source domain model, the dynamic vector and the static vector are projected into a common latent space through a multi-layer perceptron MLP to create a combined hidden representation or embedding and sent to the encoder.

Citation Information

Patent Citations

  • Few-sample target detection method based on semantic information fusion

    CN110555475A

  • Urban traffic flow reasoning method based on dynamic multi-view graph neural network

    CN114925836A

  • LSTM-based medium and long term runoff forecasting intelligent method

    CN115115111A

  • Multi-modal trajectory prediction method and device based on space-time scene attention mechanism

    CN116182863A

  • Hydrological time sequence migration method based on Seq2Seq

    CN116205357A