Poyang Lake water level prediction method based on large model

Abstract

The invention belongs to the technical field of water level prediction, and particularly discloses a Poyang Lake water level prediction method based on a large model, which comprises the following steps: (1) collecting historical hydrological and meteorological data; (2) constructing a large model suitable for lake water level prediction; (3) training and optimizing the model; and (4) predicting the water level of the Poyang Lake based on the large model. According to the method, the open-source large model is transformed, a mapping layer from the feature vectors to the semantic space of the large model is introduced, and a prediction layer from the output vector features to the water levels of different stations is designed, so that the large model can better adapt to a water level prediction task due to the structural innovation.

Description

Technical Field

[0001] The invention belongs to the technical field of water level prediction, and specifically discloses a Poyang Lake water level prediction method based on a large model. Background Art

[0002] With global climate change and the increase in human activities, the management and protection of water resources has become increasingly important. As an important freshwater resource, the water level changes of lakes are directly related to the supply of water resources, water ecological balance and flood control safety. Accurately predicting the lake water level is of great significance for formulating water resource management strategies, protecting lake ecosystems and reducing flood disasters. As the largest freshwater lake in China, Poyang Lake is of great significance for maintaining regional ecological balance and water resource supply. Water level changes directly affect the ecological environment of the lake and the flood control safety of the surrounding areas. Accurately predicting the water level of Poyang Lake has important practical application value for lake management and flood control and disaster reduction.

[0003] In the past, water level prediction mainly relied on traditional hydrological models, such as water balance models, hydrodynamic models, etc., and statistical analysis models, such as regression moving average (ARMA), autoregressive integrated moving average (ARIMA), etc. With the development of computer technology, machine learning models such as artificial neural network (ANN), support vector machine (SVM), random forest (RF), long short-term memory model (LSTM), etc., have gradually been applied to the field of water level prediction. However, traditional machine learning models have weak ability to capture long-distance dependencies in time series data and poor feature representation capabilities. With the development of remote sensing technology and Internet of Things technology, a large amount of lake water level, rainfall, evaporation and other data can be obtained. These big data provide new opportunities for hydrological prediction, but also put forward higher requirements for data processing and analysis methods.

[0004] Among the existing technologies, the long short-term memory network (LSTM) is a commonly used time series prediction model, which has been successfully applied to the prediction of the water level of Poyang Lake. LSTM avoids the gradient vanishing problem in the traditional recurrent neural network (RNN) by introducing a gating mechanism, so that it can capture long-term dependencies. For example, Guo, Y.; Lai, XJ. Water level prediction of Lake Poyang based on long short term memory neural network. Journal of Lake Sciences, 2020, 32, 865-876. A Poyang Lake water level prediction model based on LSTM is proposed in the literature. The model uses the flow of the "Five Rivers" such as Ganjiang River and Fuhe River into the lake and the flow of the Yangtze River mainstream as input conditions to predict the water level process in different lake areas of Poyang Lake, and has achieved good prediction results. However, the LSTM model still has the problem of poor feature representation ability and insufficient ability to capture long-distance dependencies when processing ultra-long sequences. Summary of the invention

[0005] In view of the limitations of the prior art, the present invention aims to develop a lake water level prediction method based on a large model, which can process large-scale data sets, extract high-quality feature representations, and capture long-distance dependencies in time series data. The present invention can make up for the weaknesses of traditional machine learning models such as weak ability to capture long-distance dependencies in time series data and poor feature representation capabilities, and improve the accuracy of long-term hydrological time predictions. By utilizing the powerful data processing and feature extraction capabilities of large models, the present invention can better understand and predict changes in lake water levels, providing a scientific basis for water resource management and flood prevention and disaster reduction.

[0006] Specifically, the present invention provides a method for predicting the water level of Poyang Lake based on a large model, which uses the powerful data processing and feature extraction capabilities of the large model to predict the water level changes of Poyang Lake to improve the accuracy of the prediction. It includes:

[0007] (1) Collection of historical hydrological and meteorological data:

[0008] Meteorological data and hydrological data of the Poyang Lake Basin are collected. The present invention takes the data from 1960 to 2013 as an example. Meteorological data include temperature, precipitation, solar radiation, relative humidity and wind speed. Hydrological data include the flow data of the "Five Rivers" (Ganjiang River, Fuhe River, Xinjiang River, Raohe River and Xiushui River) into the lake, and the daily average water level observation data of the five monitoring stations of Hukou, Xingzi, Duchang, Tangyin and Kangshan.

[0009] (2) Construct a large model suitable for lake water level prediction:

[0010] At present, most open source big models such as the Qwen big model mainly process information such as text or images, and cannot directly model and process meteorological data and hydrological data. In order to change and migrate the open source big model to the water level prediction task, a mapping layer from feature vectors to the semantic space of the big model is introduced, and a prediction layer from the output vector features at each moment to the water level of different stations is designed. In addition, a full parameter fine-tuning and efficient fine-tuning strategy is designed based on the model and computing resources, such as Figure 1 shown.

[0011] (3) Model training and optimization:

[0012] The meteorological data from 1960 to 2000, the flow of five rivers into the lake, and the water level data of five monitoring stations were used as the training set, and the data from 2001 to 2013 were used as the test set. The meteorological data, the flow of five rivers into the lake, and the daily average water level data at the mouth of the Yangtze River (Hukou Station) were used as input data, and the daily average water level data of the four monitoring stations of Xingzi, Duchang, Tangyin and Kangshan were used as output data to train the model. All input data were normalized to ensure the stability and efficiency of model training. The learning rate, number of iterations, etc. of the model were set. The root mean square error (RMSE) was used as the loss function to train the model through the back propagation algorithm. The Adam optimizer was used for model training, and early stopping was used to prevent overfitting.

[0013] (4) Poyang Lake water level prediction based on large model:

[0014] The model was tested using data from 2001 to 2013 to evaluate the model's forecasting performance. The model's forecasting accuracy was evaluated by calculating the root mean square error (RMSE) of the test set.

[0015] Beneficial effects of the present invention:

[0016] (1) This paper uses the powerful data processing and feature extraction capabilities of the large model to predict the water level changes in Poyang Lake, which is a significant improvement over traditional machine learning models. The large model can better process large-scale data sets, improve the ability to capture long-distance dependencies in time series data, and extract high-quality feature representations.

[0017] (2) The present invention transforms the open source big model, introduces a mapping layer from feature vectors to the semantic space of the big model, and designs a prediction layer for outputting vector features to water levels at different sites. Such structural innovation enables the big model to better adapt to water level prediction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the algorithm framework proposed by the present invention;

[0019] Figure 2 It is a comparison between the measured daily average water level values ​​at four stations in Poyang Lake and the large model simulation values. DETAILED DESCRIPTION

[0020] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0021] Example 1

[0022] A method for predicting the water level of Poyang Lake based on a large model, comprising:

[0023] (1) Collection of historical hydrological and meteorological data:

[0024] Meteorological data and hydrological data of the Poyang Lake Basin are collected. The embodiment of the present invention uses data from 1960 to 2013. Meteorological data include temperature, precipitation, solar radiation, relative humidity and wind speed. Hydrological data include the flow data of the "Five Rivers" (Ganjiang River, Fuhe River, Xinjiang River, Raohe River and Xiushui River) into the lake, and the daily average water level observation data of the five monitoring stations of Hukou, Xingzi, Duchang, Tangyin and Kangshan.

[0025] (2) Construct a large model suitable for lake water level prediction:

[0026] The present invention aims to transfer the long sequence feature extraction and modeling capabilities of large models mainly used for processing text data to the water level prediction task, so as to better establish the relationship between the water level prediction target and the input meteorological and hydrological characteristics, so as to improve the accuracy of water level prediction.

[0027] At present, most open source big models such as the Qwen big model mainly process information such as text or images, and cannot directly model and process meteorological and hydrological data. In order to change and migrate the open source big model to the water level prediction task, the input layer needs to be modified to upgrade the original big model that can only input text to a big model that can input climate and hydrological feature vectors.

[0028] Specifically, for the input feature vector X d , first map the vector space. Figure 1 , for the eigenvector X d Multiply by a learnable matrix W x Mapped to the vector space H of the original model d =X d W x .W x The dimensions of the matrix W depend on the dimensions of the two vector spaces. x It is also used as part of the model parameters and solved by gradient descent method during the training process of the large model.

[0029] Then comes the specific operation of the classic large model, which models the dependency between input features through the self-attention mechanism:

[0030] Q=W Q X,K=W K X,V=W V X

[0031]

[0032] Where Q represents the query matrix, W Q represents the mapping parameters for the query matrix, X represents the model input, K represents the key matrix, and W K represents the mapping parameters for the key matrix, V represents the value matrix, and W V represents the mapping parameter for the value matrix, K T represents the transpose of matrix K, d k Represents the dimension of the semantic vector space.

[0033] Multiple attention heads are used to process information in parallel to capture more detailed features and dependencies between features. The calculation of multi-head attention is as follows:

[0034] MultiHead(Q,K,V)=Concat(head1,…,head h )W O

[0035] in, The representation of each position is further transformed and nonlinearly processed through a feedforward neural network.

[0036] Then, the output end needs to be modified. The output end of the original large model is the prediction of the next token. In this invention, hydrology needs to be predicted. Therefore, through the multi-layer self-attention mechanism and feedforward network calculation, the feature vector h of the output layer at the end moment can be obtained, and then an output vector feature is added to the prediction layer of water levels at different sites:

[0037]

[0038] in, That is, to predict the hydrological data of all stations, W q and b q These parameters, together with the original large model parameters, can be learned and optimized through the gradient back propagation algorithm.

[0039] (3) Model training and optimization:

[0040] Meteorological factors affect the runoff process and lake water level changes, therefore, meteorological elements (precipitation, temperature, relative humidity, solar radiation and wind speed) are used as input data to train the water level model of the Poyang Lake basin. The lake water level is mainly affected by the basin discharge and the Yangtze River. Therefore, the daily average flow observation data of five rivers, namely Ganjiang River, Fuhe River, Xinjiang River, Raohe River and Xiushui River, were selected as input variables. The water level data of Hukou Station was selected as the model input data to reflect the impact of the Yangtze River on the water level of Poyang Lake. The model output data are the water level data of Xingzi Station, Duchang Station, Tangyin Station and Kangshan Station. The model training period is 1960-2000, and the testing period is 2001-2013. The input data of the large model are normalized using the following function:

[0041] x′=(x i -x min ) / (x max -x min )

[0042] Where i is the position of the observation, max and min are the maximum and minimum values ​​of the data set, x' represents the normalized variable, and x i represents the variable before normalization.

[0043] Then the obtained special vector is input into the previously constructed large model for layer-by-layer feature processing and processing, and the output prediction for hydrology is obtained. In the training stage, the loss parameter uses RMSE, which is as follows:

[0044]

[0045] in is the model prediction value, y i is the true value of the data, and N is the number of samples.

[0046] For the model training phase, two different training methods are used. One is the conventional full parameter update learning method, that is, the parameters of the large model and the newly added mapping layer and output layer must be learned during the training phase; the second method uses an efficient fine-tuning training method based on LORA to cope with the high demand for video memory when training 7B and above models.

[0047] During the training and fine-tuning process, the training data set is represented as: Z = {(x i ,y i )} i＝1,…,N , where x i ,y i They are input features and output watermarks, respectively. The large language model parameters are initialized to the pre-trained weight Φ0 and updated to Φ0+ΔΦ through gradient descent. However, in efficient fine-tuning, the parameter increment By a smaller set of parameters Encode, the training goal is to Optimization. For the pre-trained weight matrix W0∈R d×k , use a low-rank decomposition to represent the parameter update ΔW, that is:

[0048] W0+ΔW=W0+BA

[0049] Among them, W0 is the original parameter of the large model, B and A are two low-rank matrices, and r is the rank parameter. Specifically: B∈R d×r ,A∈R r×k ,r<<min(d,k),

[0050] During training, the parameter W0 is frozen and only the parameters in A and B are trained, that is, for h = W0x i , the forward propagation process becomes:

[0051] h=W0x i +ΔWx i =W0x i +BAx i

[0052] Where h is the hidden layer vector of the large model. This completes the efficient fine-tuning of some parameters of the model, greatly saving computing resources.

[0053] (4) Poyang Lake water level prediction based on large model:

[0054] The model was tested using data from 2001 to 2013 to evaluate the model's forecasting performance. The model's forecasting accuracy was evaluated by calculating the root mean square error (RMSE) of the test set.

[0055] Test results:

[0056] Improved accuracy: Compared with the simulation results of the LSTM model (comparison data source: Xu J, Fan H, Luo M, et al. Transformer based water level prediction in Poyang Lake, China [J]. Water, 2023, 15 (3): 576. Xu et al., 2023), the RMSE of the large model for water level prediction at the four stations has decreased significantly (Table 1), and the prediction accuracy has increased by 32.7%, 22.7%, 16.6% and 27.0% respectively. The results fully show that compared with the traditional model, the large model has significantly improved its ability to capture long-distance dependencies in time series data and has stronger feature representation capabilities.

[0057] Table 1 Comparison of simulation results of water level data of four stations in Poyang Lake by large model and LSTM model

[0058]

Claims

1. A method for predicting the water level of Poyang Lake based on a large model, characterized in that: The steps include: (1) Collect historical hydrological and meteorological data; (2) Build a large model suitable for lake water level prediction: transfer the long sequence feature extraction and modeling capabilities of the large model that mainly processes text data to the water level prediction task; (3) Model training and optimization; (4) Poyang Lake water level prediction based on large model.

2. The Poyang Lake water level prediction method based on a large model according to claim 1 is characterized in that: Step (2) comprises: (2-1) Transform the input layer, the input feature vector X d , first map the vector space; for the eigenvector X d Multiply by a learnable matrix W x Mapped to the vector space H of the original model d =X d W x ; W x The dimensions of the matrix depend on the dimensions of the two vector spaces; W x It is also used as part of the model parameters and solved by gradient descent method during the training process of the large model; (2-2) Model the dependency between input features through the self-attention mechanism; (2-3) Process information in parallel through multiple attention heads to capture more detailed features and dependencies between features; (2-4) The output end is transformed to predict hydrology. After multi-layer self-attention mechanism and feedforward network calculation, the feature vector h of the output layer at the end is obtained, and then an output vector feature is added to the prediction layer of water levels at different stations: in, That is, to predict the hydrological data of all stations, W q and b q are the introduced prediction layer parameters.

3. The Poyang Lake water level prediction method based on a large model according to claim 1 is characterized in that: Step (3): The input data of the large model is normalized using the following function: x′=(x i -x min ) / (x max -x min ) Where i is the position of the observation, max and min are the maximum and minimum values ​​of the data set, x' represents the normalized variable, and x i represents the variable before normalization; Then the obtained special vector is input into the previously constructed large model for layer-by-layer feature processing and processing, and the output prediction for hydrology is obtained. In the training stage, the loss parameter uses RMSE, which is as follows: in, is the model prediction value, yi is the true value of the data, and N is the number of samples.

4. The Poyang Lake water level prediction method based on a large model according to claim 1 is characterized in that: Step (3): Use the LORA mechanism to efficiently and quickly fine-tune the large model. For the pre-trained weight matrix W0∈R d×k , use a low-rank decomposition to represent the parameter update ΔW, that is: W0+ΔW=W0+BA Among them, W0 is the original parameter of the large model, B and A are two low-rank matrices, and r is the rank parameter. Specifically: B∈R d×r , A∈R r ×k , r< <min(d,k), During training, the parameter W0 is frozen and only the parameters in A and B are trained, that is, for h = W0x i , the forward propagation process becomes: h=W0x i +ΔWx i =W0x i +BAx i Where h is the hidden layer vector of the large model.

Images