Reservoir water level prediction method and device based on multi-modal feature fusion

Abstract

The present invention relates to a reservoir water level prediction method and device based on multi-modal feature fusion, belonging to the technical field of hydrological prediction. The method includes the following steps: obtaining meteorological data, hydrological data and image data of the area where the reservoir is located; constructing a feature extraction and transformation module, and processing the meteorological data through the feature extraction and transformation module to obtain evaporation, reservoir precipitation and water flow; constructing a feature fusion module, the feature fusion module includes a mapping unit and a gated attention module; evaporation, water flow and reservoir precipitation are mapped to the same dimension through the mapping unit, and then pass through the gated attention module to obtain fused features; the fused features are subjected to water level prediction through a reservoir water level prediction module to obtain a water level prediction value. By dynamically fusing multi-source features such as evaporation, water flow, and precipitation through an improved gated attention unit, the present invention can effectively reduce the risk of information redundancy and loss of high-dimensional features, and improve the prediction accuracy and model robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of hydrological prediction, and particularly relates to a reservoir water level prediction method and device based on multi-modal feature fusion. Background Art

[0002] The change of reservoir water level directly affects water resource management, flood control operation, power generation efficiency, etc. Therefore, accurately predicting the reservoir water level is of great significance for aspects such as water resource scheduling, risk warning, and ecological protection. Traditional water level prediction methods mostly rely on a single data source or model, and often face problems such as low accuracy or limited application scope. Existing water level prediction methods mainly include statistical time series analysis, prediction models based on machine learning, and physical models. Although these methods can provide relatively accurate prediction results to a certain extent, they usually rely on a single data source (such as precipitation, flow, etc.), or there are difficulties in modeling complex non-linear relationships. The limitations of traditional methods make the prediction accuracy have a certain degree of instability in complex and changing environments. With the development of information technology, modern hydrometeorological observation means can already provide various types of data (such as meteorological data, remote sensing data, hydrological data, etc.). These data sources and information forms are diverse and cover multi-dimensional influencing factors. By fusing these multi-modal data, the influencing factors of reservoir water level changes can be captured more comprehensively, making up for the deficiencies of single data source methods. The multi-modal feature fusion technology can combine the characteristics of different types of data and improve the accuracy and stability of the prediction model. In recent years, multi-modal feature fusion methods have been widely used in various prediction tasks, especially in natural disaster warning, intelligent transportation, environmental monitoring and other fields. Through technologies such as deep learning and ensemble learning, heterogeneous data from different modalities can be effectively processed, more representative features can be extracted, and the prediction performance can be further improved.

[0003] The present invention proposes a reservoir water level prediction method based on multi-modal feature fusion. By fusing the features of multiple data sources such as meteorology, remote sensing, and hydrology, and combining advanced machine learning algorithms, a more accurate and stable water level prediction model is constructed. Through this method, the uncertainty and complexity of multi-source data can be better processed, thereby improving the accuracy of water level prediction and providing important support for the scientific scheduling of reservoirs. Summary of the Invention

[0004] In order to solve the above problems, the present invention provides a reservoir water level prediction method and device based on multi-modal feature fusion.

[0005] To achieve the above object, the present invention is realized through the following technical solutions:

[0006] The present invention provides a reservoir water level prediction method based on multi-modal feature fusion, including the following steps:

[0007] S1. Obtain meteorological data, hydrological data, and image data of the area where the reservoir is located;

[0008] S2. Construct a feature extraction and transformation module, which includes an evaporation extraction and transformation unit, a reservoir precipitation extraction and transformation unit, and a water flow extraction and transformation unit; the meteorological data, hydrological data, and image data of the area where the reservoir is located are processed by the feature extraction and transformation module to obtain evaporation, reservoir precipitation, and water flow;

[0009] S3. Construct a feature fusion module, which includes a mapping unit and a gated attention module; the evaporation, water flow, and reservoir precipitation are mapped to the same dimension by the mapping unit to obtain an evaporation feature vector, a water flow feature vector, and a reservoir precipitation feature vector, and then pass through the gated attention module to obtain a fused feature;

[0010] S4. The fused feature is used for water level prediction by a reservoir water level prediction module to obtain a water level prediction value.

[0011] Further, step S1 specifically includes:

[0012] The meteorological data includes air temperature , relative air humidity , wind speed , total daily solar radiation, total daily ground reflected radiation, and precipitation per unit area ; the hydrological data includes soil humidity , maximum soil water capacity ; the image data includes reservoir area and vegetation coverage .

[0013] Further, step S2 specifically includes:

[0014] Evaporation characteristics are affected by factors such as temperature T, net radiation Rn, wind speed u2, water vapor pressure qa, and relative humidity α, and E is expressed as , where respectively represent the above influencing factors, represents the evaporation obtained through calculation ;

[0015] In the evaporation extraction and transformation unit, the formula for obtaining evaporation is expressed as follows:

[0016] ,

[0017] Among them, represents the slope of the saturated water vapor pressure curve, represents a multiplication operation, represents the net radiation, represents the air density, represents the specific heat capacity, represents the wind speed, represents the saturated water vapor pressure, represents the actual water vapor pressure, represents the latent heat of water, represents the barometric constant, represents the normalized wind speed;

[0018] Evaporation In the formula representation of, the specific settings of each parameter are as follows:

[0019] ,

[0020] ,

[0021] ,

[0022] ,

[0023] ,

[0024] ,

[0025] ,

[0026] Among them, represents the air temperature, represents the total solar radiation, represents the ground reflected radiation, represents the air pressure, represents the gas constant, represents the relative air humidity, represents the wind speed correction value;

[0027] Reservoir precipitation characteristics Affected by the reservoir area and local precipitation, , where represents the reservoir area and precipitation, is the reservoir precipitation obtained by formula calculation;

[0028] In the reservoir precipitation extraction and conversion unit, the reservoir precipitation The formula representation is as follows:

[0029] ,

[0030] Among them, represents the precipitation per unit area, represents the reservoir area;

[0031] Water flow characteristics are affected by wind speed , temperature , vegetation coverage , and soil relative humidity and are expressed as , where are the above influencing factors, and is the water flow obtained through formula calculation;

[0032] In the water flow extraction and conversion unit, the water flow is expressed by the formula as follows:

[0033] ,

[0034] ,

[0035] where represents the evaporation amount, represents the soil humidity, represents the maximum soil water capacity, respectively represent the wind speed and vegetation coverage, represents the water input (precipitation) of the reservoir minus the evaporation amount, and what is obtained is the net change amount of the reservoir water except for the evaporation amount, represents the influence of soil humidity. When the soil humidity approaches the maximum capacity, the flow of water will be restricted to a certain extent, so the water flow will decrease with the increase of soil humidity, represents the regulating effect of wind speed, temperature and vegetation coverage on the water flow, represents the influence of wind speed. Usually, the greater the wind speed, the faster the evaporation of water, respectively represent the first empirical coefficient, the second empirical coefficient, the third empirical coefficient and the fourth empirical coefficient, which are automatically learned using the support vector machine supervised learning algorithm based on historical data. The historical data is the daily maximum, minimum and average wind speed, temperature in the area where the reservoir is located and the vegetation coverage of the area.

[0036] Furthermore, step S3 specifically includes:

[0037] The evaporation amount characteristics , water flow characteristics and reservoir precipitation characteristics pass through the mapping unit and are mapped to the same dimension to obtain the evaporation amount feature vector and water flow feature vector And the reservoir precipitation characteristic vector , which is expressed by the formula as follows:

[0038] ,

[0039] ,

[0040] ,

[0041] Among them, are 6×6, 5×6, and 3×6 dimensional respectively, representing the first weight matrix, the second weight matrix, and the third weight matrix, represent the first bias term, the second bias term, and the third bias term respectively;

[0042] The evaporation characteristic vector , the water flow characteristic vector and the reservoir precipitation characteristic vector undergo feature fusion through the gated attention module to obtain the fused feature .

[0043] Furthermore, the gated attention module adopts an improved gated attention mechanism and uses the gated attention mechanism of M layers for feature fusion. The specific operation steps of the layer of the gated attention mechanism are as follows:

[0044] The gated attention neural network at the layer receives the output feature vector of the layer, which is expressed by the formula as follows:

[0045] ,

[0046] Among them, represent the feature vectors of evaporation, water flow, and reservoir precipitation respectively.

[0047] Through linear transformation, each input feature vector is respectively mapped into a query vector Query, a key vector Key, and a value vector Value, which is expressed by the formula as follows:

[0048] ,

[0049] ,

[0050] ,

[0051] Among them, represents the i-th feature vector in the input feature set of the l-th layer, respectively represent at the layer and the The query vector, key vector, and value vector of a feature respectively represent the weight matrices corresponding to the query vector, key vector, and value vector of the layer, respectively represent the bias terms corresponding to the query vector, key vector, and value vector of the layer.

[0052] At the layer, by calculating the similarity between the query vector and the key vector, the attention weights between different features are obtained:

[0053] ,

[0054] where represents the attention weight between the i-th feature and the j-th feature, represents the dimension of the query vector and the key vector, are respectively the query vector of the i-th feature and the key vector of the j-th feature in the layer, The function is used to normalize the weights into a probability distribution to ensure that the weight values are between 0 and 1; according to the attention weights, the value vectors are weighted and summed to obtain the updated feature vector:

[0055] ,

[0056] where represents the representation of the i-th feature after being fused by the attention mechanism in the layer; calculate the similarity between the updated feature vector and the input data:

[0057]

[0058] where represents the similarity; according to the size of the similarity , dynamically adjust the fusion method of the features in the next layer:

[0059]

[0060] where ξ = 0.7 is the set similarity threshold. When , directly use the updated feature vector as the input of the next layer; when , fuse the original feature vector and the updated feature vector by weighting and use it as the input of the next layer;

[0061] After passing through the M-layer gated attention mechanism, all modal features are gradually fused, and the fused features are obtained by weighted summation of the value vectors of the last layer according to the weights:

[0062] ,

[0063] Among them, represents the fused feature, which is the comprehensive representation of all modal features after being fused by the M-layer gated attention mechanism. represents the global attention weight of the j-th feature in the M-th layer, represents the value vector of the j-th feature in the M-th layer.

[0064] Furthermore, step S4 specifically includes:

[0065] The reservoir water level prediction module includes an LSTM module and a fully connected layer;

[0066] Historical water level data , and the fused feature .

[0067]

[0068] Generate the final hidden state through the LSTM module, and the formula is as follows:

[0069] ,

[0070] ,

[0071] ,

[0072] ,

[0073] ,

[0074] ,

[0075] Among them, represents the forget gate, represents the input gate, represents the candidate cell state, represents the updated cell state, represents the output gate, represent the weight matrices of the forget gate, input gate, candidate cell state, and output gate respectively, represent the bias terms of the forget gate, input gate, candidate cell state, and output gate respectively, represents the hidden state at the previous moment, represents the fused feature at the current moment t, represents the hidden state at the current moment t; the final hidden state is mapped through the fully connected layer to obtain the water level prediction value , and the formula is as follows:

[0076] ,

[0077] Among them, represents the final hidden state, represents the time series length of the fused features, represents the weight matrix of the fully connected layer, represents the bias term of the fully connected layer.

[0078] The present invention also provides a reservoir water level prediction device based on multi-modal feature fusion, which executes a reservoir water level prediction method based on multi-modal feature fusion, including:

[0079] Data acquisition unit: used to acquire meteorological data, hydrological data and image data of the area where the reservoir is located;

[0080] Feature extraction and transformation module construction unit, used to construct a feature extraction and transformation module, and the feature extraction and transformation module includes an evaporation extraction and transformation unit, a reservoir precipitation extraction and transformation unit, and a water flow extraction and transformation unit; the meteorological data, hydrological data and image data of the area where the reservoir is located are processed by the feature extraction and transformation module to obtain evaporation, reservoir precipitation and water flow;

[0081] Feature fusion module construction unit, used to construct a feature fusion module, and the feature fusion module includes a mapping unit and a gated attention module; evaporation, water flow and reservoir precipitation are mapped to the same dimension through the mapping unit to obtain an evaporation feature vector, a water flow feature vector and a reservoir precipitation feature vector, and then pass through the gated attention module to obtain fused features;

[0082] Water level prediction unit: used to input the fused features into the reservoir water level prediction module for water level prediction to obtain a water level prediction value.

[0083] The advantages of the present invention are:

[0084] The present invention proposes a reservoir water level prediction method based on multi-modal feature fusion, which combines a gated attention mechanism and an LSTM model, and has the following advantages: ① By using the improved gated attention mechanism to dynamically weight and fuse multi-modal features such as evaporation, water flow, and precipitation, the risk of high-dimensional feature information loss is significantly reduced; ② Introducing a similarity threshold to dynamically adjust the weight ratio between the original features and the updated features further improves the robustness of feature fusion; ③ Using LSTM to capture the short-term and long-term dependencies of time series, combined with a fully connected layer to achieve efficient non-linear mapping, thereby improving the accuracy and stability of reservoir water level prediction, providing a scientific decision-making basis for reservoir operation, flood warning, and ecological protection. ④ Prediction based on evaporation, precipitation, and water flow directly reflects the key hydrological processes of reservoir water level changes, has a stronger physical correlation and interpretability with water level changes, and is more intuitive and easy to quantify compared with climate features, hydrological features, and image features. At the same time, they do not rely on complex environmental monitoring networks or high-precision image processing technologies, so they have lower data acquisition costs and computational complexities, and are suitable for real-time prediction and reservoir management applications.

[0085] The present invention sorts and converts meteorological data, hydrological data, and image data into three categories: evaporation, water flow, and reservoir precipitation; to reduce the loss between high-dimensional features and original features, the gated attention network is improved, and in each layer output, the similarity between the output feature and the original feature is calculated. , if the similarity is less than the threshold then the output of each layer of the gated attention neural network is weighted and fused with the original feature, and the fusion weight of this layer of feature takes , as the input data for the next layer of gating; the fused features are integrated in time series and input into LSTM for reservoir water level prediction. The experimental results show that the reservoir water level prediction method based on multi-modal feature fusion exhibits excellent prediction performance in multiple experimental groups, can significantly reduce the loss of water level prediction, and has better robustness and generalization ability compared with single data sources or single models. The reservoir water level prediction method of the present invention is applicable to fields such as reservoir management, flood warning, and drought monitoring, providing effective technical support for water resource management. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention.

[0087] Figure 1 is the step flow chart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0088] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0089] Embodiment 1

[0090] In this embodiment, as Figure 1 shown, the present invention provides a reservoir water level prediction method based on multi-modal feature fusion, and the specific steps include:

[0091] S1. Obtain meteorological data, hydrological data, and image data of the area where the reservoir is located;

[0092] Specifically, the meteorological data includes air temperature , relative air humidity , wind speed , total daily solar radiation, total daily ground reflected radiation, and precipitation per unit area ; the hydrological data includes soil humidity , maximum soil water capacity ; the image data includes reservoir area and vegetation coverage .

[0093] S2. Construct a feature extraction and conversion module, and the feature extraction and conversion module includes an evaporation extraction and conversion unit, a reservoir precipitation extraction and conversion unit, and a water flow extraction and conversion unit; the meteorological data, hydrological data, and image data of the area where the reservoir is located are processed by the feature extraction and conversion module to obtain evaporation, reservoir precipitation, and water flow;

[0094] Specifically, the evaporation feature is affected by factors such as temperature T, net radiation Rn, wind speed u2, water vapor pressure qa, and relative humidity α, and E is expressed as , where respectively represent the above influencing factors, represents the evaporation obtained through calculation ;

[0095] In the evaporation extraction and conversion unit, the formula for obtaining evaporation is expressed as follows:

[0096] ,

[0097] where, represents the slope of the saturated water vapor pressure curve, Indicates a multiplication operation, Indicates net radiation, Indicates air density, Indicates specific heat capacity, Indicates wind speed, Indicates saturated water vapor pressure, Indicates actual water vapor pressure, Indicates latent heat of water, Indicates pressure constant, Indicates normalized wind speed;

[0098] Evaporation amount In the formula representation of, the specific settings of each parameter are as follows:

[0099] ,

[0100] ,

[0101] ,

[0102] ,

[0103] ,

[0104] ,

[0105] ,

[0106] Among them, Indicates air temperature, Indicates total solar radiation amount, Indicates ground reflected radiation amount, Indicates air pressure, Indicates gas constant, Indicates relative air humidity, Indicates wind speed correction value;

[0107] Reservoir precipitation characteristics Affected by reservoir area and local precipitation, , where Indicates reservoir area and precipitation, Is the reservoir precipitation obtained through formula calculation;

[0108] In the reservoir precipitation extraction and conversion unit, the reservoir precipitation The formula representation is as follows:

[0109] ,

[0110] Among them, Indicates precipitation per unit area, represents the reservoir area;

[0111] Water flow characteristics affected by wind speed , temperature , vegetation coverage , and soil relative humidity , is expressed as , where are the above influencing factors, and is the water flow obtained by formula calculation;

[0112] In the water flow extraction and conversion unit, the water flow is expressed by the formula as follows:

[0113] ,

[0114] ,

[0115] where, represents the evaporation amount, represents the soil humidity, represents the maximum soil water capacity, respectively represent the wind speed and vegetation coverage, represents the water input (precipitation) of the reservoir minus the evaporation amount, and what is obtained is the net change amount of the reservoir water except for the evaporation amount, represents the influence of soil humidity. When the soil humidity approaches the maximum capacity, the flow of water will be restricted to a certain extent, so the water flow will decrease as the soil humidity increases, represents the regulating effect of wind speed, temperature and vegetation coverage on the water flow, represents the influence of wind speed. Usually, the greater the wind speed, the faster the evaporation of water, respectively represent the first empirical coefficient, the second empirical coefficient, the third empirical coefficient and the fourth empirical coefficient, which are automatically learned using the support vector machine supervised learning algorithm based on historical data, and the historical data is the daily maximum, minimum and average wind speed, temperature in the area where the reservoir is located and the vegetation coverage of the area.

[0116] S3. Construct a feature fusion module, and the feature fusion module includes a mapping unit and a gated attention module; the evaporation amount, water flow and reservoir precipitation are mapped to the same dimension through the mapping unit to obtain an evaporation amount feature vector, a water flow feature vector and a reservoir precipitation feature vector, and then pass through the gated attention module to obtain a fusion feature;

[0117] Specifically, the evaporation amount feature , the water flow feature and the reservoir precipitation feature​ Through the mapping unit, it is mapped to the same dimension to obtain the evaporation characteristic vector , the water flow characteristic vector and the reservoir precipitation characteristic vector , which is expressed by the formula as follows:

[0118] ,

[0119] ,

[0120] ,

[0121] Among them, are 6×6, 5×6 and 3×6 dimensional respectively, representing the first weight matrix, the second weight matrix and the third weight matrix, represent the first bias term, the second bias term and the third bias term respectively;

[0122] The evaporation characteristic vector , the water flow characteristic vector and the reservoir precipitation characteristic vector undergo feature fusion through the gated attention module to obtain the fused feature .

[0123] Specifically, the gated attention module adopts an improved gated attention mechanism and uses the gated attention mechanism of M layers for feature fusion. The specific operation steps of the layer of the gated attention mechanism are as follows:

[0124] The gated attention neural network of the layer receives the output feature vector of the layer, which is expressed by the formula as follows:

[0125] ,

[0126] Among them, respectively represent the feature vectors of evaporation, water flow and reservoir precipitation.

[0127] Through linear transformation, each input feature vector is respectively mapped into a query vector Query, a key vector Key and a value vector Value, which is expressed by the formula as follows:

[0128] ,

[0129] ,

[0130] ,

[0131] Among them, represents the $i$-th feature vector in the input feature set of the $l$-th layer, respectively represent at the layer the query vector, key vector, and value vector of the $j$-th feature, respectively represent the weight matrices corresponding to the query vector, key vector, and value vector of the layer, respectively represent the bias terms corresponding to the query vector, key vector, and value vector of the layer.

[0132] At the layer, by calculating the similarity between the query vector and the key vector, the attention weights between different features are obtained:

[0133] ,

[0134] where, represents the attention weight between the $i$-th feature and the $j$-th feature, represents the dimension of the query vector and the key vector, are respectively the query vector of the $i$-th feature and the key vector of the $j$-th feature at the layer, The function is used to normalize the weights into a probability distribution to ensure that the weight values are between 0 and 1; according to the attention weights, the value vectors are weighted and summed to obtain the updated feature vector:

[0135] ,

[0136] where, represents the representation of the $i$-th feature after being fused by the attention mechanism at the layer; calculate the similarity between the updated feature vector and the input data:

[0137]

[0138] where, represents the similarity; according to the size of the similarity , dynamically adjust the fusion method of the features in the next layer:

[0139]

[0140] where $\xi = 0.7$ is the set similarity threshold. When , directly use the updated feature vector as the input of the next layer; when , perform weighted fusion of the original feature vector and the updated feature vector and use it as the input of the next layer;

[0141] After passing through the M-layer gated attention mechanism, all modal features are gradually fused, and the fused features are obtained by weighted summation of the value vectors of the last layer according to the weights:

[0142] ,

[0143] where, represents the fused features, which is the comprehensive representation of all modal features after being fused by the M-layer gated attention mechanism. represents the global attention weight of the j-th feature in the M-th layer, represents the value vector of the j-th feature in the M-th layer.

[0144] S4. The above-mentioned fused features are used for water level prediction through the reservoir water level prediction module to obtain the water level prediction value.

[0145] Specifically, the reservoir water level prediction module includes an LSTM module and a fully connected layer;

[0146] Historical water level data , the fused features .

[0147]

[0148] Generate the final hidden state through the LSTM module, and the formula is as follows:

[0149] ,

[0150] ,

[0151] ,

[0152] ,

[0153] ,

[0154] ,

[0155] where, represents the forget gate, represents the input gate, represents the candidate cell state, represents the updated cell state, represents the output gate, respectively represent the weight matrices of the forget gate, input gate, candidate cell state, and output gate, respectively represent the bias terms of the forget gate, input gate, candidate cell state, and output gate, represents the hidden state at the previous moment, Represents the fused feature at the current moment t, represents the hidden state at the current moment t; the final hidden state is mapped through a fully connected layer to obtain the water level prediction value , and the formula is as follows:

[0156] ,

[0157] where, represents the final hidden state, represents the time series length of the fused feature, represents the weight matrix of the fully connected layer, represents the bias term of the fully connected layer.

[0158] Embodiment 2

[0159] In this embodiment, meteorological data for 180 days in Huangdao District, Qingdao is obtained from the National Meteorological Information Center - China Meteorological Data Network on a daily basis, including: temperature, humidity, wind speed, precipitation, soil humidity, solar radiation, ground-reflected solar radiation, air pressure, relative air humidity, soil maximum water capacity, etc. And the corresponding reservoir area and reservoir water level data of XXX Reservoir in Huangdao District are measured by sensors and stored in the database together. The data set is divided into a training set and a test set in a ratio of 3:1.

[0160] Based on these data, the daily evaporation, water flow, and reservoir precipitation are calculated, and at the same time, the change in the reservoir water level can be obtained. The evaporation, water flow, and reservoir precipitation are mapped to the same dimension (feature vector 1×32 of the same length) to obtain vectors R1, R2, and R3.

[0161] The number of layers of the improved gated attention network is set to 6 layers, the similarity threshold ξ is set to 0.7, the gated activation function uses softmax, and the fusion weights are calculated according to the similarity to ensure that the loss between features is reduced through weighted fusion; the LSTM hidden unit is set to 64, the input feature dimension is 3×32 dimensions, the Adam optimizer is used, the learning rate is set to 0.001, the dynamic decay coefficient is set to 0.9, the acceleration decay coefficient is set to 0.99, the mean square error is used to calculate the prediction loss, the training batch is set to 64, and the training period is set to 200.

[0162] To verify the effectiveness of the reservoir water level prediction method based on multi-modal feature fusion proposed by the present invention, the following experimental verification was carried out.

[0163] To verify the effectiveness of the feature fusion method proposed by the present invention, the following experiment was carried out:

[0164] Table 1 Experimental comparison of feature fusion and direct feature concatenation

[0165]

[0166] As can be seen from Table 1, compared with single-type data and directly concatenating various features, the fused features obtained by the feature fusion method based on gated attention proposed in this paper have more complete information and more accurate prediction results. On the other hand, it also shows that the reservoir water level prediction method based on evaporation, precipitation, and water flow proposed in the present invention has better prediction performance compared to directly concatenating historical data.

[0167] To verify the effectiveness of the method of the present invention, the following experiments are carried out:

[0168] Table 2 Experimental comparison between the method of the present invention and traditional machine learning methods

[0169]

[0170] As can be seen from Table 2, the prediction module of the reservoir water level prediction method proposed in the present invention using LSTM has certain improvements in indicators such as mean square error, root mean square error, coefficient of determination, and mean absolute error compared with traditional machine learning methods, indicating that the method of the present invention can significantly improve the accuracy of water level prediction and has better robustness and generalization ability compared with traditional machine learning methods.

[0171] Example 3

[0172] This embodiment provides a reservoir water level prediction device based on multi-modal feature fusion, which executes the reservoir water level prediction method based on multi-modal feature fusion as described in Example 1, including:

[0173] Data acquisition unit: used to acquire meteorological data, hydrological data, and image data of the area where the reservoir is located;

[0174] Feature extraction and transformation module construction unit, used to construct a feature extraction and transformation module, and the feature extraction and transformation module includes an evaporation extraction and transformation unit, a reservoir precipitation extraction and transformation unit, and a water flow extraction and transformation unit; the meteorological data, hydrological data, and image data of the area where the reservoir is located are processed by the feature extraction and transformation module to obtain evaporation, reservoir precipitation, and water flow;

[0175] Feature fusion module construction unit, used to construct a feature fusion module, and the feature fusion module includes a mapping unit and a gated attention module; evaporation, water flow, and reservoir precipitation are mapped to the same dimension through the mapping unit to obtain an evaporation feature vector, a water flow feature vector, and a reservoir precipitation feature vector, and then pass through the gated attention module to obtain a fused feature;

[0176] Water level prediction unit: used to input the fused feature into the reservoir water level prediction module for water level prediction to obtain a water level prediction value.

[0177] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A reservoir water level prediction method based on multimodal feature fusion, characterized in that: The following steps are involved: S1. Obtain meteorological data, hydrological data and image data of the reservoir area; S2. Construct a feature extraction and conversion module, which includes an evaporation extraction and conversion unit, a reservoir precipitation extraction and conversion unit, and a water flow extraction and conversion unit; the meteorological data, hydrological data, and image data of the reservoir area are processed by the feature extraction and conversion module to obtain evaporation, reservoir precipitation, and water flow; S3. construct a feature fusion module, the feature fusion module includes a mapping unit and a gated attention module; the evaporation, water flow and reservoir precipitation are mapped to the same dimension by the mapping unit to obtain an evaporation feature vector, a water flow feature vector and a reservoir precipitation feature vector, and then pass through the gated attention module to obtain a fusion feature; The gated attention module adopts an improved gated attention mechanism and uses M layers of gated attention mechanism for feature fusion. The specific operation steps of the layer-gated attention mechanism are: Gated Attention Neural Network Layer accepts The output feature vector of the layer is expressed as follows: ， in, The characteristic vectors representing evaporation, water flow and reservoir precipitation respectively; Through linear transformation, each input feature vector is mapped into a query vector Query, a key vector Key, and a value vector Value. The formula is as follows: ， ， ， in, Represents the first feature set in the input feature set of the first layer feature vectors, Respectively expressed in Tier The query vector, key vector, and value vector of each feature, Respectively represent The weight matrix corresponding to the query vector, key vector, and value vector of the layer, Respectively represent The bias terms corresponding to the query vector, key vector, and value vector of the layer; In the Layer, by calculating the similarity between the query vector and the key vector, the attention weights between different features are obtained: ， in, Indicates Features and The attention weights between features, represents the dimensions of the query vector and key vector, Respectively represent Tier The query vector of the feature and the The key vector of features, The function is used to normalize the weights to a probability distribution to ensure that the weight values ​​are between 0 and 1; according to the attention weights, the value vector is weighted summed to obtain the updated feature vector: ， in, Indicates The feature in The representation of the layer after fusion by the attention mechanism; calculate the similarity between the updated feature vector and the input data: in, Indicates similarity; Based on similarity The size of the next layer is dynamically adjusted to fusion mode: Among them, ξ=0.7 is the set similarity threshold. When , the updated feature vector is directly used as the input of the next layer; when When , the original feature vector and the updated feature vector are weightedly fused and used as the input of the next layer; After M layers of gated attention mechanism, all modal features are gradually fused, and the fused features are obtained by weighted summation of the value vector of the last layer: ， in, represents the fusion feature, which means the comprehensive representation of all modal features after fusion through the M-layer gated attention mechanism; represents the global attention weight of the j-th feature in the M-th layer, The value vector representing the j-th feature of the M-th layer; S4. The fused features are used for water level prediction by a reservoir water level prediction module to obtain a water level prediction value.

2. The reservoir water level prediction method based on multimodal feature fusion according to claim 1 is characterized in that: Step S1 specifically includes: Weather data including air temperature , relative air humidity , wind speed , daily total solar radiation, daily ground reflected radiation, and precipitation per unit area ; Hydrological data including soil moisture , maximum soil water capacity ; Image data includes reservoir area and vegetation coverage .

3. The reservoir water level prediction method based on multimodal feature fusion according to claim 2 is characterized in that: Step S2 specifically includes: Evaporation characteristics Affected by temperature T, net radiation Rn, wind speed u2, water vapor pressure qa, and relative humidity α, E is expressed as ,in Represent the above-mentioned influencing factors respectively. Indicates the calculated evaporation ; In the evaporation extraction and conversion unit, the evaporation is obtained The formula is as follows: ， in, represents the slope of the saturated water vapor pressure curve, represents the multiplication operation, represents the net radiation, represents the air density, represents the specific heat capacity, Indicates wind speed, represents the saturated water vapor pressure, Indicates the actual water vapor pressure, represents the latent heat of water, represents the air pressure constant, represents the normalized wind speed; Evaporation In the formula representation, the specific settings of each parameter are as follows: ， ， ， ， ， ， ， in, Indicates the air temperature, is the total solar radiation, is the amount of radiation reflected from the ground. Indicates air pressure, is the gas constant, Relative air humidity, Indicates wind speed correction value; Reservoir precipitation characteristics Affected by the reservoir area and local precipitation, ,in represents the reservoir area and precipitation, is the reservoir precipitation calculated by the formula; In the reservoir precipitation extraction and conversion unit, the reservoir precipitation is obtained The formula is as follows: ， in, It represents the precipitation per unit area. represents the reservoir area; Water flow characteristics Affected by wind speed ,temperature , vegetation coverage , soil relative humidity Influence, Expressed as ,in For the above factors, is the water flow rate calculated by the formula; In the water flow extraction and conversion unit, the water flow is obtained The formula is as follows: ， ， in, Indicates the evaporation rate, Indicates soil moisture, represents the maximum water capacity of the soil, represent wind speed and vegetation coverage, respectively. It represents the water input of the reservoir, which is the precipitation minus the evaporation. The net change of water in the reservoir excluding the evaporation is obtained. Represents the influence of soil moisture; when soil moisture is close to maximum capacity, the flow of water will be restricted to a certain extent, so the water flow will decrease as soil moisture increases. It indicates the regulating effect of wind speed, temperature and vegetation coverage on water flow. Indicates the influence of wind speed. Generally, the greater the wind speed, the faster the water evaporates. They respectively represent the first empirical coefficient, the second empirical coefficient, the third empirical coefficient and the fourth empirical coefficient, which are automatically learned using a support vector machine supervised learning algorithm based on historical data. The historical data are the daily maximum, minimum and average wind speed, temperature and vegetation coverage in the area where the reservoir is located.

4. The reservoir water level prediction method based on multimodal feature fusion according to claim 3 is characterized in that: Step S3 specifically includes: The evaporation characteristics , water flow characteristics and reservoir precipitation characteristics After the mapping unit is mapped to the same dimension, the evaporation feature vector is obtained , water flow characteristic vector And the reservoir precipitation characteristic vector , the formula is as follows: ， ， ， in, They are 6×6, 5×6 and 3×6 in dimension, representing the first weight matrix, the second weight matrix and the third weight matrix. Respectively represent the first bias term, the second bias term, and the third bias term; The evaporation characteristic vector , water flow characteristic vector And the reservoir precipitation characteristic vector After the gated attention module performs feature fusion, the fused feature is obtained .

5. The reservoir water level prediction method based on multimodal feature fusion according to claim 4 is characterized in that: Step S4 specifically includes: The reservoir water level prediction module includes an LSTM module and a fully connected layer; Historical water level data , fusion features ; The final hidden state is generated by the LSTM module, and the formula is as follows: ， ， ， ， ， ， in, represents the forget gate, represents the input gate, represents the candidate cell state, Indicates updating the cell status. represents the output gate, Represent the weight matrices of the forget gate, input gate, candidate cell state, and output gate respectively, Respectively represent the bias items of the forget gate, input gate, candidate cell state, and output gate, represents the hidden state at the previous moment, represents the fusion feature at the current time t, Represents the hidden state at the current time t; the final hidden state is mapped through the fully connected layer to obtain the water level prediction value , the formula is as follows: ， in, represents the final hidden state, represents the time series length of the fused features, represents the weight matrix of the fully connected layer, Represents the bias term of the fully connected layer.

6. A reservoir water level prediction device based on multimodal feature fusion, executing the reservoir water level prediction method based on multimodal feature fusion as claimed in claim 1, characterized in that: include: Data acquisition unit: used to obtain meteorological data, hydrological data and image data of the reservoir area; A feature extraction and conversion module construction unit is used to construct a feature extraction and conversion module, which includes an evaporation extraction and conversion unit, a reservoir precipitation extraction and conversion unit, and a water flow extraction and conversion unit; the meteorological data, hydrological data, and image data of the reservoir area are processed by the feature extraction and conversion module to obtain evaporation, reservoir precipitation, and water flow; A feature fusion module construction unit, used to construct a feature fusion module, the feature fusion module includes a mapping unit and a gated attention module; Evaporation, water flow and reservoir precipitation are mapped to the same dimension by the mapping unit to obtain evaporation feature vector, water flow feature vector and reservoir precipitation feature vector, and then pass through the gated attention module to obtain the fusion feature; Water level prediction unit: used to input the fusion features into the reservoir water level prediction module for water level prediction to obtain the water level prediction value.

Images