A method and system for calculating river extreme flow based on Transformer

By introducing time window mechanism and sliding window method into the Transformer model, the limitations of LSTM when handling ultra-long sequence data and extreme traffic events are solved, and more efficient calculations and more accurate extreme traffic simulation are achieved.

CN119783548BActive Publication Date: 2025-05-20NANJING HYDRAULIC RES INST
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510265506.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-03-07
Publication Date
2025-05-20
Estimated Expiration
2045-03-07

AI Technical Summary

Technical Problem

The long-term short-term memory network LSTM has limitations in processing ultra-long sequence data and capturing complex long-distance dependencies. It has long training time, is difficult to compute in parallel, and has low simulation accuracy for extreme traffic events.

Method used

The river channel extreme flow calculation method based on Transformer is adopted, and the scope of attention calculation is limited by introducing a time window mechanism, local short-term dependence characteristics are captured, and the sliding window method is used to balance the local and global feature relationships between multiple time windows.

Benefits of technology

It reduces the computational complexity, improves the simulation ability of extreme traffic events, enhances the model's ability to learn global information, and avoids the time window cutting the complete flood process and information omissions occur.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119783548B_ABST
    Figure CN119783548B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for calculating extreme flow in a river channel based on Transformer. Firstly, daily rainfall data of each station in the river channel basin is collected and the start and end time and length of the daily rainfall data of each station are kept consistent, so as to obtain the collected data. Then, the collected data is position-encoded and the encoded data is obtained. The present invention has the function of introducing a time window mechanism and a runoff classification technology in the Transformer model to limit the attention calculation range and capture local short-term dependency features, thereby reducing the computational complexity. It can not only optimize the simulation of extreme flow events and improve the simulation capability of extreme flow while ensuring the simulation accuracy of small and medium flow, but also realize the function of balancing the local and global feature relationships between multiple time windows by a sliding window method, thereby enhancing the model's ability to learn global information and avoiding the situation where the time window cuts the complete flood process and information omission occurs. The present invention is suitable for wide promotion and use.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of hydraulic engineering measurement technology, and in particular to a method and system for calculating river extreme flow based on Transformer. Background Technology

[0002] Physical models and data-driven machine learning models are widely used in the field of hydrological simulation; and the long short-term memory network LSTM, as an improved form of the recurrent neural network RNN, is widely used in hydrological simulation due to its excellent performance in processing time series data.

[0003] At present, the long short-term memory network LSTM has certain limitations in processing ultra-long sequence data and capturing complex long-distance dependencies, such as long training time, difficulty in parallel computing, and limited ability to capture global features; and because extreme flow events such as floods and droughts occur less frequently in training data, and the model does not learn enough about these rare events during training, the simulation accuracy of extreme flow is not high, and the attention mechanism tends to focus on global information and may ignore local short-term dependency features, which is not conducive to accurately simulating extreme events; therefore, it is necessary to design a Transformer-based river extreme flow measurement method and system. SUMMARY OF THE INVENTION

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and to better and effectively solve the limitations of the long short-term memory network (LSTM) in processing ultra-long sequence data and capturing complex long-distance dependencies, such as long training time, difficulty in parallel calculation, and limited ability to capture global features; and because extreme flow events such as floods and droughts appear less frequently in training data, and the model does not learn enough about these rare events during the training process, this leads to low simulation accuracy of extreme flows, and the attention mechanism tends to focus on global information and may ignore local short-term dependency features, which is also a disadvantage for accurately simulating extreme events. A method and system for calculating extreme flow in a river based on Transformer is provided, which realizes the function of introducing a time window mechanism in the Transformer model to limit the scope of attention calculation and capture local short-term dependency features, which not only reduces the computational complexity, but also optimizes the simulation of extreme flow events and improves the simulation capability of extreme flow while ensuring the accuracy of small and medium flow simulation, and also realizes the function of balancing the local and global feature relationships between multiple time windows by using a sliding window method, which not only enhances the model's learning ability of global information, but also avoids the situation where the time window cuts the complete flood process and information is omitted.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for calculating extreme river flow based on Transformer, comprising the following steps:

[0007] Step A: Collect daily rainfall data of each station in the river basin and make the start and end times and lengths of the daily rainfall data of each station consistent to obtain the collected data.

[0008] Step B: Perform positional encoding on the collected data to obtain the encoded data.

[0009] Step C: Use a sliding window mechanism to divide the sliding time window for the encoded data to obtain the data after window division.

[0010] Step D: Input the data after window division into the classification encoder and classification decoder in sequence for processing to obtain the classified runoff dataset.

[0011] Step E: Input the classified runoff dataset into the low-flow encoder, high-flow encoder, and extremely high-flow encoder respectively to obtain the calculation result of the extreme river flow, and complete the calculation operation of the extreme river flow.

[0012] For the aforementioned method for calculating extreme river flow based on Transformer, in Step A, collect daily rainfall data of each station in the river basin and make the start and end times and lengths of the daily rainfall data of each station consistent to obtain the collected data. Among them, making the start and end times and lengths of the daily rainfall data of each station consistent specifically forms matrix, is the total number of days of meteorological data records, is the number of stations.

[0013] For the aforementioned method for calculating extreme river flow based on Transformer, in Step B, perform positional encoding on the collected data to obtain the encoded data. Among them, the positional encoding is used to map the output vector to the time series, and the specific positional encoding formula is shown in Formula (1):

[0014] ,

[0015] (1)

[0016] Among them, is the positional encoding, is the encoding position, is the positional encoding dimension, is the vector dimension.

[0017] The above-mentioned method for calculating extreme river flow based on Transformer, step C, uses a sliding window mechanism to divide the encoded data into sliding time windows to obtain the data after window division, where the sliding window mechanism limits the attention calculation in the encoded data to a time window of length and reduces the computational complexity to , and the time series in the sliding time window division is , and each time window contains time steps. The time window is shown in formula (2),

[0018] (2)

[0019] where is the index number of the sliding window, used to identify the k-th time window.

[0020] The above-mentioned method for calculating extreme river flow based on Transformer, step D, sequentially inputs the data after window division into a classification encoder and a classification decoder for processing to obtain a dataset after runoff classification. The specific steps are as follows:

[0021] Step D1, input the data after window division into the classification encoder to obtain the data after classification encoding. The classification encoder includes a multi-head self-attention layer and a fully connected neural network layer. The multi-head self-attention layer is used to construct the long-range dependence of each position to capture the relationship between the data after window division. The fully connected neural network layer consists of an input layer, a hidden layer, and an output layer. The fully connected neural network layer is used to further integrate and map the relationship between the data after window division captured by the multi-head self-attention layer. Specifically, the multi-head self-attention layer linearly transforms the input sequence to obtain a query vector Q, a key vector K, and a value vector V, then calculates the attention weights for each pair of query vector Q, key vector K, and value vector V and sums them up with weights to obtain an attention representation, and then linearly transforms the output after concatenating the attention representations;

[0022] Step D2, input the data after classification encoding into the classification decoder to obtain the data after classification decoding. The classification decoder includes a full-mask self-attention layer, a multi-head cross-attention layer, and a fully connected neural network layer. The full-mask self-attention layer is used to mask the future information that has not been predicted during the decoding process and enables the classification decoder to only access the determined historical position signals at each time step, thereby preventing information leakage and ensuring that the prediction process conforms to the causality of the time series. The multi-head cross-attention layer is used to establish the long-range dependence between the output of the classification encoder and the input of the classification decoder. The fully connected neural network layer is used to process the data after classification encoding using residual connection and layer normalization.

[0023] The aforementioned method for calculating extreme river flow based on Transformer, step E: Input the classified runoff datasets into the low-flow encoder, high-flow encoder, and extremely high-flow encoder respectively to obtain the calculation results of extreme river flow and complete the operation of calculating extreme river flow. Each time window in the classified runoff dataset will output the flow prediction result of the last day of this time window . Then, according to the division order and position encoding of the classified runoff dataset, reorganize the flow output results of each sliding window and obtain the calculation result of extreme river flow as .

[0024] An extreme river flow calculation system based on Transformer includes a data acquisition module, a data encoding module, a data division module, a data classification module, and a data calculation module. The data acquisition module is used to collect the daily rainfall data of each station in the river basin and keep the start and end times and lengths of the daily rainfall data of each station consistent, so as to obtain the collected data. The data encoding module is used to perform position encoding on the collected data and obtain the encoded data. The data division module is used to divide the encoded data by using the sliding window mechanism to obtain the data after window division. The data classification module is used to input the data after window division into the classification encoder and classification decoder in sequence for processing and obtain the classified runoff dataset. The data calculation module inputs the classified runoff dataset into the low-flow encoder, high-flow encoder, and extremely high-flow encoder respectively and obtains the calculation result of extreme river flow, thus completing the operation of calculating extreme river flow.

[0025] The beneficial effects of the present invention are as follows: A method and system for calculating extreme river flow based on Transformer of the present invention first collect daily rainfall data of each station in the river basin and keep the start and end times and lengths of the daily rainfall data of each station consistent, so as to obtain the collected data. Then, position encoding is performed on the collected data to obtain the encoded data. Next, the encoded data is divided into sliding time windows by using a sliding window mechanism to obtain the data after window division. Then, the data after window division is sequentially input into a classification encoder and a classification decoder for processing to obtain the dataset after runoff classification. Subsequently, the dataset after runoff classification is respectively input into a low-flow encoder, a high-flow encoder, and an extremely high-flow encoder to obtain the calculation result of the extreme river flow, thus completing the operation of calculating the extreme river flow; effectively realizing that the method and system for calculating the extreme river flow have the function of introducing a time window mechanism in the Transformer model to limit the scope of attention calculation and capture local short-term dependence features, and can directly model the dependence relationship between any positions in the sequence, with the advantage of parallel computing, which shows higher efficiency and performance when processing long sequence data. It not only reduces the computational complexity, but also can be optimized for the simulation of extreme flow events and improve the simulation ability of extreme flow while ensuring the simulation accuracy of medium and small flows. It also realizes the function of balancing the local and global feature relationships between multiple time windows in a sliding window manner, which not only enhances the model's learning ability of global information, but also avoids the situation of cutting the complete flood process by time windows and missing information. At the same time, the runoff classification simulation method is introduced for training and prediction respectively, which enhances the model's learning ability of extreme events. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is the overall flowchart of a method for calculating extreme river flow based on Transformer of the present invention;

[0027] Figure 2 is a schematic diagram of the working principle of the multi-head self-attention layer of the present invention;

[0028] Figure 3 is a comparison chart of the simulated daily runoff results during the test period in the actual example of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The present invention will be further described below in conjunction with the accompanying drawings of the specification.

[0030] As Figure 1 shown, a method for calculating extreme river flow based on Transformer of the present invention includes the following steps

[0031] Step A, collect the daily rainfall data of each station in the river basin and make the start and end times and lengths of the daily rainfall data of each station consistent to obtain the collected data. Specifically, making the start and end times and lengths of the daily rainfall data of each station consistent means forming matrix, is the total number of days of meteorological data records, is the number of stations.

[0032] Step B, perform position encoding on the collected data to obtain the encoded data. The position encoding is used to map the output vector to the time series. The specific position encoding formula is shown in Formula (1),

[0033] ,

[0034] (1)

[0035] Among them, is the position encoding, is the encoding position, is the position encoding dimension, is the vector dimension.

[0036] Step C, use the sliding window mechanism to perform sliding time window partitioning on the encoded data to obtain the window-partitioned data. The sliding window mechanism limits the attention calculation in the encoded data to a time window with a length of and reduces the computational complexity to , and the time series in the sliding time window partitioning is , and each time window contains time steps. The time window is shown in Formula (2),

[0037] (2)

[0038] Among them, is the index number of the sliding window, which is used to identify the k-th time window.

[0039] As Figure 2 shown, Step D, sequentially input the window-partitioned data into the classification encoder and the classification decoder for processing to obtain the runoff classification dataset. The specific steps are as follows,

[0040] Step D1: Input the data after window division into the classification encoder to obtain the data after classification encoding. The classification encoder includes a multi-head self-attention layer and a fully-connected neural network layer. The multi-head self-attention layer is used to construct the long-range dependence of each position to capture the relationship between the data after window division. The fully-connected neural network layer consists of an input layer, a hidden layer, and an output layer, and is used to further integrate and map the relationship between the data after window division captured by the multi-head self-attention layer. Specifically, the multi-head self-attention layer linearly transforms the input sequence to obtain the query vector Q, the key vector K, and the value vector V, then calculates the attention weights for each pair of the query vector Q, the key vector K, and the value vector V and sums them up with weights to obtain the attention representation. Furthermore, the attention representations are concatenated and linearly transformed for output.

[0041] Step D2: Input the data after classification encoding into the classification decoder to obtain the data after classification decoding. The classification decoder includes a full-mask self-attention layer, a multi-head cross-attention layer, and a fully-connected neural network layer. The full-mask self-attention layer is used to mask the future information that has not been predicted during the decoding process and enables the classification decoder to only access the determined historical position signals at each time step, thereby preventing information leakage and ensuring that the prediction process conforms to the causality of the time series. The multi-head cross-attention layer is used to establish the long-range dependence between the output of the classification encoder and the input of the classification decoder. The fully-connected neural network layer is used to process the data after classification encoding by using residual connection and layer normalization.

[0042] Step E: Input the dataset after runoff classification into the low-flow encoder, high-flow encoder, and extremely high-flow encoder respectively to obtain the calculation results of the extreme channel flow, and complete the operation of calculating the extreme channel flow. Each time window in the dataset after runoff classification will output the flow prediction result of the last day of this time window , and then reorganize the flow output results of each sliding window according to the division order and position encoding of the dataset after runoff classification to obtain the calculation result of the extreme channel flow as .

[0043] To better illustrate the usage effect of the present invention, a specific embodiment of the present invention is introduced below.

[0044] S1. Improve the computational efficiency of the model: Introduce the time window mechanism, reduce the complexity of attention calculation, and improve the computational efficiency of the model. Constructed numerical examples and generated a random dataset containing 10,000, 50,000, and 100,000 samples respectively. Each sample contains 20 feature values and 1 target value. Use the LSTM model and the original

[0045] The Transformer model and the Transformer model proposed by the present invention are trained. To ensure that the results are statistically significant, in this embodiment, each model is trained independently 10 times, the training time for each training is recorded, and the average training time is calculated. The computer CPU configuration used in the numerical example is i7-12700k, and the GPU configuration is RTX3060Ti. The following table shows the training efficiency of the LSTM (abbreviated as L), the original Transformer (abbreviated as T), and the Transformer of the present invention (abbreviated as MT) models under different sizes of data sets. The results show that the Transformer model proposed by the present invention has a training efficiency improvement of 6.5 - 7.5 times and 3.8 - 4 times respectively compared with the LSTM and the original Transformer models.

[0046] Table 1. Comparison of the training efficiency of each model under different scales of data sets (unit: second)

[0047]

[0048] S2. Enhance the simulation ability for extreme flows: By means of runoff classification simulation, the model's learning and prediction abilities for extreme flow events are significantly improved, making up for the deficiencies of traditional models in extreme event simulation. At the same time, good accuracy can also be maintained in the simulation of medium and small flows. In this embodiment, based on the daily precipitation data of 39 rain gauges in a certain basin from 1960 to 2019 and the daily runoff data of a certain hydrological station in this basin, the Transformer model proposed by the present invention, the original Transformer model, the LSTM model, and other machine learning models commonly used in the field of rainfall-runoff prediction are respectively used for training and verification. The model training period is from 1972 to 2007 (with 12,783 days of valid data records), the verification period is from 2008 to 2012 (with 1,827 days of valid data records), and the test period is from 2014 to 2018 (with 1,826 days of valid data records). Some years are excluded due to the lack of flow data at some stations. The proportions of the number of days in the training period, verification period, and test period are 78%, 11%, and 11% respectively. To ensure the comparability of the training effects of each model, a consistent early stopping strategy for training is adopted. The simulation effect in the verification period is evaluated during each training. When the evaluation coefficient of the model in the verification period does not show a significant improvement after consecutive multiple trainings, the training is terminated, and the simulation results in the test period are used as the basis for evaluating the simulation effect of the model. The Nash-Sutcliffe efficiency (NSE), the ratio of root mean square error to the standard deviation of observed values (RSR), the percentage bias (PBIAS), and the absolute percentage error of extreme flows in the top 2% of the discharge frequency (APE-2%) commonly used in hydrological models are used as simulation effect evaluation indicators. The simulation results of each model in the verification period and test period are shown in the following table. The Transformer model of the present invention shows significantly better simulation performance than other models in both the verification period and the test period. The NSE values of the Transformer model of the present invention in the verification period and test period are 0.91 and 0.85 respectively, which are significantly higher than those of the original Transformer (0.79 and 0.79), LSTM (0.73 and 0.68), and the other common models (the NSE of ANN, WLSTM, and BiLSTM in the verification period and test period is lower than 0.8). At the same time, the RSR values of the Transformer model of the present invention in the verification period and test period are 0.29 and 0.39 respectively, which are significantly lower than those of other models, showing the robustness of the model in the measurement of relative standard deviation of errors. In terms of the bias index (PBIAS), the Transformer of the present invention is 3.96% in the verification period and -0.14% in the test period, which is basically controlled within a very low bias range.For the simulation of high-flow extreme values (APE-2% index), the absolute percentage errors of the Transformer model of the present invention are 23.80% and 21.53% respectively during the validation period and the test period, significantly better than models such as the original Transformer (about 24.46% - 30.07%) and LSTM (about 27.59% - 28.86%).

[0049] Overall, the Transformer model of the present invention is significantly superior to traditional and existing deep learning models in various evaluation indicators, and has remarkable improvements in fitting accuracy, error control, and the ability to capture extreme events. This result indicates that the Transformer structure of the present invention has more excellent generalization and stability performance in runoff prediction based on precipitation data from multiple rain gauges.

[0050] Table 2. Comparison table of simulation effects of each model during the validation period and the test period

[0051]

[0052] As Figure 3 shown, in the visual comparison of the runoff simulation results during the test period, the prediction curve of the Transformer model of the present invention is closer to the measured data at the high-flow peak. In contrast, there are more obvious deviations and underestimations in the simulation of several flood-season peaks by the original Transformer, especially the inaccurate capture of the timing and amplitude of large-flow events. The Transformer of the present invention is not only more consistent with the observed data in the overall trend, but also more sensitive to the flow peak, thus showing better characterization ability in extreme runoff simulation. This visual result is consistent with the improvement trend of the aforementioned quantitative indicators (such as NSE, RSR, PBIAS, APE-2%), further verifying the superior performance of the Transformer of the present invention in practical application scenarios.

[0053] A Transformer-based extreme river flow measurement system includes a data acquisition module, a data encoding module, a data partitioning module, a data classification module, and a data measurement module. The data acquisition module is used to collect daily rainfall data of each station in the river basin and keep the start and end times and lengths of the daily rainfall data of each station consistent, so as to obtain the collected data. The data encoding module is used to perform position encoding on the collected data and obtain the encoded data. The data partitioning module is used to partition the encoded data using a sliding window mechanism to obtain the data after window partitioning. The data classification module is used to input the data after window partitioning into a classification encoder and a classification decoder in sequence for processing and obtain the classified runoff dataset. The data measurement module inputs the classified runoff dataset into a low-flow encoder, a high-flow encoder, and an extremely high-flow encoder respectively and obtains the measurement result of the extreme river flow, thus completing the operation of measuring the extreme river flow.

[0054] In summary, for the Transformer-based extreme river flow measurement method and system of the present invention, first, the daily rainfall data of each station in the river basin is collected and the start and end times and lengths of the daily rainfall data of each station are kept consistent to obtain the collected data. Then, position encoding is performed on the collected data to obtain the encoded data. Next, the encoded data is partitioned using a sliding window mechanism to obtain the data after window partitioning. Then, the data after window partitioning is input into a classification encoder and a classification decoder in sequence for processing to obtain the classified runoff dataset. Subsequently, the classified runoff dataset is input into a low-flow encoder, a high-flow encoder, and an extremely high-flow encoder respectively to obtain the measurement result of the extreme river flow, thus completing the operation of measuring the extreme river flow. It effectively realizes the functions that the extreme river flow measurement method and system can introduce a time window mechanism in the Transformer model to limit the scope of attention calculation and capture local short-term dependence features, and can directly model the dependence relationship between any positions in the sequence, with the advantage of parallel computing, which shows higher efficiency and performance when processing long sequence data. It not only reduces the computational complexity, but also can optimize the simulation of extreme flow events and improve the simulation ability of extreme flows while ensuring the simulation accuracy of medium and small flows. It also realizes the function of balancing the local and global feature relationships between multiple time windows in a sliding window manner, which not only enhances the model's learning ability of global information, but also avoids the situation that the time window cuts the complete flood process and causes information omission. At the same time, a runoff classification simulation method is introduced for training and prediction respectively, which enhances the model's learning ability of extreme events.

[0055] The basic principles, main features and advantages of the present invention have been shown and described above. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for calculating extreme flow in a river based on Transformer, characterized in that: The following steps are included: Step A, collecting daily rainfall data of each station in the river basin and keeping the start and end time and length of the daily rainfall data of each station consistent to obtain collected data; Step B, position encoding the collected data to obtain encoded data; Step C, using a sliding window mechanism to divide the encoded data into sliding time windows to obtain window-divided data; Step D, inputting the window-divided data into the classification encoder and classification decoder in sequence for processing to obtain a runoff classification data set; Step E: Input the classified runoff dataset into the low flow encoder, high flow encoder and extremely high flow encoder respectively to obtain the river extreme flow calculation result and complete the river extreme flow calculation operation. (k-1)s+1 ,x (k-1)s+2 ,…,x (k-1)s+w ] will output the traffic forecast result q for the last day of the time window (k-1)s+w Then, according to the data set division order and position coding after runoff classification, the flow output results of each sliding window are reorganized and the river extreme flow calculation result is obtained as [q w-1 ,q w ,q w+1 …,q n ].

2. According to the Transformer-based river extreme flow measurement method of claim 1, it is characterized by: Step A, collect daily rainfall data of each station in the river basin and keep the start and end time and length of the daily rainfall data of each station consistent to obtain collected data, wherein keeping the start and end time and length of the daily rainfall data of each station consistent specifically forms an n×m matrix, n is the total number of days for meteorological data recording, and m is the number of stations.

3. The method for calculating river extreme flow based on Transformer according to claim 2 is characterized in that: Step B, position encoding the collected data to obtain encoded data, wherein the position encoding is used to map the output vector to a time series. The specific position encoding formula is shown in formula (1): Among them, PE is the position encoding, pos is the encoding position, i is the position encoding dimension, and d m is the vector dimension.

4. The method for calculating river extreme flow based on Transformer according to claim 3 is characterized by: Step C, using a sliding window mechanism to divide the encoded data into sliding time windows to obtain window-divided data, wherein the sliding window mechanism limits the attention calculation in the encoded data to a time window of length w and reduces the computational complexity to O(w 2 ), and the time series in the sliding time window partition is X = [x1, x2, ..., x n ], and each time window contains w time steps, and the time window is shown in formula (2). X k =[x (k-1)s+1 ,x (k-1)s+2 ,…,x (k-1)s+w ] (2) Among them, k is the index number of the sliding window, which is used to identify the kth time window.

5. The method for calculating river extreme flow based on Transformer according to claim 4, characterized in that: Step D: Input the window-divided data into the classification encoder and classification decoder in turn for processing to obtain the runoff classification data set. The specific steps are as follows: Step D1, inputting the window-divided data into a classification encoder to obtain the classified encoded data, wherein the classification encoder includes a multi-head self-attention layer and a fully connected neural network layer, wherein the multi-head self-attention layer is used to construct the long-range dependency of each position so as to capture the relationship between the data after the window division, wherein the fully connected neural network layer is composed of an input layer, a hidden layer and an output layer, wherein the fully connected neural network layer is used to further integrate and map the relationship between the window-divided data captured by the multi-head self-attention layer, wherein the multi-head self-attention layer specifically uses a linear transformation on the input sequence to obtain a query vector Q, a key vector K and a value vector V, and then calculates the attention weight for each pair of query vector Q, key vector K and value vector V and performs a weighted sum to obtain an attention representation, and then linearly transforms the spliced ​​attention representations and outputs them; Step D2, input the classified encoded data into the classified decoder to obtain the classified decoded data, the classified decoder includes a fully masked self-attention layer, a multi-head cross-attention layer and a fully connected neural network layer, the fully masked self-attention layer is used to shield the future information that has not been predicted during the decoding process and make the classified decoder only access the determined historical position signal at each time step to prevent information leakage and ensure that the prediction process conforms to the causal relationship of the time series, the multi-head cross-attention layer is used to establish the long-range dependency between the classification encoder output and the classification decoder input, and the fully connected neural network layer is used to process the classified encoded data using residual connection and layer normalization.

6. A river channel extreme flow measurement system based on Transformer, wherein the specific measurement process of the river channel extreme flow measurement system is based on the river channel extreme flow measurement method according to any one of claims 1 to 5, characterized in that: It includes a data collection module, a data encoding module, a data division module, a data classification module and a data calculation module. The data collection module is used to collect daily rainfall data of each station in the river basin and keep the start and end time and length of the daily rainfall data of each station consistent, so as to obtain the collected data; The data encoding module is used to perform position encoding on the collected data and obtain encoded data; The data partitioning module is used to partition the encoded data into sliding time windows using a sliding window mechanism and obtain the data after window partitioning; The data classification module is used to input the window-divided data into the classification encoder and the classification decoder in sequence for processing and obtain the runoff classification data set; The data calculation module inputs the runoff classification data set into the low flow encoder, the high flow encoder and the extremely high flow encoder respectively and obtains the river extreme flow calculation result, thereby completing the river extreme flow calculation operation.

Citation Information

Patent Citations

  • Transform-based comprehensive feature network traffic classification method

    CN118740428A

  • Runoff prediction method based on improved Transform model

    CN119089943A