Prediction optimization method and system based on hydrological flood information abnormal data processing, and medium
By constructing the XGBoost model and deep learning framework, and combining Bayesian optimization and adaptive learning strategies, the problems of misjudgment and omission in traditional hydrological flood reporting methods are solved, realizing automated processing of hydrological data and efficient anomaly detection, thereby improving data quality and flood early warning capabilities.
Patent Information
- Application Number
- CN202510939737.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-08
- Publication Date
- 2025-11-14
AI Technical Summary
Traditional methods for processing abnormal hydrological data rely on human experience and simple threshold algorithms, leading to misjudgments and omissions. They are ill-suited to handling massive amounts of real-time changing hydrological data, and the volatility and uncertainty of hydrological data increase the processing difficulty, especially in extreme weather conditions where it is difficult to accurately capture abnormal fluctuations.
By employing the XGBoost model combined with Bayesian optimization and adaptive learning strategies, a flood forecasting model is constructed using threshold judgment, interpolation, and a deep learning framework. This enables automated data processing and correction, improving the accuracy and efficiency of abnormal data detection.
It has improved the uniqueness and regularity of hydrological flood reporting data, enabled comprehensive monitoring and management of abnormal data, reduced flood losses, and provided more reliable data support.
Smart Images
Figure CN120951150A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of hydrological flood forecasting, and more specifically, to a forecasting optimization method, system, and medium based on the processing of hydrological flood forecasting anomaly data. Background Technology
[0002] Hydrological reporting serves as a crucial support for water resource management, flood and drought control decision-making, and the operation and scheduling of water conservancy projects. It plays a vital role in ensuring the stable development of society and the safety of people's lives and property. Accurate and timely hydrological data can provide a scientific basis for flood warning and rational allocation of water resources, thereby effectively reducing flood disaster losses and improving water resource utilization efficiency.
[0003] Current hydrological reporting and flood control work faces numerous technical challenges. On the one hand, traditional methods for judging abnormal data mainly rely on manual experience and simple threshold algorithms. Manual judgment is inefficient and easily influenced by subjective factors, making it difficult to handle massive and real-time changing hydrological data. Simple threshold algorithms have poor adaptability and cannot accurately identify complex and ever-changing abnormal data patterns, leading to frequent misjudgments and omissions. On the other hand, hydrological data itself is highly volatile and uncertain, influenced by a combination of factors such as seasonal changes, climate change, and human activities. For example, under extreme weather events such as torrential rains and floods, hydrological data can change drastically, and traditional methods struggle to accurately capture and process these abnormal fluctuations. Furthermore, the data quality varies significantly between different hydrological monitoring points, and some monitoring equipment may be faulty or have accuracy deviations, further increasing the difficulty of processing abnormal data. Moreover, with the continuous expansion of the hydrological monitoring network and the explosive growth of data volume, how to efficiently process and analyze large-scale hydrological data is also an urgent problem to be solved. Summary of the Invention
[0004] The purpose of this application is to provide a forecasting optimization method, system, and medium based on hydrological flood reporting anomaly data processing, which realizes an automated data processing flow, greatly improves the efficiency of hydrological flood reporting, and provides more reliable data support for water resources management and flood and drought control decision-making.
[0005] To achieve the above objectives, this application provides the following technical solution:
[0006] In a first aspect, embodiments of this application provide a forecast optimization method based on hydrological flood reporting anomaly data processing, comprising the following specific steps:
[0007] Step S1: Collect hydrological data from various hydrological stations in the study area, input them into a pre-built threshold judgment model, filter out suspicious data, and construct an initial abnormal dataset.
[0008] Step S2: Construct the XGBoost model, use the Bayesian optimization algorithm to optimize the parameters and hyperparameters of the XGBoost model, and optimize the XGBoost model based on the adaptive learning strategy;
[0009] Step S3: Extract the initial abnormal dataset and input it into the optimized XGBoost model. Solve the model to obtain the abnormal dataset. Based on the abnormal dataset, determine the location and range of each abnormal data point. Collect the normal data points adjacent to each abnormal data point and use interpolation to correct the abnormal data points to obtain the corrected hydrological station hydrological data.
[0010] Step S4: Extract the corrected hydrological data from the hydrological stations and input it into the pre-constructed flood forecasting model to simulate and obtain the process forecast of the flood level within the forecast period.
[0011] Step S1 specifically involves:
[0012] Step S11: Collect hydrological data from various hydrological stations in the study area, including water level, flow rate, and water quality parameters;
[0013] Step S12: Construct a threshold judgment model based on statistical confidence test;
[0014] Step S13: Extract hydrological data from each hydrological monitoring point in the study area and input them into a threshold judgment model based on statistical confidence test. Solve the model to obtain suspicious data and construct an initial abnormal dataset.
[0015] Step S12 specifically involves:
[0016] Step S12a: Clean the hydrological data, use linear interpolation to supplement the missing data in the hydrological data, and use the Yeo-Johnson formula to convert the non-normally distributed data into normally distributed data.
[0017] Step S12b: Transform the hydrological data using the Yeo-Johnson formula to convert the non-negative interval expression, and calculate the mean and standard deviation of the transformed data;
[0018] Step S12c: Filter out the data that is outside the threshold to obtain suspicious data.
[0019] Step S2 is as follows:
[0020] Step S21: Extract hydrological data and divide the hydrological data into training set and test set, with the first 70% being the training set and the last 30% being the validation set;
[0021] Step S22: Construct the XGBoost model and optimize the parameters and hyperparameters of the XGBoost model using the Bayesian optimization algorithm;
[0022] Step S23: Optimize the XGBoost model based on the adaptive learning strategy to obtain the optimized XGBoost model;
[0023] Step S22 further comprises:
[0024] Step S22a: Create an XGBoost model object and set the model parameters and hyperparameters, including: learning rate, number of trees, tree depth, and leaf node weights;
[0025] Step S22b: Input the training set into the XGBoost model for training, and use the Bayesian optimization algorithm to optimize the model parameters and hyperparameters based on the training results and validation set;
[0026] Step S22c: Iterate repeatedly until complete convergence to obtain the optimal parameter combination;
[0027] Step S23 further comprises:
[0028] Step S23a: Extract the optimized XGBoost model from the training set input parameters, and solve the model to obtain the training results;
[0029] Step S23b: Adopt an adaptive learning strategy, dynamically adjust the weights of the training data based on the results of each training session, and set an early stopping mechanism;
[0030] Step S23c: Iterate repeatedly until the performance metrics on the validation set no longer improve, then stop training to obtain the optimized XGBoost model.
[0031] Step S3 specifically involves:
[0032] Step S31: Extract the initial abnormal dataset, fill in the missing data using linear interpolation, and generate normal distributed random values with the same number of samples using Gaussian distribution, which are then combined with the samples to form new sample points;
[0033] Step S32: Calculate the smallest hypersphere for the new sample points using the SVDD model, then filter out the data outside the hypersphere, which are the outliers, and obtain the outlier dataset.
[0034] Step S33: Based on the outlier detection results, determine the location and range of outlier data points in the time series or data matrix, and mark the data that needs to be corrected;
[0035] Step S34: For each abnormal data point, analyze the normal data points that are adjacent to it before and after, and obtain the time interval and value change trend of the adjacent data.
[0036] Step S35: Based on data characteristics and neighborhood analysis results, use interpolation to calculate the reasonable data value that the abnormal data point should have based on the values of adjacent normal data points;
[0037] Step S36: Replace the original abnormal data points with the calculated correction values to complete the correction of abnormal data and obtain the corrected hydrological station hydrological data.
[0038] Step S4 specifically involves:
[0039] Step S41: Construct a flood forecasting model based on the gated cyclic unit model using the stochastic gradient descent method;
[0040] Step S42: Collect measured flood data to train the flood forecasting model and obtain an optimized flood forecasting model;
[0041] Step S43: Extract the corrected hydrological data from the hydrological stations and input it into the optimized flood forecasting model to simulate and obtain the process forecast of the flood level within the forecast period;
[0042] Step S41 further comprises:
[0043] Step S41a: Construct a gated recurrent unit model using a deep learning framework, including an input layer, a GRU layer, and an output layer;
[0044] Step S41b: Select mean squared error as the loss function;
[0045] Step S41c: Use stochastic gradient descent as the optimizer to update the model parameters to minimize the loss function.
[0046] Step S42 further comprises:
[0047] Step S42a: Collect measured flood data and divide it into training set, validation set and test set according to a ratio of 70%-15%-15%;
[0048] Step S42b: Input the training set into the flood forecast model for forward propagation to obtain the prediction results. Input the prediction results and the validation set into the loss function to calculate the loss value. Perform backpropagation and calculate the gradient of the model parameters based on the loss value. Update the model parameters using the optimizer based on the calculated gradient.
[0049] Step S42c: Use the test set to test the flood forecasting model after each parameter update until the model performance reaches the threshold, then stop training to obtain the optimized flood forecasting model. Step S43 further includes:
[0050] Step S43a: Extract the corrected hydrological data from the hydrological stations and input it into the optimized flood forecasting model to simulate and obtain the predicted flood level for the foreseeable period.
[0051] Step S43b: Perform inverse normalization on the predicted values output by the model to obtain the actual flood level values, and organize the prediction results into a time series, that is, the process forecast of the flood level within the forecast period.
[0052] Secondly, embodiments of this application provide a forecast optimization system based on hydrological flood forecasting anomaly data processing. The system includes a memory and a processor. The memory includes a program for a forecast optimization method based on hydrological flood forecasting anomaly data processing. When the program for the forecast optimization method based on hydrological flood forecasting anomaly data processing is executed by the processor, it implements the steps of the forecast optimization method based on hydrological flood forecasting anomaly data processing as described above.
[0053] Thirdly, embodiments of this application provide a computer-readable storage medium storing program code, which, when executed by a processor, implements the steps of the forecast optimization method based on hydrological flood reporting anomaly data processing as described above.
[0054] Compared with the prior art, the beneficial effects of the present invention are:
[0055] A forecasting optimization method based on hydrological flood reporting anomaly data processing is adopted to ensure that the flood reporting data has higher uniqueness and regularity, and to realize comprehensive monitoring and management of data in terms of lateness and anomalies, providing a support tool for improving the quality of hydrological flood reporting data. Attached Figure Description
[0056] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0057] Figure 1 This is a flowchart of the present invention. Detailed Implementation
[0058] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. It should be noted that similar reference numerals and letters in the following drawings indicate similar items; therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.
[0059] The terms “comprising,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0060] The terms “first,” “second,” etc., are used only to distinguish one entity or operation from another, and should not be construed as indicating or implying relative importance, nor as requiring or implying any such actual relationship or order between these entities or operations.
[0061] like Figure 1 As shown in the figure, this invention provides a forecast optimization method based on hydrological flood reporting anomaly data processing, including the following steps:
[0062] Step S1: Collect hydrological data from various hydrological stations in the study area, input them into a pre-built threshold judgment model, filter out suspicious data, and construct an initial abnormal dataset.
[0063] Step S2: Construct the XGBoost model, use the Bayesian optimization algorithm to optimize the parameters and hyperparameters of the XGBoost model, and optimize the XGBoost model based on the adaptive learning strategy;
[0064] Step S3: Extract the initial abnormal dataset and input it into the optimized XGBoost model. Solve the model to obtain the abnormal dataset. Based on the abnormal dataset, determine the location and range of each abnormal data point. Collect the normal data points adjacent to each abnormal data point and use interpolation to correct the abnormal data points to obtain the corrected hydrological station hydrological data.
[0065] Step S4: Extract the corrected hydrological data from the hydrological stations and input it into the pre-constructed flood forecasting model to simulate and obtain the process forecast of the flood level within the forecast period.
[0066] Step S1 further comprises:
[0067] Step S11: Collect hydrological data from various hydrological stations in the study area, including water level, flow rate, and water quality parameters;
[0068] Step S12: Construct a threshold judgment model based on statistical confidence test;
[0069] Step S13: Extract hydrological data from each hydrological monitoring point in the study area and input them into a threshold judgment model based on statistical confidence test. Solve the model to obtain suspicious data and construct an initial abnormal dataset.
[0070] Step S12 further comprises:
[0071] Step S12a: Clean the hydrological data, use linear interpolation to supplement the missing data in the hydrological data, and use the Yeo-Johnson formula to convert the non-normally distributed data into normally distributed data.
[0072] Step S12b: Transform the hydrological data using the Yeo-Johnson formula to convert the non-negative interval expression, and calculate the mean and standard deviation of the transformed data;
[0073] Step S12c: Filter out the data that is outside the threshold to obtain suspicious data.
[0074] In this embodiment, the threshold judgment model is constructed using the 3-δ criterion, also known as the Raida criterion. The Raida criterion generally assumes that the data follows a normal distribution. The mean and standard deviation of the basic distribution can be estimated by calculating the mean and standard deviation of the data. Then, the probability of each object under this distribution can be estimated. Based on the normal distribution probability, the numerical distribution is... The probability is 0.6826, and the values are distributed as follows: The probability is 0.9544, and the values are distributed as follows: The probability is 0.9974. When the data is distributed in... Anything outside of these categories can be identified as an anomaly.
[0075] In one embodiment, specifically:
[0076] Data preprocessing begins with cleaning the data, filling in missing data using linear interpolation, and transforming the data by converting non-normally distributed data into normally distributed data using the Yeo-Johnson formula.
[0077] For model training, since the hydrological data are non-negative, the Yeo-Johnson non-negative interval expression is used for transformation.
[0078] formula
[0079] Calculate the mean and standard deviation of the transformed data. Distribute the data as follows: Filter out the data that is not included;
[0080] The metrics TP indicate that anomalies were correctly detected, FN indicates that anomalies were falsely detected as normal by the algorithm, FP indicates that normal data was falsely detected as anomalies, and TN indicates that normal data was correctly detected. Accuracy. Detection rate False positive rate Comprehensive indicators .
[0081] Step S2 further comprises:
[0082] Step S21: Extract hydrological data and divide the hydrological data into training set and test set, with the first 70% being the training set and the last 30% being the validation set;
[0083] Step S22: Construct the XGBoost model and optimize the parameters and hyperparameters of the XGBoost model using the Bayesian optimization algorithm;
[0084] Step S23: Optimize the XGBoost model based on the adaptive learning strategy to obtain the optimized XGBoost model.
[0085] In this embodiment, the Bayesian optimization algorithm is used to iteratively optimize the parameters and hyperparameters of the XGBoost model. At the same time, an adaptive learning strategy is adopted during training. Bayesian optimization constructs a probabilistic model and intelligently searches for the optimal parameter combination, reducing blind search and lowering computational costs. The adaptive learning strategy dynamically adjusts the data weights according to the training results, making the model pay more attention to difficult-to-detect anomalous samples, improving the learning ability of complex anomalous patterns, and greatly improving the accuracy and efficiency of outlier detection.
[0086] Step S22 further comprises:
[0087] Step S22a: Create an XGBoost model object and set the model parameters and hyperparameters, including: learning rate, number of trees, tree depth, and leaf node weights;
[0088] Step S22b: Input the training set into the XGBoost model for training, and use the Bayesian optimization algorithm to optimize the model parameters and hyperparameters based on the training results and validation set;
[0089] Step S22c: Iterate repeatedly until complete convergence to obtain the optimal parameter combination.
[0090] Bayesian optimization algorithms, based on Bayes' theorem, can intelligently search for the optimal combination of parameters in the parameter space by constructing a probabilistic model of the objective function. Compared with traditional methods such as grid search and random search, Bayesian optimization algorithms can make full use of existing parameter evaluation results and avoid searching in invalid parameter regions, thereby significantly reducing computational costs and search time.
[0091] Step S23 further comprises:
[0092] Step S23a: Extract the optimized XGBoost model from the training set input parameters, and solve the model to obtain the training results;
[0093] Step S23b: Adopt an adaptive learning strategy, dynamically adjust the weights of the training data based on the results of each training session, and set an early stopping mechanism;
[0094] Step S23c: Iterate repeatedly until the performance metrics on the validation set no longer improve, then stop training to obtain the optimized XGBoost model.
[0095] An adaptive learning strategy is adopted during model training. This strategy dynamically adjusts the weights of training data based on the results of each training session. For samples that the model predicts incorrectly, especially those abnormal data samples that are difficult to detect accurately, higher weights are assigned according to the degree of error. This allows the model to pay more attention to these challenging samples in subsequent training and continuously learn complex abnormal patterns, thereby greatly improving the ability to detect various abnormal situations. Through this optimization method, the XGBoost model demonstrates higher accuracy and stability in the task of detecting abnormal data in hydrological flood reporting, and can accurately identify abnormal data points that are easily missed or misjudged by traditional methods.
[0096] In one embodiment, specifically:
[0097] In hydrological monitoring, traditional anomaly detection algorithms based on simple threshold judgment have long been used. When faced with abnormal flow data caused by sudden floods, due to the complex changes in flow during floods, traditional algorithms are unable to accurately capture their characteristics, with a detection accuracy of only 60%. After introducing the optimized XGBoost model of this invention, the model parameters are first fine-tuned using a Bayesian optimization algorithm. After multiple iterations, the learning rate is adjusted from the initial 0.3 to 0.18, which enables the model to converge more smoothly during the learning process and avoids performance problems caused by learning too fast or too slow.
[0098] The tree depth was optimized from 5 to 7, increasing the model's learning levels of data features and enabling it to better uncover complex data patterns. Combined with an adaptive learning strategy, during training, the weights of sudden flood abnormal flow data samples that were misjudged by the model were increased by 2-3 times. After this optimization, the model's detection accuracy for such abnormal data was significantly improved to 85%. In subsequent flood season monitoring, it successfully issued early warnings for multiple flood events, effectively reducing the losses caused by floods.
[0099] Step S3 further comprises:
[0100] Step S31: Extract the initial abnormal dataset, fill in the missing data using linear interpolation, and generate normal distributed random values with the same number of samples using Gaussian distribution, which are then combined with the samples to form new sample points;
[0101] Step S32: Calculate the smallest hypersphere for the new sample points using the SVDD model, then filter out the data outside the hypersphere, which are the outliers, and obtain the outlier dataset.
[0102] Step S33: Based on the outlier detection results, determine the location and range of outlier data points in the time series or data matrix, and mark the data that needs to be corrected;
[0103] Step S34: For each abnormal data point, analyze the normal data points that are adjacent to it before and after, and obtain the time interval and value change trend of the adjacent data.
[0104] Step S35: Based on data characteristics and neighborhood analysis results, use interpolation to calculate the reasonable data value that the abnormal data point should have based on the values of adjacent normal data points;
[0105] Step S36: Replace the original abnormal data points with the calculated correction values to complete the correction of the abnormal data and obtain the corrected hydrological station data. The system of this invention integrates multiple algorithms, including the XGBoost model, the mean-standard difference anomaly detection model, and the neural network anomaly detection algorithm, and has made innovative improvements to model fusion. The XGBoost model, with its powerful feature learning ability and efficient computational performance, can deeply mine complex data patterns; the mean-standard difference anomaly detection model, based on the statistical characteristics of the data, can quickly identify data points that deviate from the normal statistical range; and the neural network anomaly detection algorithm, through learning from a large amount of historical data, constructs a complex nonlinear model, which has good adaptability to various anomaly patterns.
[0106] By employing multi-dimensional cross-detection, different algorithms are first used to independently detect the data. Then, the detection results of each algorithm are combined, which avoids the limitations of a single algorithm. At the same time, an ensemble learning method is used to construct multiple XGBoost sub-models with different initial parameters. Through weighted fusion or voting mechanisms, the detection results of each sub-model are combined to further improve the accuracy and reliability of outlier detection. The complementary advantages of different algorithms enable the system to adapt to complex and ever-changing hydrological environments and effectively improve the ability to process various types of outlier data.
[0107] Step S4 specifically involves:
[0108] Step S41: Construct a flood forecasting model based on the gated cyclic unit model using the stochastic gradient descent method;
[0109] Step S42: Collect measured flood data to train the flood forecasting model and obtain an optimized flood forecasting model;
[0110] Step S43: Extract the corrected hydrological data from the hydrological stations and input it into the optimized flood forecasting model to simulate and obtain the process forecast of the flood level within the forecast period;
[0111] Step S41 further comprises:
[0112] Step S41a: Construct a gated recurrent unit model using a deep learning framework, including an input layer, a GRU layer, and an output layer;
[0113] Step S41b: Select mean squared error as the loss function;
[0114] Step S41c: Use stochastic gradient descent as the optimizer to update the model parameters to minimize the loss function.
[0115] Gated recurrent units (ROUs) are a special type of recurrent neural network (RNN) structure. Traditional RNNs encounter the vanishing or exploding gradient problem when processing long-sequence data, making it difficult to capture long-term dependencies in the data. Gated RNUs solve this problem by introducing two gating mechanisms: an update gate and a reset gate. The update gate determines how much information from the previous hidden state needs to be passed to the current time step, while the reset gate controls how much information from the previous hidden state needs to be ignored. This gating mechanism enables gated RNUs to better process hydrological data with time-series characteristics.
[0116] In this embodiment, a deep learning framework is used to build the GRU model, and the model structure includes the following parts:
[0117] Input layer: Receives the input hydrological data; the number of neurons in the input layer is equal to the number of input features.
[0118] GRU layer: Contains multiple GRU units, which process the input data step by step, learn the temporal dependencies in the data, and the hidden state of the GRU layer is updated as the time step progresses and can be passed to the next time step or subsequent layers.
[0119] Output layer: Maps the output of the GRU layer to the final flood level prediction. The output layer typically has only one neuron because it only needs to predict one flood level.
[0120] Subsequently, mean squared error was chosen as the loss function, which measures the difference between the model's predicted values and the actual values.
[0121] Finally, stochastic gradient descent is used for optimization. Stochastic gradient descent updates the model parameters to minimize the loss function. In each iteration, stochastic gradient descent randomly selects a mini-batch of data from the training data, calculates the gradient of the loss function on this mini-batch, and then updates the model parameters according to the direction of the gradient. Specifically:
[0122] in, These are the model parameters at the t-th iteration. It's the learning rate, which controls the step size for updating parameters. It is the loss function with respect to the parameters The gradient.
[0123] Step S42 further comprises:
[0124] Step S42a: Collect measured flood data and divide it into training set, validation set and test set according to a ratio of 70%-15%-15%;
[0125] Step S42b: Input the training set into the flood forecast model for forward propagation to obtain the prediction results. Input the prediction results and the validation set into the loss function to calculate the loss value. Perform backpropagation and calculate the gradient of the model parameters based on the loss value. Update the model parameters using the optimizer based on the calculated gradient.
[0126] Step S42c: Use the test set to test the flood forecast model after each parameter update until the model performance reaches the threshold, then stop training and obtain the optimized flood forecast model.
[0127] The preprocessed data is divided into a training set, a validation set, and a test set. The training set is used to train the model; the validation set is used to evaluate the model's performance and adjust its hyperparameters during training; and the test set is used to finally evaluate the model's generalization ability.
[0128] In this embodiment, the constructed GRU model is trained using the training set data. In each training cycle, the model iterates through the training set multiple times. In each iteration, a small batch of data is selected for forward and backward propagation. During forward propagation, the model calculates the predicted value based on the input data. During backward propagation, the gradient is calculated based on the loss function, and the model parameters are updated using stochastic gradient descent. Through continuous iterative training, the model parameters are gradually adjusted to the optimal state, causing the value of the loss function to continuously decrease.
[0129] During training, the model's performance is evaluated using data from the validation set. Through continuous adjustments and optimizations, the model achieves good performance on the validation set. Once the model's performance on the validation set is satisfactory, the model is finally evaluated using data from the test set. If the model's performance on the test set also meets the requirements, the model training is considered complete, and an optimized flood forecasting model is obtained.
[0130] Step S43 further comprises:
[0131] Step S43a: Extract the corrected hydrological data from the hydrological stations and input it into the optimized flood forecasting model to simulate and obtain the predicted flood level for the foreseeable period.
[0132] Step S43b: Perform inverse normalization on the predicted values output by the model to obtain the actual flood level values, and organize the prediction results into a time series, that is, the process forecast of the flood level within the forecast period.
[0133] The preprocessed hydrological data is input into the optimized flood forecasting model. The model performs forward propagation calculations based on the input data to obtain the predicted flood level for the forecast period. The predicted value is a time series, reflecting the changes in the flood level at different times within the forecast period. Since the input and output data of the model have been normalized, the prediction results need to be denormalized to restore them to the actual flood level values. Finally, the denormalized prediction results are output in an appropriate format. At the same time, uncertainty analysis can be performed on the prediction results to provide confidence intervals for the predicted values, providing more comprehensive information for flood control decisions.
[0134] This application provides a forecast optimization system based on hydrological flood forecasting anomaly data processing. The system includes a memory and a processor. The memory includes a program for a forecast optimization method based on hydrological flood forecasting anomaly data processing. When the program for the forecast optimization method based on hydrological flood forecasting anomaly data processing is executed by the processor, it implements the steps of the forecast optimization method based on hydrological flood forecasting anomaly data processing as described above.
[0135] This application provides a computer-readable storage medium storing program code. When the program code is executed by a processor, it implements the steps of the forecast optimization method based on hydrological flood reporting anomaly data processing as described above.
[0136] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0137] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0138] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0139] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0140] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0141] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0142] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0143] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A forecast optimization method based on hydrological flood reporting anomaly data processing, characterized in that, The specific steps include the following: Step S1: Collect hydrological data from various hydrological stations in the study area, input them into a pre-built threshold judgment model, filter out suspicious data, and construct an initial abnormal dataset. Step S2: Construct the XGBoost model, use the Bayesian optimization algorithm to optimize the parameters and hyperparameters of the XGBoost model, and optimize the XGBoost model based on the adaptive learning strategy; Step S3: Extract the initial abnormal dataset and input it into the optimized XGBoost model. Solve the model to obtain the abnormal dataset. Based on the abnormal dataset, determine the location and range of each abnormal data point. Collect the normal data points adjacent to each abnormal data point and use interpolation to correct the abnormal data points to obtain the corrected hydrological station hydrological data. Step S4: Extract the corrected hydrological data from the hydrological stations and input it into the pre-constructed flood forecasting model to simulate and obtain the process forecast of the flood level within the forecast period.
2. The forecast optimization method based on hydrological flood reporting anomaly data processing according to claim 1, characterized in that, Step S1 specifically involves: Step S11: Collect hydrological data from various hydrological stations in the study area, including water level, flow rate, and water quality parameters; Step S12: Construct a threshold judgment model based on statistical confidence test; Step S13: Extract hydrological data from each hydrological monitoring point in the study area and input them into a threshold judgment model based on statistical confidence test. Solve the model to obtain suspicious data and construct an initial abnormal dataset.
3. The forecast optimization method based on hydrological flood reporting anomaly data processing according to claim 2, characterized in that, Step S12 specifically involves: Step S12a: Clean the hydrological data, use linear interpolation to supplement the missing data in the hydrological data, and use the Yeo-Johnson formula to convert the non-normally distributed data into normally distributed data. Step S12b: Transform the hydrological data using the Yeo-Johnson formula to convert the non-negative interval expression, and calculate the mean and standard deviation of the transformed data; Step S12c: Filter out the data that is outside the threshold to obtain suspicious data.
4. The forecast optimization method based on hydrological flood reporting anomaly data processing according to claim 1, characterized in that, Step S2 is as follows: Step S21: Extract hydrological data and divide the hydrological data into training set and test set, with the first 70% being the training set and the last 30% being the validation set; Step S22: Construct the XGBoost model and optimize the parameters and hyperparameters of the XGBoost model using the Bayesian optimization algorithm; Step S23: Optimize the XGBoost model based on the adaptive learning strategy to obtain the optimized XGBoost model; Step S22 further comprises: Step S22a: Create an XGBoost model object and set the model parameters and hyperparameters, including: learning rate, number of trees, tree depth, and leaf node weights; Step S22b: Input the training set into the XGBoost model for training, and use the Bayesian optimization algorithm to optimize the model parameters and hyperparameters based on the training results and validation set; Step S22c: Iterate repeatedly until complete convergence to obtain the optimal parameter combination; Step S23 further comprises: Step S23a: Extract the optimized XGBoost model from the training set input parameters, and solve the model to obtain the training results; Step S23b: Adopt an adaptive learning strategy, dynamically adjust the weights of the training data based on the results of each training session, and set an early stopping mechanism; Step S23c: Iterate repeatedly until the performance metrics on the validation set no longer improve, then stop training to obtain the optimized XGBoost model.
5. The forecast optimization method based on hydrological flood reporting anomaly data processing according to claim 1, characterized in that, Step S3 specifically involves: Step S31: Extract the initial abnormal dataset, fill in the missing data using linear interpolation, and generate normal distributed random values with the same number of samples using Gaussian distribution, which are then combined with the samples to form new sample points; Step S32: Calculate the smallest hypersphere for the new sample points using the SVDD model, then filter out the data outside the hypersphere, which are the outliers, and obtain the outlier dataset. Step S33: Based on the outlier detection results, determine the location and range of outlier data points in the time series or data matrix, and mark the data that needs to be corrected; Step S34: For each abnormal data point, analyze the normal data points that are adjacent to it before and after, and obtain the time interval and value change trend of the adjacent data. Step S35: Based on data characteristics and neighborhood analysis results, use interpolation to calculate the reasonable data value that the abnormal data point should have based on the values of adjacent normal data points; Step S36: Replace the original abnormal data points with the calculated correction values to complete the correction of abnormal data and obtain the corrected hydrological station hydrological data.
6. The forecast optimization method based on hydrological flood reporting anomaly data processing according to claim 1, characterized in that, Step S4 specifically involves: Step S41: Construct a flood forecasting model based on the gated cyclic unit model using the stochastic gradient descent method; Step S42: Collect measured flood data to train the flood forecasting model and obtain an optimized flood forecasting model; Step S43: Extract the corrected hydrological data from the hydrological stations and input it into the optimized flood forecasting model to simulate and obtain the process forecast of the flood level within the forecast period; Step S41 further comprises: Step S41a: Construct a gated recurrent unit model using a deep learning framework, including an input layer, a GRU layer, and an output layer; Step S41b: Select mean squared error as the loss function; Step S41c: Use stochastic gradient descent as the optimizer to update the model parameters to minimize the loss function.
7. The forecast optimization method based on hydrological flood reporting anomaly data processing according to claim 6, characterized in that, Step S42 further comprises: Step S42a: Collect measured flood data and divide it into training set, validation set and test set according to a ratio of 70%-15%-15%; Step S42b: Input the training set into the flood forecast model for forward propagation to obtain the prediction results. Input the prediction results and the validation set into the loss function to calculate the loss value. Perform backpropagation and calculate the gradient of the model parameters based on the loss value. Update the model parameters using the optimizer based on the calculated gradient. Step S42c: Use the test set to test the flood forecast model after each parameter update until the model performance reaches the threshold, then stop training and obtain the optimized flood forecast model.
8. The forecast optimization method based on hydrological flood reporting anomaly data processing according to claim 6, characterized in that, Step S43 further comprises: Step S43a: Extract the corrected hydrological data from the hydrological stations and input it into the optimized flood forecasting model to simulate and obtain the predicted flood level for the foreseeable period. Step S43b: Perform inverse normalization on the predicted values output by the model to obtain the actual flood level values, and organize the prediction results into a time series, that is, the process forecast of the flood level within the forecast period.
9. A forecasting optimization system based on hydrological flood reporting anomaly data processing, characterized in that, The system includes a memory and a processor. The memory contains a program for a forecast optimization method based on hydrological flood anomaly data processing. When the processor executes the program for the forecast optimization method based on hydrological flood anomaly data processing, it performs the following steps: Step S1: Collect hydrological data from various hydrological stations in the study area, input them into a pre-built threshold judgment model, filter out suspicious data, and construct an initial abnormal dataset. Step S2: Construct the XGBoost model, use the Bayesian optimization algorithm to optimize the parameters and hyperparameters of the XGBoost model, and optimize the XGBoost model based on the adaptive learning strategy; Step S3: Extract the initial abnormal dataset and input it into the optimized XGBoost model. Solve the model to obtain the abnormal dataset. Based on the abnormal dataset, determine the location and range of each abnormal data point. Collect the normal data points adjacent to each abnormal data point and use interpolation to correct the abnormal data points to obtain the corrected hydrological station hydrological data. Step S4: Extract the corrected hydrological data from the hydrological stations and input it into the pre-constructed flood forecasting model to simulate and obtain the process forecast of the flood level within the forecast period.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, which, when executed by a processor, implements the steps of the forecast optimization method based on hydrological flood reporting anomaly data processing as described in any one of claims 1 to 8.