A hydrological and water resources monitoring method based on big data

Abstract

The present invention discloses a hydrological and water resources monitoring method based on big data, which relates to the technical field of water quality monitoring, including preparation before monitoring, data collection, building a data collection system, water quality assessment, using a water quality index algorithm to assess water quality, obtaining a comprehensive water quality index Q, risk prediction, using an anomaly detection algorithm to obtain an abnormal fluctuation value D, and finally combining the comprehensive water quality index Q and the abnormal fluctuation value D to predict future water quality changes and obtain a future water quality trend value P. t+1 The results show that the water quality comprehensive index Q can integrate different hydrological parameters to comprehensively reflect the overall quality of the water body. Then, the abnormal fluctuation value D is obtained through the anomaly detection algorithm, which can respond to sudden water quality changes in a timely manner. Finally, the future water quality trend value P is obtained by combining the water quality comprehensive index Q and the abnormal fluctuation value D. t+1 , enabling staff to effectively take preventive measures to reduce the effects of harm to water resources.

Description

Technical Field

[0001] The present invention relates to the technical field of water quality monitoring, and in particular to a hydrological and water resources monitoring method based on big data. Background Art

[0002] The hydrological monitoring system is suitable for the hydrological department to conduct real-time monitoring of hydrological parameters such as rivers, lakes, reservoirs, channels and groundwater. With the increasingly serious global water shortage problem, the safe monitoring and management of water resources has become particularly important. Based on the water resources monitoring results, we can understand the self-purification capacity of regional water resources and the demand for water resources in the region, and then make reasonable arrangements in the development and utilization of water resources, providing scientific data support and decision-making basis for decision makers.

[0003] At present, traditional hydrological and water resources monitoring methods, such as the manual "fixed-point instantaneous" sampling method, are unable to obtain dynamic data on changing water quality in a timely and accurate manner. Monitoring is usually carried out based on the thresholds of each set of water quality parameters, which makes it impossible for managers to fully understand the water quality, affecting subsequent decision-making, responding slowly to sudden changes in water quality, and failing to detect anomalies in a timely manner, resulting in the inability to effectively warn of water quality problems. Summary of the invention

[0004] The purpose of the present invention is to provide a hydrological and water resources monitoring method based on big data, which solves the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a hydrological and water resources monitoring method based on big data, comprising the following steps:

[0006] Preparation before monitoring: determine the location and number of monitoring sites according to the monitoring objectives and scope, and install sensor equipment at each monitoring site;

[0007] Data collection: building a data collection system, which includes a sensor network, data transmission equipment and a data center. The hydrological data is collected through the sensor equipment and transmitted to the data center for storage through the data transmission equipment.

[0008] Water quality assessment: using the hydrological data collected by sensor equipment and the water quality index algorithm to evaluate water quality, we can get the comprehensive water quality index Q;

[0009] Risk prediction: use anomaly detection algorithm and combine with water quality comprehensive index Q to evaluate abnormal fluctuation of water body, and get abnormal fluctuation value D. Finally, combine water quality comprehensive index Q and abnormal fluctuation value D to predict future water quality changes, and get future water quality trend value P. t+1 , according to the future water quality trend value P t+1 Achieve preventive effects;

[0010] Result display: organize the monitoring results into a monitoring report and show it to management personnel.

[0011] Optionally, the hydrological data includes water level, flow, temperature, pH value and heavy metal content.

[0012] Optionally, the water quality index algorithm process in the water quality assessment step is as follows:

[0013] ;

[0014] Where Q is the comprehensive water quality index;

[0015] V i is the actual value of the i-th water quality parameter;

[0016] L i is the minimum limit of the ith water quality parameter;

[0017] U i is the upper limit of the ith water quality parameter;

[0018] n is the total number of water quality parameters;

[0019] W i is the weight of the i-th water quality parameter;

[0020] The water quality index algorithm calculates the comprehensive water quality index Q by weighted average, and the influence of each water quality parameter is determined by its W i It is determined that water quality parameters with high weights have a greater impact on the comprehensive water quality index Q;

[0021] During the actual monitoring process, the staff can also adjust the weight of each water quality parameter in real time according to the actual situation. For example, during the flood season, in order to strengthen the monitoring effect of the flood situation, the weight of water quality parameters such as flow rate and water level can be increased to improve the response speed to the flood situation, so that this method can monitor specific situations in special periods.

[0022] Optionally, the anomaly detection algorithm process is as follows:

[0023] ;

[0024] Where D t is the abnormal fluctuation value at time t;

[0025] Q t is the comprehensive water quality index at time t;

[0026] Q avg is the mean of the historical comprehensive water quality index;

[0027] P Q is the average deviation of the historical water quality composite index;

[0028] Q avg With P Q The calculation process is as follows:

[0029] ;

[0030] Where M is the number of historical data points;

[0031] Q e is the comprehensive water quality index at the e-th time point;

[0032] Setting D t The threshold value is Y1, when D t When it is greater than Y1, it indicates that the current water quality comprehensive index is abnormal and triggers an alarm to remind the staff to investigate. At this time, W in the water quality index algorithm is i Will be automatically replaced by XW i , specifically:

[0033] ;

[0034] Among them XW i is the new weight of the i-th water quality parameter;

[0035] W i is the weight of the i-th water quality parameter;

[0036] γ is the adjustment coefficient used to adjust D t XW i The influence of is in the range of 0 to 1;

[0037] D t is the abnormal fluctuation value at time t.

[0038] Optionally, the future water quality trend value P t+1 The calculation process is as follows:

[0039] ;

[0040] Where P t+1 is the future water quality trend value;

[0041] Q t is the comprehensive water quality index at time t;

[0042] D t is the abnormal fluctuation value at time t;

[0043] σQ is the standard deviation of the comprehensive water quality index;

[0044] Q avg is the mean of the historical comprehensive water quality index;

[0045] β is the anomaly detection influence coefficient, ranging from 0 to 1;

[0046] Set the future water quality trend value P t+1 The threshold value is Y2, when P t+1 When it is greater than Y2, it indicates that the water quality may change suddenly in the future, and an alarm will be automatically triggered to remind the staff to take preventive measures in advance to prevent problems before they occur.

[0047] Optionally, the data center in the data collection step includes data processing, and the data processing includes noise processing and missing value processing.

[0048] Optionally, the data transmission device in the data collection step includes a DTU device, and the data center includes a Hadoop big data platform.

[0049] Optionally, in the data collection step, the sensor device is calibrated regularly, the sensor device error is corrected by comparing the standard data with the sensor device, the data collection frequency of the sensor device is set, and the data collection system and the sensor device are maintained regularly.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The present invention evaluates multiple hydrological parameters through a water quality index algorithm to obtain a comprehensive water quality index Q. The comprehensive water quality index Q can integrate each single hydrological parameter so that it can reflect the overall quality of the water body and enable managers to fully understand the water quality. Subsequently, the abnormal fluctuation value D is calculated by combining the comprehensive water quality index Q and historical data through an anomaly detection algorithm, which can monitor hydrological data in real time and respond to sudden water quality changes in a timely manner. Finally, the future water quality changes are predicted by combining the comprehensive water quality index Q and the abnormal fluctuation value D to obtain the future water quality trend value P. t+1 , and by setting the future water quality trend value P t+1 The threshold value is used to achieve the warning effect, and the comprehensive water quality index Q will be dynamically adjusted under the change of abnormal fluctuation value D, which will eventually affect the future water quality trend value P t+1 The numerical value makes the decision-making process more data-based, reduces the subjectivity of decision-making, enhances the capacity of the early warning system, and enables staff to take preventive measures in advance to reduce damage to water resources.

[0052] 2. In the actual monitoring process of the present invention, the staff adjusts the weight of each water quality parameter in real time according to the actual situation. In the summer flood season, in order to strengthen the monitoring effect of the flood situation, the weight of water quality parameters such as flow rate and water level can be increased, and the collection frequency of the sensor can be shortened to improve the response speed to the changes in the flood situation and avoid the expansion of the disaster. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] For examples, see Figure 1 This implementation provides a hydrological and water resources monitoring method based on big data, including the following steps:

[0056] Step S1: Preparation before monitoring: determine the location and number of monitoring sites according to the monitoring objectives and scope, and install sensor equipment at each monitoring site;

[0057] Specifically, according to the hydrological and water resources monitoring objectives and scope, the location and number of monitoring sites should be reasonably planned to ensure the comprehensiveness and representativeness of the monitoring data. For example, when monitoring the flow rate of water, sensors can be installed at different water levels to monitor the flow rate of water at different depths in real time, thereby improving the accuracy of the monitoring data.

[0058] Step S2: Data collection, building a data collection system, which includes a sensor network, data transmission equipment and a data center. The hydrological data is collected through the sensor equipment, and the data is transmitted to the data center for storage through the data transmission equipment;

[0059] The data center uses a combination of cloud platforms and local servers to store data. The combination of cloud platforms and local servers can fully utilize the advantages of both, achieve complementarity in terms of capacity and cost, data security, flexibility and scalability, collaborative work and data sharing, and meet the needs of hydrological and water resources monitoring;

[0060] In the data collection step, the data transmission equipment includes DTU equipment, and the data center includes the Hadoop big data platform;

[0061] Specifically, DTU devices are used when transmitting the collected hydrological data. DTU devices can quickly transmit the collected hydrological data to the data center or cloud platform in real time through the wireless network, ensuring the immediacy and timeliness of the data. It is very suitable for water quality monitoring and early warning systems that require rapid response. In addition, DTU devices support a variety of data acquisition interfaces and protocols, and can be connected with various types of water quality monitoring sensors to achieve data collection in different scenarios, so that a variety of hydrological and water resource data can be monitored simultaneously.

[0062] Hadoop can be integrated with data visualization tools to convert analysis results into intuitive charts and dashboards, helping management to better understand and use big data. Hadoop can dynamically move data between various monitoring sites and ensure the dynamic balance of data at each monitoring site, speeding up data processing. It is suitable for processing large-scale real-time hydrological data, and can automatically save multiple copies of data during data transmission to ensure the continuity and stability of data processing.

[0063] Hydrological data include water level, flow, temperature, pH value and heavy metal content;

[0064] By collecting different hydrological parameters, the water quality comprehensive index Q obtained later can more comprehensively reflect the situation of the monitored waters. During actual monitoring, different hydrological data can be replaced according to specific monitoring objectives, so that each sensor device can be reasonably used to avoid waste of resources and improve the accuracy of monitoring data.

[0065] Step S3: water quality assessment, using the hydrological data collected by the sensor equipment and the water quality index algorithm to assess the water quality and obtain a comprehensive water quality index Q;

[0066] Step S4: Risk prediction: use the anomaly detection algorithm and the water quality comprehensive index Q to evaluate the abnormal fluctuation of the water body, and obtain the abnormal fluctuation value D. Finally, combine the water quality comprehensive index Q and the abnormal fluctuation value D to predict the future water quality changes and obtain the future water quality trend value P. t+1 , according to the future water quality trend value P t+1 Achieve preventive effects;

[0067] Step S5: Result display: organize the monitoring results into a monitoring report and display it to the management personnel.

[0068] By organizing the monitoring results into a monitoring report, managers can quickly and comprehensively understand the water quality and quickly identify problems, thereby speeding up the decision-making process and improving the efficiency of handling water resource issues.

[0069] More specifically, in this embodiment: by installing sensor equipment at each monitoring site and building a data acquisition system, the hydrological and water resource data can be collected and stored in real time, the dynamic data of the constantly changing water quality can be obtained in a timely and accurate manner, and the automation level of hydrological monitoring can be improved. The sensor equipment can collect a variety of hydrological parameters, and then use the water quality index algorithm to evaluate the multiple hydrological parameters to obtain a comprehensive water quality index Q. The comprehensive water quality index Q can integrate each single hydrological parameter into a unified water quality index, so that it can fully reflect the overall quality of the water body and enable management personnel to fully understand the water quality. Subsequently, the abnormal fluctuation value D is calculated by the anomaly detection algorithm and combined with the comprehensive water quality index Q and historical data, so that the hydrological data can be monitored in real time and sudden water quality changes can be responded to in a timely manner. Finally, the future water quality changes are predicted by combining the comprehensive water quality index Q and the abnormal fluctuation value D to obtain the future water quality trend value P. t+1 , and by setting the future water quality trend value P t+1 The threshold is set to achieve the effect of early warning, so that staff can effectively take preventive measures to reduce the harm to water resources.

[0070] Furthermore, the water quality index algorithm process in the water quality assessment step is as follows:

[0071] ;

[0072] Where Q is the comprehensive water quality index;

[0073] V i is the actual value of the i-th water quality parameter;

[0074] L i is the minimum limit of the ith water quality parameter;

[0075] U i is the upper limit of the ith water quality parameter;

[0076] n is the total number of water quality parameters;

[0077] W i is the weight of the i-th water quality parameter;

[0078] The water quality index algorithm calculates the comprehensive water quality index Q by weighted average, and the influence of each water quality parameter is determined by its W i It is determined that water quality parameters with higher weights have a greater impact on the comprehensive water quality index Q.

[0079] Specifically, in the actual monitoring process, the staff can also adjust the weight of each water quality parameter in real time according to the actual situation. For example, in heavy rain, in order to strengthen the monitoring effect of flood conditions, the weights of water quality parameters such as flow rate and water level can be increased to improve the response speed to flood conditions, so that this method can monitor specific situations in special periods. Under normal circumstances, by integrating multiple water quality parameters, the comprehensive water quality index Q can reflect the comprehensive situation of the monitored water area and display it quantitatively, so that the staff can easily understand the water quality status and improve the accuracy of subsequent decision-making.

[0080] Furthermore, the anomaly detection algorithm process is as follows:

[0081] ;

[0082] Where D t is the abnormal fluctuation value at time t;

[0083] Q t is the comprehensive water quality index at time t;

[0084] Q avg is the mean of the historical comprehensive water quality index;

[0085] P Q is the average deviation of the historical water quality composite index;

[0086] Q avg With P Q The calculation process is as follows:

[0087] ;

[0088] Where M is the number of historical data points;

[0089] Q e is the comprehensive water quality index at the e-th time point;

[0090] Setting D t The threshold value is Y1, when D t When it is greater than Y1, it indicates that the current water quality comprehensive index is abnormal and triggers an alarm to remind the staff to investigate. At this time, W in the water quality index algorithm is i Will be automatically replaced by XW i , specifically:

[0091] ;

[0092] Among them XW i is the new weight of the i-th water quality parameter;

[0093] W i is the weight of the i-th water quality parameter;

[0094] γ is the adjustment coefficient used to adjust D t XW i The influence of is in the range of 0 to 1;

[0095] D t is the abnormal fluctuation value at time t.

[0096] Specifically, the anomaly detection algorithm detects abnormal fluctuations by comparing the difference between the comprehensive water quality index Q and historical data, and introduces the mean value Q of the historical comprehensive water quality index. avg The average deviation P from the historical water quality comprehensive index Q , can sensitively identify abnormal fluctuations in water quality data, so that the anomaly detection algorithm can adapt to different water quality conditions and changing trends, and by setting D t The threshold is Y1. When the abnormal fluctuation value is greater than the set range, it can automatically alarm and respond to sudden changes in water quality in a timely manner. It also realizes automated monitoring, reduces dependence on manual intervention, and reduces errors.

[0097] And through D t To affect W in the water quality index algorithm i , which in turn affects the comprehensive water quality index Q, achieving the effect of dynamically adjusting the weights of water quality parameters. When there are sudden changes in water quality, the anomaly detection algorithm responds more obviously to sudden changes in water quality, thereby improving the sensitivity, accuracy and responsiveness of hydrological and water resources monitoring.

[0098] Furthermore, the future water quality trend value P t+1 The calculation process is as follows:

[0099] ;

[0100] Where P t+1 is the future water quality trend value;

[0101] Q t is the comprehensive water quality index at time t;

[0102] D t is the abnormal fluctuation value at time t;

[0103] σQ is the standard deviation of the comprehensive water quality index;

[0104] Q avg is the mean of the historical comprehensive water quality index;

[0105] β is the anomaly detection influence coefficient, ranging from 0 to 1;

[0106] Set the future water quality trend value P t+1 The threshold value is Y2, when P t+1When it is greater than Y2, it indicates that the water quality may change suddenly in the future. The change includes excessive change in water flow rate or water quality, which will automatically trigger an alarm to remind staff to take preventive measures in advance to prevent the water quality problem from worsening.

[0107] Specifically, by combining the comprehensive water quality index Q and the abnormal fluctuation value D to predict future water quality changes, the future water quality trend value P is obtained. t+1 , indicating the future water quality, and thus calculating the future water quality trend value P t+1 It can predict the future water quality. By predicting water quality changes, it can more effectively allocate monitoring and governance resources and reduce resource waste. The comprehensive water quality index Q will be dynamically adjusted under the change of abnormal fluctuation value D, which will eventually affect the future water quality trend value P. t+1 The numerical value makes the decision-making process more data-based, reduces the subjectivity of decision-making, enhances the capacity of the early warning system, enables water quality problems to be dealt with before their impact expands, and reduces damage to water resources.

[0108] Furthermore, the data center in the data collection step includes data processing, and the data processing includes noise processing and missing value processing.

[0109] Specifically, noise processing includes filtering and denoising. Filtering methods include limiting filtering, median filtering, mean filtering and recursive average filtering, which can smooth the signal, suppress noise, reduce random fluctuations and outliers in the data, and make the collected hydrological data higher in quality, providing clearer signals for subsequent analysis, and retaining important signal features such as trends and periodic changes. It is suitable for long-term monitoring of hydrology and water resources. Missing value processing can ensure the comprehensiveness of data analysis and will not miss important information or trends due to incomplete data.

[0110] Furthermore, in the data collection step, the sensor equipment needs to be calibrated regularly, and the sensor equipment error needs to be corrected by comparing the standard data with the sensor equipment. The data collection frequency of the sensor equipment needs to be set, and the data collection system and sensor equipment need to be maintained regularly to ensure the sensitivity of the sensor equipment and the quality of hydrological data collection.

[0111] Specifically, when setting the data collection frequency of sensor equipment, it can be decided according to the specific situation. For example, during the summer flood season, the collection frequency of specific sensors can be shortened to improve the response speed to changes in flood conditions and avoid the expansion of disasters.

[0112] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A hydrological and water resources monitoring method based on big data, characterized in that: The following steps are involved: Step S1: Preparation before monitoring: determine the location and number of monitoring sites according to the monitoring objectives and scope, and install sensor equipment at each monitoring site; Step S2: Data collection, building a data collection system, which includes a sensor network, data transmission equipment and a data center. The hydrological data is collected through the sensor equipment, and the data is transmitted to the data center for storage through the data transmission equipment; Step S3: water quality assessment, using the hydrological data collected by the sensor equipment and the water quality index algorithm to assess the water quality and obtain a comprehensive water quality index Q; Step S4: Risk prediction: use the anomaly detection algorithm and the water quality comprehensive index Q to evaluate the abnormal fluctuation of the water body, and obtain the abnormal fluctuation value D. Finally, combine the water quality comprehensive index Q and the abnormal fluctuation value D to predict the future water quality changes and obtain the future water quality trend value P. t+1 , according to the future water quality trend value P t+1 Achieve preventive effects; Step S5: Result display, compiling the monitoring results into a monitoring report and displaying it to the management personnel; The hydrological data include water level, flow, temperature, pH value and heavy metal content; The water quality index algorithm process in the water quality assessment step is as follows: ； Where Q is the comprehensive water quality index; V i is the actual value of the i-th water quality parameter; L i is the minimum limit of the ith water quality parameter; U i is the upper limit of the ith water quality parameter; n is the total number of water quality parameters; W i is the weight of the i-th water quality parameter; The water quality index algorithm calculates the comprehensive water quality index Q by weighted average, and the influence of each water quality parameter is determined by its W i It is determined that water quality parameters with high weights have a greater impact on the comprehensive water quality index Q; The anomaly detection algorithm process is as follows: ； Where D t is the abnormal fluctuation value at time t; Q t is the comprehensive water quality index at time t; Q avg is the mean of the historical comprehensive water quality index; P Q is the average deviation of the historical water quality composite index; Q avg With P Q The calculation process is as follows: ； Where M is the number of historical data points; Q e is the comprehensive water quality index at the e-th time point; Setting D t The threshold value is Y1, when D t When it is greater than Y1, it indicates that the current water quality comprehensive index is abnormal and triggers an alarm to remind staff to investigate. At this time, W in the water quality index algorithm i Will be automatically replaced by XW i , specifically: ； Among them XW i is the new weight of the i-th water quality parameter; W i is the weight of the i-th water quality parameter; γ is the adjustment coefficient used to adjust D t XW i The influence of is in the range of 0 to 1; D t is the abnormal fluctuation value at time t.

2. The hydrological and water resources monitoring method based on big data according to claim 1 is characterized in that: The future water quality trend value P t+1 The calculation process is as follows: ； Where P t+1 is the future water quality trend value; Q t is the comprehensive water quality index at time t; D t is the abnormal fluctuation value at time t; σQ is the standard deviation of the comprehensive water quality index; Q avg is the mean of the historical comprehensive water quality index; β is the anomaly detection influence coefficient, ranging from 0 to 1; Set the future water quality trend value P t+1 The threshold value is Y2, when P t+1 When it is greater than Y2, it indicates that the water quality may change suddenly in the future, and an alarm will be automatically triggered to remind the staff to take preventive measures in advance.

3. The hydrological and water resources monitoring method based on big data according to claim 1 is characterized by: The data center in the data collection step includes data processing, and the data processing includes noise processing and missing value processing.

4. The hydrological and water resources monitoring method based on big data according to claim 1 is characterized by: The data transmission device in the data collection step includes a DTU device, and the data center includes a Hadoop big data platform.

5. The hydrological and water resources monitoring method based on big data according to claim 1 is characterized by: In the data collection step, the sensor device is calibrated regularly, the sensor device error is corrected by comparing the standard data with the sensor device, the data collection frequency of the sensor device is set, and the data collection system and the sensor device are maintained regularly.

Images