Water quality monitoring data gradient auditing method and system

Abstract

The application discloses a kind of water quality monitoring data gradient auditing method and system, the method includes: setting the monitoring index of target water quality monitoring point's examination value;Obtain the several monitoring data collected in each monitoring index in each sampling time period;The one-dimensional gradient vector field value corresponding to monitoring data is calculated;If there is the one-dimensional gradient vector field value of the absolute value of the continuous monitoring data including the monitoring data outside the examination tolerance range after smoothing, and greater than gradient limit, then the monitoring data is retained;If the retained monitoring data is within the experimental tolerance range, it is retained, otherwise it is rejected as invalid data.The application judges and eliminates the change of monitoring data caused by abnormal factors in monitoring data, and finally ensures the consistency of the published data and the actual water quality.

Description

Technical Field

[0001] This invention belongs to the field of water quality monitoring technology, and in particular relates to a method and system for gradient verification of water quality monitoring data. Background Technology

[0002] Water is the source of life and the foundation of human survival and development. However, with rapid industrialization and urbanization, water pollution has become increasingly serious, posing a significant threat to human health and the ecological environment. Therefore, water quality monitoring has become an important means of safeguarding water resources. Through real-time monitoring and analysis of water quality, water pollution problems can be detected and resolved in a timely manner, ensuring the safety of drinking water for the people, and also contributing to the protection and improvement of the ecological environment.

[0003] The development of water quality monitoring technology has gone through several stages. Early water quality monitoring relied mainly on manual sampling and simple physicochemical analysis. This method was not only inefficient but also unable to reflect real-time changes in water quality. With technological advancements, automatic monitoring technology has gradually replaced traditional manual monitoring. Automatic monitoring technology can monitor water quality in real time, providing more accurate data and enabling long-term continuous monitoring. In recent years, with the rise of technologies such as the Internet of Things and big data, intelligent water quality monitoring has become a new development trend.

[0004] Current water quality monitoring relies on sensors, offering continuous monitoring compared to traditional manual chemical methods. However, these sensors don't directly sense the monitored indicators (such as total phosphorus and total nitrogen); instead, they interpret electrical or optical signals generated by the water. Therefore, it's possible for other factors to cause a sustained increase in these signals, even if it's not a change in the monitored indicator itself. To avoid discrepancies between the monitoring data and actual conditions, the data acquired by the sensors needs to be reviewed. Summary of the Invention

[0005] In view of this, the present invention provides a gradient review method and system for water quality monitoring data, which uses a gradient review method to remove invalid data from the monitoring data. To solve the above technical problems, the technical solution of the present invention is to adopt a gradient verification method for water quality monitoring data, including: Set the assessment value (TC) of the monitoring indicators for the target water quality monitoring points. m ; Acquire several monitoring data points for each monitoring indicator within each sampling time period; Calculate the one-dimensional gradient vector field value corresponding to the monitoring data; If a smoothed monitoring data point is outside the tolerance range of the assessment, and the absolute value of the one-dimensional gradient vector field value of a continuous monitoring data segment including that monitoring data point is greater than the gradient limit, then the monitoring data point will be retained. If the retained monitoring data is within the experimental tolerance range, it will be retained; otherwise, it will be discarded as invalid data.

[0006] As an improvement, methods for calculating the one-dimensional gradient vector field value corresponding to the monitoring data include: Construct a function y with time sampling points as independent variables and monitoring data as dependent variables. m =f m (t), where y is the monitoring data, m is the monitoring indicator, and t is the time; Using the function y m =f m (t) Obtain the directional derivative value over time; The one-dimensional gradient vector field value is obtained by using the directional derivative value and the time unit value.

[0007] As a further improvement, the method for obtaining the directional derivative value over time is to use the formula: ; Calculate the directional derivative value over time; where, The directional derivative value for time. The time interval for data collection is defined as follows: Vmn is the nth monitoring data of monitoring indicator m, and Vm(n-1) is the (n-1)th monitoring data of monitoring indicator m.

[0008] As another further improvement, methods for obtaining one-dimensional gradient vector field values ​​include using the formula: ; Calculate the field value of the one-dimensional gradient vector; where, Let be the gradient vector field value of the nth monitoring data of monitoring indicator m. denoted by , where j is the directional derivative of time, and j is the unit of time.

[0009] As an improvement, the one-dimensional gradient vector field value of the first monitoring data collected in each sampling time period is itself and is not included in the calculation.

[0010] As an improvement, methods for determining whether smoothed monitoring data falls outside the tolerance range include using the formula: ; Make a judgment; if so, the smoothed monitoring data is outside the tolerance range of the assessment; then, C mi Let t be the i-th monitoring data point for monitoring indicator m, k be the assessment tolerance coefficient, and TC be the tagged data point.m The assessment value of monitoring indicator m; Methods for determining whether retained monitoring data is within the experimental tolerance range include using the following formula: ; Make a judgment; if so, the retained monitoring data is within the experimental tolerance range. Among them, C mi Let be the i-th monitoring data of monitoring indicator m, k' be the experimental tolerance coefficient, and CT be the ith monitoring data. m Experimental data for monitoring index m.

[0011] As an improvement, methods for smoothing monitoring data include: The monitoring data calibrated using concentrated water samples within the calibration period are smoothed; the calibration period is the time interval between two calibrations. The method for smoothing monitoring data within the calibration period is to use the following formula: ; The monitoring data is smoothed; among which, C mt To monitor the smoothed data of indicator m, V mt The monitoring data of the sensor feedback monitoring index m, k mt To monitor the rate of change of indicator m, b mt The change in the monitoring index m is offset.

[0012] As an improvement, the rate of change is obtained using the formula: ; Obtain the rate of change; where k mt To monitor the rate of change of indicator m, k nowm Let m be the calibration slope of the concentrated water sample within the current calibration period, and k be the calibration slope of the monitoring index. lastm The calibration slope of the concentrated water sample in the previous calibration cycle is the monitoring index m, n is the amount of monitoring data of the monitoring index m generated in one calibration cycle, and t is the time interval between the monitoring data of two monitoring indices m. The method for obtaining the calibration slope of concentrated water samples is by using the formula: ; Obtain the calibration slope of the concentrated water sample; where k m To monitor the calibration slope of index m, C m These are the observed values ​​from the original water sample. V represents the observed value of the concentrated water sample. m These are the monitoring values ​​of the original water sample. These are the monitoring values ​​for the raw water sample; The method for obtaining the change offset is by using the formula: ; Obtain the change offset; where b mt To monitor the change and shift of indicator m, b nowm To monitor the calibration offset of concentrated water samples within the current calibration period for indicator m, b lastm The calibration offset of the concentrated water sample in the previous calibration cycle is denoted as m, n is the amount of monitoring data of the monitoring indicator m generated in one calibration cycle, and t is the time interval between the monitoring data of two monitoring indicators m. The method for obtaining the calibration offset of concentrated water samples is by using the formula: ; Obtain the calibration offset of the concentrated water sample; where b m To monitor the calibration offset of index m, k m To monitor the calibration slope of index m, C m V represents the observed value of the original water sample. m These are the monitoring values ​​for the original water sample.

[0013] As an improvement, a concentrated water sample calibration model is constructed using the concentrated water sample calibration slope and concentrated water sample calibration offset to calibrate the monitoring data within the monitoring period; the concentrated water sample calibration model is as follows: y m = k m x m + b m ; Among them, y m For the calibrated monitoring data of monitoring indicator m, x m For monitoring indicator m, the monitoring data is k. m To monitor the calibration slope of index m, b m This is to monitor the calibration offset of index m.

[0014] This invention also provides a water quality monitoring data gradient verification system, comprising: The assessment value setting module is used to set the assessment values ​​for the monitoring indicators of the target water quality monitoring points; The monitoring data acquisition module is used to acquire several monitoring data points collected for each monitoring indicator during each sampling time period; The gradient value calculation module is used to calculate the one-dimensional gradient vector field value corresponding to the monitoring data; The data retention module is used to retain monitoring data when a smoothed monitoring data point is outside the assessment tolerance range, and the absolute value of the one-dimensional gradient vector field value of a continuous monitoring data segment including the monitoring data point is greater than the gradient limit; the assessment tolerance range is determined based on the assessment value. The data rejection module is used to retain monitoring data if the retained data is within the experimental tolerance range; otherwise, it will be rejected as invalid data.

[0015] The advantages of this invention are: In this invention, assessment values ​​are first set for the monitoring indicators of the target water quality monitoring points. Then, monitoring data for each indicator collected by the monitoring equipment during the sampling period are acquired. Based on this monitoring data, a function is constructed with the time sampling point as the independent variable and the monitoring data as the dependent variable. The directional derivative of time is obtained using this function; then, a one-dimensional gradient vector field value is obtained using the directional derivative of time and the time unit value. Finally, the one-dimensional gradient vector field value is used to judge the smoothed monitoring data; invalid data is discarded. This invention judges and removes monitoring data changes caused by abnormal factors, ultimately ensuring the consistency between the published data and the actual water quality. Attached Figure Description

[0016] Figure 1 This is a flowchart of Embodiment 1 of the present invention.

[0017] Figure 2 This is a structural diagram of Embodiment 1 of the present invention.

[0018] Marked in the image: Detailed Implementation To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to specific embodiments.

[0019] Example 1 like Figure 1 As shown, the present invention provides a gradient verification method for water quality monitoring data, including: S1 sets the assessment value of the monitoring indicators for the target water quality monitoring points as TC m .

[0020] Based on the assessment requirements of the management department for the definition of the monitored water section, an assessment value TC is set for each monitoring indicator of this section (monitoring point). m This assessment value is set by engineers based on the actual conditions of the watershed, and is not limited in this invention.

[0021] S2 acquires several monitoring data points for each monitoring indicator during each sampling time period.

[0022] When monitoring equipment collects monitoring data, it can be considered to acquire data continuously, and the time interval between two acquisitions is determined according to the frequency of the acquisition equipment.

[0023] In engineering applications, the density of acquired data does not necessarily need to match the device frequency; therefore, sampling time points are set. The duration between two adjacent sampling time points is the sampling time period. The data acquired in this step is the monitoring data obtained by the monitoring device within the sampling time period. ; V mn This is the nth monitoring data point for monitoring indicator m.

[0024] It is worth noting that the monitoring data in this step has not been corrected.

[0025] S3 calculates the one-dimensional gradient vector field value corresponding to the monitoring data.

[0026] In this embodiment, the method for calculating the one-dimensional gradient vector field value corresponding to the monitoring data includes: S31 constructs a function y with time sampling points as independent variables and monitoring data as dependent variables. m =f m (t), where y is the monitoring data, m is the monitoring indicator, and t is the time.

[0027] For a continuous period of monitoring data, a function can be constructed with time as the independent variable and monitoring data as the dependent variable to describe the relationship between time and the changes in monitoring data. The specific method for constructing this function is not the focus of this invention; existing technologies can be consulted, and it will not be elaborated upon here.

[0028] S32 uses the function y m =f m (t) Obtain the directional derivative value of time.

[0029] Specifically, using the formula: ; Calculate the directional derivative value over time; where, and All are the directional derivative values ​​for time. The time interval for data collection is obtained by engineers based on the sensor collection frequency; ∆y is the increment of the monitoring data, dy is the derivative of the monitoring data, dt is the derivative of time, Vmn is the nth monitoring data of monitoring index m, and Vm(n-1) is the (n-1)th monitoring data of monitoring index m.

[0030] S33 uses the directional derivative value over time and the time unit value to obtain the one-dimensional gradient vector field value.

[0031] Specifically, using the formula: ; Calculate the field value of the one-dimensional gradient vector; where, Let be the gradient vector field value of the nth monitoring data of monitoring indicator m. is the directional derivative value of time, and j is the time unit value, which can be customized according to the specific project.

[0032] For the monitoring data throughout the entire sampling period ; Matrix can be obtained ; It is worth noting that the one-dimensional gradient vector field value of the first monitoring data collected in each sampling time period is itself. = It is not used in calculations and is only used for format consistency.

[0033] S4 If a smoothed monitoring data point is outside the assessment tolerance range, and the absolute value of the one-dimensional gradient vector field value of a continuous monitoring data segment including that monitoring data point is greater than the gradient limit, then the monitoring data point is retained. The assessment tolerance range is determined based on the assessment value.

[0034] In this embodiment, the method for determining whether the smoothed monitoring data is outside the tolerance range includes using the following formula: ; Make a judgment; if so, the smoothed monitoring data is outside the tolerance range of the assessment; then, C mi Let t be the i-th monitoring data point for monitoring indicator m, k be the assessment tolerance coefficient, and TC be the tagged data point. m The assessment value of monitoring indicator m; In other words, if it exists and , If the data is not published, the intelligent sensor station will trigger the automatic sampling device to retain the sample, and personnel will be dispatched to collect and analyze it. If there is no sampling device, maintenance personnel will be dispatched to collect the sample on-site and perform analysis. In this embodiment, the tolerance coefficient k is set to 1.5 by default. The gradient limit can be customized.

[0035] It is worth noting that the monitoring data used for this step is the smoothed monitoring data C. mi For the method of smoothing the monitoring data, please refer to Example 3.

[0036] S5 If the retained monitoring data is within the experimental tolerance range, it will be retained as normal data; otherwise, it will be discarded as invalid data.

[0037] Specifically, using the formula: ; Determine whether the retained monitoring data is within the experimental tolerance range; if so, the retained monitoring data is within the experimental tolerance range; where C mi Let be the i-th monitoring data of monitoring indicator m, and k' be the experimental tolerance coefficient. The default value is 0.5; CT m Experimental data for monitoring index m.

[0038] Example 2 like Figure 2 As shown, the present invention also provides a water quality monitoring data gradient verification system, comprising: The assessment value setting module is used to set the assessment values ​​for the monitoring indicators of the target water quality monitoring points; The monitoring data acquisition module is used to acquire several monitoring data points collected for each monitoring indicator during each sampling time period; The gradient value calculation module is used to calculate the one-dimensional gradient vector field value corresponding to the monitoring data; The data retention module is used to retain monitoring data when a smoothed monitoring data point is outside the assessment tolerance range, and the absolute value of the one-dimensional gradient vector field value of a continuous monitoring data segment including the monitoring data point is greater than the gradient limit; the assessment tolerance range is determined based on the assessment value. The data rejection module is used to retain monitoring data if the retained data is within the experimental tolerance range; otherwise, it will be rejected as invalid data.

[0039] Example 3 This embodiment is used to smooth the monitoring data.

[0040] Water quality in flowing water areas changes over time. To ensure the sensor remains in optimal monitoring condition, it needs to be calibrated periodically using concentrated water samples. For specific calibration methods, please refer to the applicant's invention, "A Background Sample Concentration Device and Concentration Method, and a Calibration Method for Detection Equipment," authorized invention patent number: ZL202311430449.7.

[0041] Concentrated water samples are samples obtained by concentrating the original water sample from the monitored area. After calibration using concentrated water samples, the sensor's monitored values ​​and actual values ​​will be closer, making the monitoring results more accurate.

[0042] S41 The calibration cycle described in this implementation is the time interval between two concentrated water sample calibrations. Specifically, it is achieved through the concentrated water sample calibration model: y m = k m x m + b m ; The monitoring data within the monitoring period are calibrated; where y m For the calibrated monitoring data of monitoring indicator m, x m For monitoring indicator m, the monitoring data is k. m To monitor the calibration slope of index m, b m This is to monitor the calibration offset of index m.

[0043] The concentrated water sample calibration model is constructed using the concentrated water sample calibration slope and the concentrated water sample calibration offset. The concentrated water sample calibration slope is obtained using the formula: ; Obtain the calibration slope of the concentrated water sample; where k m To monitor the calibration slope of index m, C m These are the observed values ​​from the original water sample. V represents the observed value of the concentrated water sample. m These are the monitoring values ​​of the original water sample. These are the monitoring values ​​for the raw water sample. The so-called observed value refers to the value obtained through experiments or other methods; here it can be considered the true value of the monitoring indicator m, and the same applies below.

[0044] The method for obtaining the calibration offset of concentrated water samples is by using the formula: ; Obtain the calibration offset of the concentrated water sample; where b m To monitor the calibration offset of index m, k m To monitor the calibration slope of index m, C m V represents the observed value of the original water sample. m These are the monitoring values ​​for the original water sample.

[0045] After performing concentrated sample calibration, S42 smooths the calibrated monitoring data. In this implementation, the method for smoothing the monitoring data within the calibration period is to use the following formula: ; The monitoring data is smoothed; among which, C mt To monitor the smoothed data of indicator m, V mt The monitoring data of the sensor feedback monitoring index m, k mt To monitor the rate of change of indicator m, b mt The change in the monitoring index m is offset.

[0046] More specifically, the rate of change is obtained using the formula: ; Obtain the rate of change; where k mt To monitor the rate of change of indicator m, knowm Let m be the calibration slope of the concentrated water sample within the current calibration period, and k be the calibration slope of the monitoring index. lastm Let be the calibration slope of the concentrated water sample for monitoring index m within the previous calibration cycle, n be the amount of monitoring data for monitoring index m generated within one calibration cycle, and t be the time interval between the monitoring data of two monitoring indices m. The introduction of the rate of change is mainly to take into account the possibility of water quality changes between two calibration cycles due to the time interval. The rate of change is used to simulate the pattern of water quality changes, thereby improving the accuracy of the monitoring results.

[0047] The method for obtaining the change offset is by using the formula: ; Obtain the change offset; where b mt To monitor the change and shift of indicator m, b nowm To monitor the calibration offset of concentrated water samples within the current calibration period for indicator m, b lastm The calibration offset of the concentrated water sample in the previous calibration cycle is denoted as m, n is the amount of monitoring data of the monitoring indicator m generated in one calibration cycle, and t is the time interval between the monitoring data of two monitoring indicators m.

[0048] The above are merely preferred embodiments of the present invention. It should be noted that the above preferred embodiments should not be considered as limitations on the present invention, and the scope of protection of the present invention should be determined by the scope defined in the claims. For those skilled in the art, several improvements and modifications can be made without departing from the spirit and scope of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for gradient verification of water quality monitoring data, characterized in that... include: Set the assessment values ​​for the monitoring indicators of the target water quality monitoring points; Acquire several monitoring data points for each monitoring indicator within each sampling time period; Calculate the one-dimensional gradient vector field value corresponding to the monitoring data; If a smoothed monitoring data point is outside the tolerance range, and the absolute value of the one-dimensional gradient vector field value of a continuous monitoring data segment including that monitoring data point is greater than the gradient limit, then that monitoring data point will be retained; the tolerance range is determined based on the assessment value. If the retained monitoring data is within the experimental tolerance range, it will be retained; otherwise, it will be discarded as invalid data. Methods for calculating the one-dimensional gradient vector field value corresponding to the monitoring data include: Construct a function y with time sampling points as independent variables and monitoring data as dependent variables. m =f m (t), where y is the monitoring data, m is the monitoring indicator, and t is the time; Using the function y m =f m (t) Obtains the directional derivative value over time; The one-dimensional gradient vector field value is obtained by using the directional derivative value and the time unit value; The method for obtaining the directional derivative value in time is to use the formula: ； Calculate the directional derivative value over time; where, The directional derivative value for time. The time interval for data collection is defined as follows: Vmn is the nth monitoring data of monitoring indicator m, and Vm(n-1) is the (n-1)th monitoring data of monitoring indicator m. Methods for obtaining the field value of a one-dimensional gradient vector include using the formula: ； Calculate the field value of the one-dimensional gradient vector; where, Let m be the one-dimensional gradient vector field value of the nth monitoring data of monitoring indicator m. denoted by , where j is the directional derivative of time, and j is the unit of time.

2. The water quality monitoring data gradient verification method according to claim 1, characterized in that: The one-dimensional gradient vector field value of the first monitoring data collected in each sampling time period is itself and is not included in the calculation.

3. The water quality monitoring data gradient verification method according to claim 1, characterized in that: Methods for determining whether smoothed monitoring data falls outside the tolerance range include using the following formula: ； Make a judgment; if so, the smoothed monitoring data is outside the tolerance range of the assessment; then, C mi Let t be the i-th monitoring data point for monitoring indicator m, k be the assessment tolerance coefficient, and TC be the tagged data point. m The assessment value for monitoring indicator m; Methods for determining whether retained monitoring data is within the experimental tolerance range include using the following formula: ; Make a judgment; if so, the retained monitoring data is within the experimental tolerance range; where C mi Let be the i-th monitoring data of monitoring indicator m, k' be the experimental tolerance coefficient, and CT be the ith monitoring data. m Experimental data for monitoring index m.

4. The water quality monitoring data gradient verification method according to claim 1, characterized in that: Methods for smoothing monitoring data include: The monitoring data calibrated using concentrated water samples within the calibration period are smoothed; the calibration period is the time interval between two calibrations. The method for smoothing monitoring data within the calibration period is to use the following formula: ； The monitoring data is smoothed; among which, C mt To monitor the smoothed data of indicator m, V mt The monitoring data of the sensor feedback monitoring index m, k mt To monitor the rate of change of indicator m, b mt The change in the monitoring index m is offset.

5. The water quality monitoring data gradient verification method according to claim 3, characterized in that: The rate of change is obtained using the formula: ； Obtain the rate of change; where k mt To monitor the rate of change of indicator m, k nowm Let m be the calibration slope of the concentrated water sample within the current calibration period, and k be the calibration slope of the monitoring index. lastm The calibration slope of the concentrated water sample in the previous calibration cycle is the monitoring index m, n is the amount of monitoring data of the monitoring index m generated in one calibration cycle, and t is the time interval between the monitoring data of two monitoring indices m. The method for obtaining the calibration slope of concentrated water samples is by using the formula: ； Obtain the calibration slope of the concentrated water sample; where k m To monitor the calibration slope of index m, C m These are the observed values ​​from the original water sample. V represents the observed value of the concentrated water sample. m These are the monitoring values ​​of the original water sample. These are monitoring values ​​for concentrated water samples; The method for obtaining the change offset is by using the formula: ； Obtain the change offset; where b mt To monitor the change and shift of indicator m, b nowm To monitor the calibration offset of concentrated water samples within the current calibration period for indicator m, b lastm The calibration offset of the concentrated water sample in the previous calibration cycle is the monitoring index m, n is the amount of monitoring data of the monitoring index m generated in one calibration cycle, and t is the time interval between the monitoring data of two monitoring indices m. The method for obtaining the calibration offset of concentrated water samples is by using the formula: ； Obtain the calibration offset of the concentrated water sample; where b m To monitor the calibration offset of index m, k m To monitor the calibration slope of index m, C m V represents the observed value of the original water sample. m These are the monitoring values ​​for the original water sample.

6. The water quality monitoring data gradient verification method according to claim 3, characterized in that: A concentrated water sample calibration model is constructed using the concentrated water sample calibration slope and concentrated water sample calibration offset to calibrate the monitoring data within the monitoring period; the concentrated water sample calibration model is as follows: y m = k m x m + b m ; Among them, y m For the calibrated monitoring data of monitoring indicator m, x m For monitoring indicator m, the monitoring data is k. m To monitor the calibration slope of index m, b m This is to monitor the calibration offset of index m.

7. A water quality monitoring data gradient verification system, used to deploy the water quality monitoring data gradient verification method according to any one of claims 1 to 6, characterized in that... include: The assessment value setting module is used to set the assessment values ​​for the monitoring indicators of the target water quality monitoring points; The monitoring data acquisition module is used to acquire several monitoring data points collected for each monitoring indicator during each sampling time period; The gradient value calculation module is used to calculate the one-dimensional gradient vector field value corresponding to the monitoring data; The data retention module is used to retain monitoring data when a smoothed monitoring data point is outside the assessment tolerance range, and the absolute value of the one-dimensional gradient vector field value of a continuous monitoring data segment including the monitoring data point is greater than the gradient limit; the assessment tolerance range is determined based on the assessment value. The data rejection module is used to retain monitoring data if the retained data is within the experimental tolerance range; otherwise, it will be rejected as invalid data.

Images