A self-diagnosis method for water quality sensors used for surface water monitoring

Abstract

The present invention provides a water quality sensor self-diagnosis method for surface water monitoring, including water quality sensor setting, training data collection, training data analysis, and real-time monitoring implementation. The method is based on the kernel principal component analysis method, and projects the water quality data of historical collection and real-time monitoring into the kernel principal component space for analysis, so as to capture the comprehensive characteristics and overall relationship of the water quality data, effectively distinguish whether the anomaly of real-time monitoring comes from the sensor or the water quality anomaly, and realize self-diagnosis of the water quality sensor during the long-term application, thereby significantly improving the accuracy of water quality monitoring and the operation and maintenance efficiency of the surface water monitoring network, and further comprehensively improving the implementation level of surface water quality monitoring.

Description

Technical Field

[0001] The present invention relates to the field of surface water monitoring, and in particular to a water quality sensor self-diagnosis method for surface water monitoring. Background Art

[0002] As surface water environmental protection is increasingly valued, surface water quality monitoring has become an important part of environmental monitoring. Surface water buoys are widely used in surface water quality monitoring due to their simple structure, flexible layout, and strong adaptability. At present, buoys can realize real-time monitoring of various water quality indicators such as pH value, dissolved oxygen, conductivity, ammonia nitrogen, etc. on demand by carrying various precision water quality sensors. However, in the actual application of buoys, there is still a problem of long-term accuracy of detection data to be solved. Once the buoy is put into use, the water quality sensor on it will start to work for a long time and be in a complex hydrological environment. It will be subjected to water flow impact, water temperature changes, biological attachment, and even human damage for a long time. Degradation factors may cause various abnormalities in the sensor, and then output erroneous data, affecting the accuracy of water quality monitoring and interfering with the judgment of the real water quality situation by relevant departments; in addition, since the surface water buoys are widely deployed and many of them are in remote locations, it is difficult to ensure the accurate operation of all water quality sensors through frequent manual maintenance in actual applications.

[0003] In response to the above problems, the present invention provides a water quality sensor self-diagnosis method for surface water monitoring, which aims to enable the monitoring system to quickly and autonomously detect sensor abnormalities through advanced sensor data analysis algorithms, thereby significantly improving the accuracy of water quality monitoring data, thereby greatly improving the operation and maintenance efficiency of the surface water monitoring network, and comprehensively improving the effective level of surface water quality monitoring. Summary of the invention

[0004] The object of the present invention is to provide a water quality sensor self-diagnosis method for surface water monitoring to solve the above-mentioned problems of the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A water quality sensor self-diagnosis method for surface water monitoring comprises the following steps:

[0007] S101 water quality sensor settings;

[0008] The water quality sensor setting includes setting m water quality monitoring indicators according to water quality monitoring requirements, and correspondingly setting m water quality sensors on the surface water buoy;

[0009] S102 data collection;

[0010] The data collection includes putting the surface water buoy into various surface water environments, collecting water quality data at a certain time interval t, and organizing the collected data into a data set A, A = {a i | i = 1, 2,.., n}, where n is the number of data entries, and each data a i contains measurement values of m water quality monitoring indicators, a i = {a i1 , a i2 ,..., a im};

[0011] The data collection also includes that the data collection period covers all seasons and various weather conditions;

[0012] S103 Data analysis;

[0013] The data analysis includes preprocessing the data set A. The preprocessing includes outlier identification, outlier replacement, missing value filling, and data standardization. The outlier identification uses the moving average method. The outlier replacement and missing value filling both use the spline interpolation method based on the periodic spline type. The data standardization uses the Z - score method;

[0014] The data analysis also includes using the kernel principal component analysis method to analyze the data set A, obtaining k principal components that reflect the main contained features of the data set A, and using the k principal components to construct a principal component space, where m ≤ k < n. The kernel matrix eigenvalues of the kernel principal component analysis corresponding to the k principal components are λ j , and the corresponding eigenvectors are v j , where j = 1~k;

[0015] The data analysis also includes mapping the data set A to the principal component space, calculating the mean value i of the projection value c ji of all data a of the data set A in each principal component direction j, j = 1~k, i = 1~n, and obtaining the center point after the data set A is mapped to the principal component space;

[0016] S104 Real - time monitoring implementation;

[0017] The real - time monitoring implementation includes the following steps:

[0018] a) Put the surface water buoy into the application environment to monitor water quality indicators in real - time. Use the sliding window method to preprocess the real - time collected data in the same way as in S103, and obtain the preprocessed real - time data set B, B = {b i | i = 1, 2,.., p}, where p is the window width;

[0019] b) Mapping the real-time data set B to the principal component space and calculating all data b of the real-time data set B i Projection value d in each principal component direction j ji The mean , j=1~k, i=1~p, get the center point of the real-time data set B after mapping it to the principal component space ; Calculate the center point and the center point obtained by S103 The distance d in each principal component direction j j , and further obtain the distance d between the two in the principal component space, the distance , j=1~k; the distance d between the two in space is calculated using the Euclidean distance, and the expression of d is formula (1):

[0020] (1)

[0021] c) comparing the distance d with an abnormality determination standard. If the abnormality determination standard is reached, it is determined that an abnormality has occurred and the next step of analysis is continued;

[0022] d) Calculate the center point and The distance d in each principal component direction j j The contribution x to the distance d between the two in the principal component space j , j = 1~k; x j The maximum value x in q Compared with the sensor abnormality judgment standard, if the sensor abnormality judgment standard is met, the current abnormality is judged as a sensor abnormality and x is recorded. q The corresponding principal component direction q, continue to the next step of analysis; the center point and The distance d in each principal component direction j The contribution x to the distance d between the two in the principal component space j Use expression (2) to calculate:

[0023] (2)

[0024] e) When it is identified that a sensor abnormality currently occurs, a sensor r associated with the principal component direction q is further identified, wherein 1≤r≤m, then the sensor r is an abnormal sensor, and the result is outputted.

[0025] Furthermore, in S103, the kernel function of the kernel principal component analysis method adopts a radial basis function, and the expression of the radial basis function is formula (3):

[0026] (3)

[0027] Among them, ||a i -a j || means a i and a j The Euclidean distance of , σ is the adjustment coefficient.

[0028] Furthermore, in step S103, the method steps of using kernel principal component analysis to obtain k principal components reflecting the main implied features of the data set A are as follows:

[0029] a) selecting a kernel function and constructing a kernel matrix of the data set A;

[0030] b) Centralizing and eigensolving the kernel matrix to obtain n eigenvalues ​​λ j , the corresponding n eigenvectors are v j And the corresponding n principal components, where j=1~n;

[0031] c) calculating the information contribution of each principal component and sorting them from large to small, wherein the information contribution of each principal component is the ratio of the eigenvalue corresponding to the principal component to the sum of all eigenvalues;

[0032] d) retaining the first k principal components whose cumulative information contribution is greater than or equal to 90%, and using them to construct the principal component space, and the eigenvalues ​​corresponding to the retained k principal components are λ j , the corresponding eigenvector is v j , j = 1~k;

[0033] Furthermore, in the step S104 c), the abnormality determination standard is set by using the standard deviation method. 阈 The expression of is formula (4):

[0034] d 阈 =ηε, (4)

[0035] Where ε is the distance from the center point to the data set A. The standard deviation of the distance, η is the adjustment coefficient, which is related to the water quality of the monitoring area and takes a value of 2~3.

[0036] Furthermore, in step d) of S104, the sensor abnormality determination criteria are composed of expressions (5) and (6), and the expressions (5) and (6) are:

[0037]

[0038] Among them, m is the number of monitoring indicators for each water quality data.

[0039] Furthermore, in the step e) of S104, the method for identifying the sensor r associated with the principal component direction q adopts a principal component load analysis method.

[0040] The beneficial effects of the present invention are as follows:

[0041] The present invention provides a water quality sensor self-diagnosis method for surface water monitoring. By using a kernel principal component analysis method, water quality data collected historically and monitored in real time are projected onto a kernel principal component space for analysis, thereby capturing the comprehensive characteristics and internal structure of the water quality data. By analyzing the data as a whole, it is determined whether the abnormal data comes from sensor abnormalities or water quality abnormalities, thereby distinguishing the method from the existing single water quality sensor data analysis and diagnosis method, avoiding misjudgment of sensor abnormalities caused by abnormal changes in the water quality environment, and greatly improving the accuracy of sensor self-diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flow chart of a water quality sensor self-diagnosis method for surface water monitoring provided by the present invention. DETAILED DESCRIPTION

[0043] The specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments given here are only used to illustrate and explain the present invention, and cannot be used to limit the present invention.

[0044] It should be noted that many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention may also have other implementations and variations thereof. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0045] like Figure 1 As shown, a water quality sensor self-diagnosis method for surface water monitoring includes the following steps:

[0046] S101 water quality sensor settings;

[0047] The water quality sensor setting includes setting m water quality monitoring indicators according to water quality monitoring requirements, and correspondingly setting m water quality sensors on the surface water buoy;

[0048] S102 data collection;

[0049] The data collection includes placing the surface water buoy into various surface water environments, collecting water quality data at a certain time interval t, and collating the collected data into a data set A, A={a i |i=1,2,..,n}, where n is the number of data items, and each data a i Contains m measured values ​​of the water quality monitoring indicators, a i={a i1 , a i2 ,..., a im};

[0050] The data collection also includes that the data collection period covers all seasons and various weather conditions;

[0051] S103 Data analysis;

[0052] The data analysis includes preprocessing the dataset A. The preprocessing includes outlier identification, outlier replacement, missing value filling, and data standardization. The outlier identification uses the moving average method. The outlier replacement and missing value filling both use the spline interpolation method based on the periodic spline type. The data standardization uses the Z-score method;

[0053] The data analysis also includes analyzing the dataset A using the kernel principal component analysis method to obtain k principal components reflecting the main contained features of the dataset A, and constructing a principal component space using the k principal components, where m ≤ k < n. The kernel matrix eigenvalues of the kernel principal component analysis corresponding to the k principal components are λ j , and the corresponding eigenvectors are v j , where j = 1~k;

[0054] The data analysis also includes mapping the dataset A to the principal component space, calculating the mean value i c ji of the projection values of all data a of the dataset A in each principal component direction j, j = 1~k, i = 1~n, to obtain the center point of the dataset A mapped to the principal component space ;

[0055] S104 Real-time monitoring implementation;

[0056] The real-time monitoring implementation includes the following steps:

[0057] a) Deploy the surface water buoy to the application environment to monitor water quality indicators in real time. Use the sliding window method to perform preprocessing on the real-time collected data in the same way as in S103 to obtain the preprocessed real-time dataset B, B = {b i | i = 1, 2,.., p}, where p is the window width;

[0058] b) Map the real-time dataset B to the principal component space, calculate the mean value i d ji of the projection values of all data b of the real-time dataset B in each principal component direction j, j = 1~k, i = 1~p, to obtain the center point of the real-time dataset B mapped to the principal component space ; Calculate the center point and the center point obtained by S103 The distance d in each principal component direction j j , and further obtain the distance d between the two in the principal component space, the distance , j=1~k; the distance d between the two in space is calculated using the Euclidean distance, and the expression of d is formula (1):

[0059] (1)

[0060] c) comparing the distance d with an abnormality determination standard. If the abnormality determination standard is reached, it is determined that an abnormality has occurred and the next step of analysis is continued;

[0061] d) Calculate the center point and The distance d in each principal component direction j j The contribution x to the distance d between the two in the principal component space j , j = 1~k; x j The maximum value x in q Compared with the sensor abnormality judgment standard, if the sensor abnormality judgment standard is met, the current abnormality is judged as a sensor abnormality and x is recorded. q The corresponding principal component direction q, continue to the next step of analysis; the center point and The distance d in each principal component direction j The contribution x to the distance d between the two in the principal component space j Use expression (2) to calculate:

[0062] (2)

[0063] e) When it is identified that a sensor abnormality currently occurs, a sensor r associated with the principal component direction q is further identified, wherein 1≤r≤m, then the sensor r is an abnormal sensor, and the result is outputted.

[0064] Furthermore, in S103, the kernel function of the kernel principal component analysis method adopts a radial basis function, and the expression of the radial basis function is formula (3):

[0065] (3)

[0066] Among them, ||a i -a j || means a i and a j The Euclidean distance of , σ is the adjustment coefficient.

[0067] Furthermore, in step S103, the method steps of using kernel principal component analysis to obtain k principal components reflecting the main implied features of the data set A are as follows:

[0068] a) selecting a kernel function and constructing a kernel matrix of the data set A;

[0069] b) Centralizing and eigensolving the kernel matrix to obtain n eigenvalues ​​λ j , the corresponding n eigenvectors are v j And the corresponding n principal components, where j=1~n;

[0070] c) calculating the information contribution of each principal component and sorting them from large to small, wherein the information contribution of each principal component is the ratio of the eigenvalue corresponding to the principal component to the sum of all eigenvalues;

[0071] d) retaining the first k principal components whose cumulative information contribution is greater than or equal to 90%, and using them to construct the principal component space, and the eigenvalues ​​corresponding to the retained k principal components are λ j , the corresponding eigenvector is v j , j = 1~k;

[0072] Furthermore, in the step S104 c), the abnormality determination standard is set by using the standard deviation method. 阈 The expression of is formula (4):

[0073] d 阈 =ηε, (4)

[0074] Where ε is the distance from the center point to the data set A. The standard deviation of the distance, η is the adjustment coefficient, which is related to the water quality of the monitoring area and takes a value of 2~3.

[0075] Furthermore, in step d) of S104, the sensor abnormality determination criteria are composed of expressions (5) and (6), and the expressions (5) and (6) are:

[0076]

[0077] Among them, m is the number of monitoring indicators for each water quality data.

[0078] Furthermore, in the step e) of S104, the method for identifying the sensor r associated with the principal component direction q adopts a principal component load analysis method.

[0079] Usually, water quality changes are caused by multiple factors, which will cause data changes in multiple principal component directions. When the distance between abnormal data and training data in the principal component space is mainly contributed by a single principal component direction, this reflects that the changes in abnormal data are mainly concentrated in a specific direction. This situation conforms to the data characteristics of a single sensor anomaly. The data has a large deviation in a certain direction of the principal component space. Therefore, by detecting this abnormal distribution, it can be determined whether the sensor anomaly or the water quality anomaly occurs. When the number of monitoring indicators of each water quality data is more, that is, the number of sensors arranged is more, the collected data has a more complex internal relationship, and each principal component direction covers more complex information. Therefore, when judging a single sensor failure, a larger threshold is required to avoid the accumulation of small changes in multiple sensor data in a principal component direction when water quality anomalies occur, resulting in a large abnormal distance in one of the principal component directions, resulting in the misjudgment of water quality anomalies as single sensor anomalies. By increasing the discrimination threshold, it is possible to reliably identify the situation in which a single principal component direction has a significant contribution caused by a single sensor anomaly, thereby improving the accuracy of judgment.

[0080] It should be understood that the above embodiment is only one embodiment of the present invention, and there are many other embodiments and variations thereof based on the present invention; the variations and modifications made by ordinary technicians in this industry through the present invention without making pioneering innovations all fall within the protection scope of the present invention.

Claims

1. A water quality sensor self-diagnosis method for surface water monitoring, characterized in that: The following steps are involved: S101 water quality sensor settings; The water quality sensor setting includes setting m water quality monitoring indicators according to water quality monitoring requirements, and correspondingly setting m water quality sensors on the surface water buoy; S102 data collection; The data collection includes placing the surface water buoy into various surface water environments, collecting water quality data at a certain time interval t, and collating the collected data into a data set A, A={a i |i=1,2,..,n}, where n is the number of data items, and each data a i Contains m measured values ​​of the water quality monitoring indicators, a i ={a i1 ,a i2 ,...,a im }; The data collection also includes data collection periods covering all seasons and all weather conditions; S103 data analysis; The data analysis includes preprocessing the data set A, wherein the preprocessing includes outlier identification, outlier replacement, missing value filling and data standardization, wherein the outlier identification adopts a moving average method, the outlier replacement and missing value filling both adopt a spline interpolation method based on a periodic spline type, and the data standardization adopts a Z-score method; The data analysis further includes analyzing the dataset A by using the kernel principal component analysis method to obtain k principal components reflecting the main contained features of the dataset A, and constructing a principal component space by using the k principal components, where m ≤ k < n, and the kernel matrix eigenvalues of the kernel principal component analysis method corresponding to the k principal components are λ j , and the corresponding eigenvector is v j , where j = 1 to k; The data analysis also includes mapping the data set A to the principal component space, calculating all data a of the data set A i Projection value c in each principal component direction j ji The mean , j=1~k, i=1~n, get the center point of the data set A after mapping it to the principal component space ; S104 Real-time monitoring implementation; The real-time monitoring implementation comprises the following steps: a) placing the surface water buoy in the application environment to monitor water quality indicators in real time, and using the sliding window method to preprocess the real-time collected data in the same way as S103 to obtain a preprocessed real-time data set B, B = {b i |i=1,2,..,p}, where p is the window width; b) Mapping the real-time data set B to the principal component space and calculating all data b of the real-time data set B i Projection value d in each principal component direction j ji The mean , j=1~k, i=1~p, get the center point of the real-time data set B after mapping it to the principal component space ; Calculate the center point and the center point obtained by S103 The distance d in each principal component direction j j , and further obtain the distance d between the two in the principal component space, the distance , j=1~k; the distance d between the two in space is calculated using the Euclidean distance, and the expression of d is formula (1): （1） c) comparing the distance d with an abnormality determination standard. If the abnormality determination standard is reached, it is determined that an abnormality has occurred and the next step of analysis is continued; d) Calculate the center point and The distance d in each principal component direction j j The contribution x to the distance d between the two in the principal component space j , j = 1~k; x j The maximum value x in q Compared with the sensor abnormality judgment standard, if the sensor abnormality judgment standard is met, the current abnormality is judged as a sensor abnormality and x is recorded. q The corresponding principal component direction q, continue to the next step of analysis; the center point and The distance d in each principal component direction j The contribution x to the distance d between the two in the principal component space j Use expression (2) to calculate: （2） e) When it is identified that a sensor abnormality currently occurs, a sensor r associated with the principal component direction q is further identified, wherein 1≤r≤m, then the sensor r is an abnormal sensor, and the result is outputted.

2. A water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In S103, the kernel function of the kernel principal component analysis method adopts a radial basis function, and the expression of the radial basis function is formula (3): （3） Among them, ||a i -a j || means a i and a j The Euclidean distance of , σ is the adjustment coefficient.

3. The water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In step S103, the method steps of using kernel principal component analysis to obtain k principal components reflecting the main implicit features of the data set A are as follows: a) selecting a kernel function and constructing a kernel matrix of the data set A; b) Centralizing and eigensolving the kernel matrix to obtain n eigenvalues ​​λ j , the corresponding n eigenvectors are v j And the corresponding n principal components, where j=1~n; c) calculating the information contribution of each principal component and sorting them from large to small, wherein the information contribution of each principal component is the ratio of the eigenvalue corresponding to the principal component to the sum of all eigenvalues; d) retaining the first k principal components whose cumulative information contribution is greater than or equal to 90%, and using them to construct the principal component space, and the eigenvalues ​​corresponding to the retained k principal components are λ j , the corresponding eigenvector is v j , j=1~k.

4. A water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In the step S104 c), the abnormality determination standard is set by using the standard deviation method. 阈 The expression of is formula (4): d 阈 = ee, (4) Where ε is the distance from the center point to the data set A. The standard deviation of the distance, η is the adjustment coefficient, which is related to the water quality of the monitoring area and takes a value of 2~3.

5. A water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In step S104 d), the sensor abnormality judgment standard is composed of expressions (5) and (6), and the expressions (5) and (6) are: Among them, m is the number of monitoring indicators for each water quality data.

6. A water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In the step e) of S104, the method for identifying the sensor r associated with the principal component direction q adopts the principal component load analysis method.

Images