Water quality sensor self-diagnosis method for surface water monitoring

Abstract

The invention provides a water quality sensor self-diagnosis method for surface water monitoring, which comprises the steps of water quality sensor setting, training data acquisition, training data analysis and real-time monitoring implementation, and is characterized in that based on a kernel principal component analysis method, water quality data acquired historically and monitored in real time is projected to a kernel principal component space for analysis; the comprehensive characteristics and the overall relation of water quality data are captured, whether the real-time monitoring abnormity is from the sensor or the water quality abnormity is effectively distinguished, self-diagnosis of the water quality sensor in the long-term application process is achieved, and therefore the water quality monitoring accuracy and the operation and maintenance efficiency of a surface water monitoring network are remarkably improved; and thus, the implementation level of surface water quality monitoring is comprehensively improved.

Description

Technical Field

[0001] The present invention relates to the field of surface water monitoring, and in particular to a water quality sensor self-diagnosis method for surface water monitoring. Background Art

[0002] As surface water environmental protection is increasingly valued, surface water quality monitoring has become an important part of environmental monitoring. Surface water buoys are widely used in surface water quality monitoring due to their simple structure, flexible layout, and strong adaptability. At present, buoys can realize real-time monitoring of various water quality indicators such as pH value, dissolved oxygen, conductivity, ammonia nitrogen, etc. on demand by carrying various precision water quality sensors. However, in the actual application of buoys, there is still a problem of long-term accuracy of detection data to be solved. Once the buoy is put into use, the water quality sensor on it will start to work for a long time and be in a complex hydrological environment. It will be subjected to water flow impact, water temperature changes, biological attachment, and even human damage for a long time. Degradation factors may cause various abnormalities in the sensor, and then output erroneous data, affecting the accuracy of water quality monitoring and interfering with the judgment of the real water quality situation by relevant departments; in addition, since the surface water buoys are widely deployed and many of them are in remote locations, it is difficult to ensure the accurate operation of all water quality sensors through frequent manual maintenance in actual applications.

[0003] In response to the above problems, the present invention provides a water quality sensor self-diagnosis method for surface water monitoring, which aims to enable the monitoring system to quickly and autonomously detect sensor abnormalities through advanced sensor data analysis algorithms, thereby significantly improving the accuracy of water quality monitoring data, thereby greatly improving the operation and maintenance efficiency of the surface water monitoring network, and comprehensively improving the effective level of surface water quality monitoring. Summary of the invention

[0004] The object of the present invention is to provide a water quality sensor self-diagnosis method for surface water monitoring to solve the above-mentioned problems of the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions: A water quality sensor self-diagnosis method for surface water monitoring comprises the following steps: S101 water quality sensor settings; The water quality sensor setting includes setting m water quality monitoring indicators according to water quality monitoring requirements, and correspondingly setting m water quality sensors on the surface water buoy; S102 data collection; The data collection includes placing the surface water buoy into various surface water environments, collecting water quality data at a certain time interval t, and collating the collected data into a data set A, A={a i|i = 1, 2,.., n}, where n is the number of data entries, and each data a i contains the measured values of m of the water quality monitoring indicators, a i = {a i1 , a i2 ,..., a im}; The data collection also includes that the data collection period covers all seasons and various weather conditions; S103 Data analysis; The data analysis includes preprocessing the dataset A. The preprocessing includes outlier identification, outlier replacement, missing value imputation, and data standardization. The outlier identification uses the moving average method, the outlier replacement and missing value imputation both use the spline interpolation method based on the periodic spline type, and the data standardization uses the Z - score method; The data analysis also includes analyzing the dataset A using the kernel principal component analysis method to obtain k principal components that reflect the main contained features of the dataset A, and using the k principal components to construct a principal component space, where m ≤ k < n, and the kernel matrix eigenvalues of the kernel principal component analysis corresponding to the k principal components are λ j , and the corresponding eigenvectors are v j , where j = 1~k; The data analysis also includes mapping the dataset A to the principal component space, calculating the mean value i of the projection values c ji of all data a of the dataset A in each principal component direction j, j = 1~k, i = 1~n, to obtain the center point after the dataset A is mapped to the principal component space ; S104 Real - time monitoring implementation; The real - time monitoring implementation includes the following steps: a) Deploy the surface water buoy into the application environment to monitor water quality indicators in real - time. Use the sliding window method to perform preprocessing on the real - time collected data using the same method as in S103 to obtain the preprocessed real - time dataset B, B = {b i |i = 1, 2,.., p}, where p is the window width; b) Map the real - time dataset B to the principal component space, calculate the mean value i of the projection values d ji of all data b of the real - time dataset B in each principal component direction j, j = 1~k, i = 1~p, to obtain the center point after the real - time dataset B is mapped to the principal component space ; Calculate the center point and the center point obtained in S103 The distance d in each principal component direction j j , and further obtain the distance d between the two in the principal component space, the distance , j = 1~k; the distance d between the two in space is calculated using the Euclidean distance, and the expression of d is formula (1): (1) c) comparing the distance d with an abnormality determination standard. If the abnormality determination standard is reached, it is determined that an abnormality has occurred and the next step of analysis is continued; d) Calculate the center point and The distance d in each principal component direction j j The contribution x to the distance d between the two in the principal component space j , j = 1~k; x j The maximum value x in q Compared with the sensor abnormality judgment standard, if the sensor abnormality judgment standard is met, the current abnormality is judged as a sensor abnormality and x is recorded. q The corresponding principal component direction q, continue to the next step of analysis; the center point and The distance d in each principal component direction j The contribution x to the distance d between the two in the principal component space j Use expression (2) to calculate: (2) e) When it is identified that a sensor abnormality currently occurs, a sensor r associated with the principal component direction q is further identified, wherein 1≤r≤m, then the sensor r is an abnormal sensor, and the result is output.

[0006] Furthermore, in S103, the kernel function of the kernel principal component analysis method adopts a radial basis function, and the expression of the radial basis function is formula (3): (3) Among them, ||a i -a j || means a i and a j The Euclidean distance of , σ is the adjustment coefficient.

[0007] Furthermore, in step S103, the method steps of using kernel principal component analysis to obtain k principal components reflecting the main implied features of the data set A are as follows: a) selecting a kernel function and constructing a kernel matrix of the data set A; b) Centralizing and eigensolving the kernel matrix to obtain n eigenvalues ​​λj , the corresponding n eigenvectors are v j And the corresponding n principal components, where j=1~n; c) calculating the information contribution of each principal component and sorting them from large to small, wherein the information contribution of each principal component is the ratio of the eigenvalue corresponding to the principal component to the sum of all eigenvalues; d) retaining the first k principal components whose cumulative information contribution is greater than or equal to 90%, and using them to construct the principal component space, and the eigenvalues ​​corresponding to the retained k principal components are λ j , the corresponding eigenvector is v j , j = 1~k; Furthermore, in the step S104 c), the abnormality determination standard is set by using the standard deviation method. 阈 The expression of is formula (4): d 阈 =ηε, (4) Where ε is the distance from the center point to the data set A. The standard deviation of the distance, η is the adjustment coefficient, which is related to the water quality of the monitoring area and takes a value of 2~3.

[0008] Furthermore, in step d) of S104, the sensor abnormality determination criteria are composed of expressions (5) and (6), and the expressions (5) and (6) are:

[0009] Among them, m is the number of monitoring indicators for each water quality data.

[0010] Furthermore, in the step e) of S104, the method for identifying the sensor r associated with the principal component direction q adopts a principal component load analysis method.

[0011] The beneficial effects of the present invention are as follows: The present invention provides a water quality sensor self-diagnosis method for surface water monitoring. By using a kernel principal component analysis method, water quality data collected historically and monitored in real time are projected onto a kernel principal component space for analysis, thereby capturing the comprehensive characteristics and internal structure of the water quality data. By analyzing the data as a whole, it is determined whether the abnormal data comes from sensor abnormalities or water quality abnormalities, thereby distinguishing the method from the existing single water quality sensor data analysis and diagnosis method, avoiding misjudgment of sensor abnormalities caused by abnormal changes in the water quality environment, and greatly improving the accuracy of sensor self-diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A flow chart of a water quality sensor self-diagnosis method for surface water monitoring provided by the present invention. DETAILED DESCRIPTION

[0013] The specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments given here are only for illustrating and explaining the present invention and cannot be used to limit the present invention.

[0014] It should be noted that many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention may have other embodiments and variations, and thus the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0015] As Figure 1 shown, a water quality sensor self-diagnosis method for surface water monitoring includes the following steps: S101 Water quality sensor setting; The water quality sensor setting includes setting m water quality monitoring indicators according to water quality monitoring requirements and correspondingly setting m water quality sensors on the surface water buoy. S102 Data collection; The data collection includes putting the surface water buoy into various surface water environments, collecting water quality data at a certain time interval t, and organizing the collected data into a data set A, A = {a i |i = 1, 2,.., n}, where n is the number of data entries, and each piece of data a i contains the measured values of m water quality monitoring indicators, a i = {a i1 , a i2 ,..., a im}; The data collection also includes that the data collection period covers all seasons and various weather conditions; S103 Data analysis; The data analysis includes preprocessing the data set A. The preprocessing includes outlier identification, outlier replacement, missing value filling, and data standardization. The outlier identification uses the moving average method, the outlier replacement and missing value filling both use the spline interpolation method based on the periodic spline type, and the data standardization uses the Z-score method; The data analysis also includes analyzing the data set A using the kernel principal component analysis method to obtain k principal components reflecting the main characteristics contained in the data set A, and using the k principal components to construct a principal component space, where m ≤ k < n, and the kernel matrix eigenvalues of the kernel principal component analysis corresponding to the k principal components are λ j , and the corresponding eigenvectors are v j , where j = 1~k; The data analysis also includes mapping the data set A to the principal component space, calculating all data a of the data set A i Projection value c in each principal component direction j ji The mean , j=1~k, i=1~n, get the center point of the data set A after mapping it to the principal component space ; S104 Real-time monitoring implementation; The real-time monitoring implementation comprises the following steps: a) placing the surface water buoy in the application environment to monitor water quality indicators in real time, and using the sliding window method to preprocess the real-time collected data in the same way as S103 to obtain a preprocessed real-time data set B, B = {b i |i=1,2,..,p}, where p is the window width; b) Mapping the real-time data set B to the principal component space and calculating all data b of the real-time data set B i Projection value d in each principal component direction j ji The mean , j=1~k, i=1~p, get the center point of the real-time data set B after mapping it to the principal component space ; Calculate the center point and the center point obtained by S103 The distance d in each principal component direction j j , and further obtain the distance d between the two in the principal component space, the distance , j = 1~k; the distance d between the two in space is calculated using the Euclidean distance, and the expression of d is formula (1): (1) c) comparing the distance d with an abnormality determination standard. If the abnormality determination standard is reached, it is determined that an abnormality has occurred and the next step of analysis is continued; d) Calculate the center point and The distance d in each principal component direction j j The contribution x to the distance d between the two in the principal component space j , j = 1~k; x j The maximum value x in q Compared with the sensor abnormality judgment standard, if the sensor abnormality judgment standard is met, the current abnormality is judged as a sensor abnormality and x is recorded. q The corresponding principal component direction q, continue to the next step of analysis; the center point and The distance d in each principal component direction jThe contribution x to the distance d between the two in the principal component space j Use expression (2) to calculate: (2) e) When it is identified that a sensor abnormality currently occurs, a sensor r associated with the principal component direction q is further identified, wherein 1≤r≤m, then the sensor r is an abnormal sensor, and the result is outputted.

[0016] Furthermore, in S103, the kernel function of the kernel principal component analysis method adopts a radial basis function, and the expression of the radial basis function is formula (3): (3) Among them, ||a i -a j || means a i and a j The Euclidean distance of , σ is the adjustment coefficient.

[0017] Furthermore, in step S103, the method steps of using kernel principal component analysis to obtain k principal components reflecting the main implied features of the data set A are as follows: a) selecting a kernel function and constructing a kernel matrix of the data set A; b) Centralizing and eigensolving the kernel matrix to obtain n eigenvalues ​​λ j , the corresponding n eigenvectors are v j And the corresponding n principal components, where j=1~n; c) calculating the information contribution of each principal component and sorting them from large to small, wherein the information contribution of each principal component is the ratio of the eigenvalue corresponding to the principal component to the sum of all eigenvalues; d) retaining the first k principal components whose cumulative information contribution is greater than or equal to 90%, and using them to construct the principal component space, and the eigenvalues ​​corresponding to the retained k principal components are λ j , the corresponding eigenvector is v j , j = 1~k; Furthermore, in the step S104 c), the abnormality determination standard is set by using the standard deviation method. 阈 The expression of is formula (4): d 阈 =ηε, (4) Where ε is the distance from the center point to the data set A. The standard deviation of the distance, η is the adjustment coefficient, which is related to the water quality of the monitoring area and takes a value of 2~3.

[0018] Furthermore, in step d) of S104, the sensor abnormality determination criteria are composed of expressions (5) and (6), and the expressions (5) and (6) are: Among them, m is the number of monitoring indicators for each water quality data.

[0019] Furthermore, in the step e) of S104, the method for identifying the sensor r associated with the principal component direction q adopts a principal component load analysis method.

[0020] Usually, water quality changes are caused by multiple factors, which will cause data changes in multiple principal component directions. When the distance between abnormal data and training data in the principal component space is mainly contributed by a single principal component direction, this reflects that the changes in abnormal data are mainly concentrated in a specific direction. This situation conforms to the data characteristics of a single sensor anomaly. The data has a large deviation in a certain direction of the principal component space. Therefore, by detecting this abnormal distribution, it can be determined whether the sensor anomaly or the water quality anomaly occurs. When the number of monitoring indicators of each water quality data is more, that is, the number of sensors arranged is more, the collected data has a more complex internal relationship, and each principal component direction covers more complex information. Therefore, when judging a single sensor failure, a larger threshold is required to avoid the accumulation of small changes in multiple sensor data in a principal component direction when water quality anomalies occur, resulting in a large abnormal distance in one of the principal component directions, resulting in the misjudgment of water quality anomalies as single sensor anomalies. By increasing the discrimination threshold, it is possible to reliably identify the situation in which a single principal component direction has a significant contribution caused by a single sensor anomaly, thereby improving the accuracy of judgment.

[0021] It should be understood that the above embodiment is only one embodiment of the present invention, and there are many other embodiments and variations thereof based on the present invention; the variations and modifications made by ordinary technicians in this industry through the present invention without making pioneering innovations all fall within the scope of protection of the present invention.

Claims

1. A water quality sensor self-diagnosis method for surface water monitoring, characterized in that: The following steps are involved: S101 water quality sensor settings; The water quality sensor setting includes setting m water quality monitoring indicators according to water quality monitoring requirements, and correspondingly setting m water quality sensors on the surface water buoy; S102 data collection; The data collection includes placing the surface water buoy into various surface water environments, collecting water quality data at a certain time interval t, and collating the collected data into a data set A, A={a i |i=1,2,..,n}, where n is the number of data items, and each data a i Contains m measured values ​​of the water quality monitoring indicators, a i ={a i1 ,a i2 ,...,a im }; The data collection also includes data collection periods covering all seasons and all weather conditions; S103 data analysis; The data analysis includes preprocessing the data set A, wherein the preprocessing includes outlier identification, outlier replacement, missing value filling and data standardization, wherein the outlier identification adopts a moving average method, the outlier replacement and missing value filling both adopt a spline interpolation method based on a periodic spline type, and the data standardization adopts a Z-score method; The data analysis further includes analyzing the dataset A by using the kernel principal component analysis method to obtain k principal components reflecting the main contained features of the dataset A, and constructing a principal component space by using the k principal components, where m ≤ k < n, and the kernel matrix eigenvalues of the kernel principal component analysis method corresponding to the k principal components are λ j , and the corresponding eigenvector is v j , where j = 1 to k; The data analysis also includes mapping the data set A to the principal component space, calculating all data a of the data set A i Projection value c in each principal component direction j ji The mean , j=1~k, i=1~n, get the center point of the data set A after mapping it to the principal component space ; S104 Real-time monitoring implementation; The real-time monitoring implementation comprises the following steps: a) placing the surface water buoy in the application environment to monitor water quality indicators in real time, and using the sliding window method to preprocess the real-time collected data in the same way as S103 to obtain a preprocessed real-time data set B, B = {b i |i=1,2,..,p}, where p is the window width; b) Mapping the real-time data set B to the principal component space and calculating all data b of the real-time data set B i Projection value d in each principal component direction j ji The mean , j=1~k, i=1~p, get the center point of the real-time data set B after mapping it to the principal component space ; Calculate the center point and the center point obtained by S103 The distance d in each principal component direction j j , and further obtain the distance d between the two in the principal component space, the distance , j=1~k; the distance d between the two in space is calculated using the Euclidean distance, and the expression of d is formula (1): （1） c) comparing the distance d with an abnormality determination standard. If the abnormality determination standard is reached, it is determined that an abnormality has occurred and the next step of analysis is continued; d) Calculate the center point and The distance d in each principal component direction j j The contribution x to the distance d between the two in the principal component space j , j = 1~k; x j The maximum value x in q Compared with the sensor abnormality judgment standard, if the sensor abnormality judgment standard is met, the current abnormality is judged as a sensor abnormality and x is recorded. q The corresponding principal component direction q, continue to the next step of analysis; the center point and The distance d in each principal component direction j The contribution x to the distance d between the two in the principal component space j Use expression (2) to calculate: （2） e) When it is identified that a sensor abnormality currently occurs, a sensor r associated with the principal component direction q is further identified, wherein 1≤r≤m, then the sensor r is an abnormal sensor, and the result is output.

2. A water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In S103, the kernel function of the kernel principal component analysis method adopts a radial basis function, and the expression of the radial basis function is formula (3): （3） Among them, ||a i -a j || means a i and a j The Euclidean distance of , σ is the adjustment coefficient.

3. A water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In step S103, the method steps of using kernel principal component analysis to obtain k principal components reflecting the main implicit features of the data set A are as follows: a) selecting a kernel function and constructing a kernel matrix of the data set A; b) Centralizing and eigensolving the kernel matrix to obtain n eigenvalues ​​λ j , the corresponding n eigenvectors are v j And the corresponding n principal components, where j=1~n; c) calculating the information contribution of each principal component and sorting them from large to small, wherein the information contribution of each principal component is the ratio of the eigenvalue corresponding to the principal component to the sum of all eigenvalues; d) retaining the first k principal components whose cumulative information contribution is greater than or equal to 90%, and using them to construct the principal component space, and the eigenvalues ​​corresponding to the retained k principal components are λ j , the corresponding eigenvector is v j , j=1~k.

4. A water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In the step S104 c), the abnormality determination standard is set by using the standard deviation method. 阈 The expression of is formula (4): d 阈 = ee, (4) Where ε is the distance from the center point to the data set A. The standard deviation of the distance, η is the adjustment coefficient, which is related to the water quality of the monitoring area and takes a value of 2~3.

5. A water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In step S104 d), the sensor abnormality judgment standard is composed of expressions (5) and (6), and the expressions (5) and (6) are: Among them, m is the number of monitoring indicators for each water quality data.

6. A water quality sensor self-diagnosis method for surface water monitoring according to claim 1, characterized in that: In the step e) of S104, the method for identifying the sensor r associated with the principal component direction q adopts the principal component load analysis method.

Images