Dynamic identification method for abnormal water quality

Abstract

The invention discloses a dynamic identification method for abnormal water quality. The method comprises the following steps: collecting multi-modal molecular spectrum data; performing multi-scale feature extraction on the multi-modal molecular spectrum data, and fusing the extracted multi-scale features to obtain comprehensive features; constructing a time sequence data matrix of different time windows for each modal data in the multi-modal molecular spectrum data, and calculating the distance of the time sequence data matrix corresponding to each modal data in different periods; according to the distance of the time sequence data matrix corresponding to each modal data in different periods, calculating an abnormal index matrix corresponding to the multi-modal molecular spectrum data; based on the comprehensive characteristics and the abnormal index matrix, performing water quality abnormal dynamic identification by adopting a self-adaptive adjustment threshold value; according to the method, improvement schemes such as multi-data source fusion, dynamic time sequence anomaly detection and self-adaptive statistical threshold setting are provided, so that the comprehensiveness, accuracy and robustness of water quality monitoring are improved, and the complex and changeable water quality problem can be better solved.

Description

Technical Field

[0001] The present invention relates to the technical field of water quality monitoring, and in particular to a method for dynamically identifying abnormal water quality. Background Art

[0002] With the acceleration of industrialization and urbanization, water pollution is becoming increasingly serious, threatening human health and the ecological environment. Water quality anomaly identification and early warning technology is a key means to cope with the current water environment challenges. This technology integrates sensors, data collection, transmission and intelligent analysis to achieve real-time monitoring and early warning of various water parameters. As a core component, the sensor can accurately capture key indicators such as pH value, dissolved oxygen, and temperature of the water body to ensure the real-time and accuracy of the data. The online monitoring and early warning system quickly transmits this data to the background, automatically judges the water quality status through preset thresholds and advanced data analysis algorithms, and immediately issues an alarm once an abnormality is found. In addition, data mining and machine learning technologies can also be used to build water quality prediction models to improve the accuracy and timeliness of early warnings. These models can not only help identify current water quality problems, but also predict future water quality trends and provide forward-looking guidance for water environment protection.

[0003] Water quality monitoring involves multiple parameters, including but not limited to pH value, dissolved oxygen, temperature, turbidity, chemical oxygen demand (COD), ammonia nitrogen, total phosphorus (TP), two-dimensional absorption spectrum, three-dimensional fluorescence spectrum and other data. These parameters are not only diverse, but each parameter may also change with time, space, meteorological conditions and other factors. Therefore, the amount of data generated by water quality monitoring is extremely large, and these data often show a high degree of complexity and nonlinear relationships. Therefore, the existing abnormal identification based on a single indicator or the identification of several indicators is difficult to effectively identify abnormal data.

[0004] In addition, the warning threshold is an important basis for judging whether the water quality is abnormal. However, in practical applications, the setting of the warning threshold is often unreasonable. If the warning threshold is set too conservatively (that is, too close to the critical value of normal water quality), the system may frequently issue false alarms, resulting in frequent responses from relevant departments and personnel, wasting resources, and may produce a "wolf cry" effect, reducing people's attention to the real warning signal. On the contrary, if the warning threshold is set too aggressively (that is, far higher than the critical value of normal water quality), the system may miss some actual water quality abnormalities, so that related problems cannot be discovered and handled in time, which may lead to more serious environmental problems or health risks. Moreover, many existing warning systems only use a single threshold to judge whether the water quality is abnormal, ignoring the mutual influence and comprehensive effect between different water quality parameters. This single threshold setting may not fully reflect the actual situation of water quality, resulting in one-sidedness of the warning results.

[0005] Therefore, the main challenges faced by the current water quality anomaly identification technology system are the singleness of the data source, the limitations of short-term warnings, and the robustness of anomaly detection. These problems have a profound impact on the accuracy and timeliness of water quality monitoring. First, the use of a single data source limits the comprehensiveness of water quality monitoring. Due to the complexity and diversity of the water environment, a single sensor or data type is often difficult to capture all potential water quality changes. For example, relying solely on pH data may not fully reflect the presence and changes of other pollutants in the water body, resulting in the neglect or misjudgment of abnormal phenomena. This limitation not only affects the accuracy of water quality monitoring, but may also delay the timely response to potential environmental risks. Secondly, the limitation of the short-term warning system is that it pays too much attention to short-term data changes and ignores long-term trends and gradual changes. Moreover, water quality problems are often cumulative and lagging, and data fluctuations in the short term may not be sufficient to reflect the true state of water quality. Therefore, relying solely on the short-term warning system may not be able to detect and warn long-standing water quality problems in a timely manner, resulting in a slow response to the gradually deteriorating water quality conditions. This limitation not only weakens the effectiveness of the warning system, but may also aggravate the deterioration of water quality problems, posing a potential threat to the ecological environment and human health. Finally, the robustness of anomaly detection is also a major challenge facing current water quality monitoring technology. Due to noise and temporary fluctuations in the data, traditional anomaly detection methods may produce false positive or false negative results, affecting the accuracy of the early warning system. This instability not only reduces the reliability of the system, but may also cause unnecessary panic or ignore real risks. Therefore, improving the robustness of anomaly detection is one of the important directions for improving current water quality monitoring technology. Summary of the invention

[0006] The technical problem to be solved by the present invention is how to improve the performance and application effect of water quality monitoring technology.

[0007] The present invention solves the above technical problems by the following technical means:

[0008] A method for dynamic identification of water quality anomalies is proposed, the method comprising:

[0009] Collecting multimodal molecular spectral data, wherein the multimodal molecular spectral data includes three-dimensional fluorescence spectral data, two-dimensional absorption spectral data and water quality data;

[0010] Extracting multi-scale features from the multimodal molecular spectral data, and fusing the extracted multi-scale features to obtain comprehensive features;

[0011] Constructing a time series data matrix of different time windows for each modal data in the multimodal molecular spectral data, and calculating the distance of the time series data matrix corresponding to each modal data in different periods;

[0012] Calculating the abnormal index matrix corresponding to the multimodal molecular spectral data according to the distance of the time series data matrix corresponding to each modal data at different periods;

[0013] Based on the comprehensive features and the abnormal index matrix, an adaptive threshold is used to dynamically identify water quality anomalies.

[0014] Furthermore, the multimodal molecular spectral data is collected, and the multimodal molecular spectral data includes three-dimensional fluorescence spectral data, two-dimensional absorption spectral data and water quality data, including:

[0015] Collecting three-dimensional fluorescence spectrum data at different time points, wherein the three-dimensional fluorescence spectrum data includes spatial position, wavelength and time;

[0016] collecting two-dimensional absorption spectrum data at different time points, wherein the two-dimensional absorption spectrum data includes wavelength and time;

[0017] The water quality data at different time points are collected, and the water quality data include temperature, pH value, conductivity, COD, ammonia nitrogen, and total phosphorus.

[0018] Furthermore, after collecting the multimodal molecular spectroscopy data, the method further comprises:

[0019] performing normalization processing on the multimodal molecular spectral data;

[0020] Accordingly, the multi-scale feature extraction is performed on the multi-modal molecular spectral data, and the extracted multi-scale features are fused to obtain comprehensive features, including:

[0021] Multi-scale feature extraction is performed on the normalized multimodal molecular spectral data, and the extracted multi-scale features are fused to obtain comprehensive features.

[0022] Furthermore, the multi-scale feature extraction is performed on the multi-modal molecular spectral data, and the extracted multi-scale features are fused to obtain comprehensive features, including:

[0023] Using a three-dimensional convolutional neural network to extract features from the three-dimensional fluorescence spectrum data to obtain three-dimensional features;

[0024] Using a two-dimensional convolutional neural network to extract features from the two-dimensional absorption spectrum data to obtain spectral features;

[0025] A one-dimensional convolutional neural network is used to extract features of the water quality data to obtain features of the time series data;

[0026] The three-dimensional features, the spectral features and the features of the time series data are fused to obtain comprehensive features.

[0027] Furthermore, constructing a time series data matrix of different time windows for each modal data in the multimodal molecular spectral data, and calculating the distance of the time series data matrix corresponding to each modal data in different periods, includes:

[0028] Constructing a three-dimensional time series data matrix at different time points according to the three-dimensional fluorescence spectrum data;

[0029] Constructing a two-dimensional time series data matrix at different time points according to the two-dimensional absorption spectrum data;

[0030] According to the water quality data, construct a one-dimensional time series data matrix at different time points;

[0031] The distances of the three-dimensional time series data matrix, the two-dimensional time series data matrix, and the one-dimensional time series data matrix in different periods are calculated respectively.

[0032] Furthermore, the calculation of the abnormal index matrix corresponding to the multimodal molecular spectral data according to the distance of the time series data matrix corresponding to each modal data at different periods includes:

[0033] According to the distance of the time series data matrix corresponding to each modal data in different periods, the abnormal index set corresponding to each modal data is calculated;

[0034] According to the abnormal index set corresponding to each modal data, the abnormal index matrix corresponding to the multimodal molecular spectral data is calculated.

[0035] Furthermore, the calculation of the abnormal index set corresponding to each modal data according to the distance of the time series data matrix corresponding to each modal data in different periods includes:

[0036] According to the distance of the time series data matrix corresponding to each modal data in different periods, the abnormal index corresponding to each modal data at different time points is calculated. The formula is expressed as:

[0037]

[0038] In the formula, w n is the abnormal index at time n, D n,n+1 is the distance between the time series data matrix at time n and time n+1, D n-1,n is the distance between the time series data matrix at time n-1 and time n;

[0039] According to the abnormal index corresponding to each modal data at different time points, the abnormal index set corresponding to each modal data is calculated.

[0040] Furthermore, the method of dynamically identifying water quality anomalies by adaptively adjusting thresholds based on the comprehensive features and the anomaly index matrix includes:

[0041] According to the abnormal index matrix, setting an adaptive adjustment threshold;

[0042] Modeling the comprehensive features using a distribution estimation method to obtain a distribution model of the comprehensive features;

[0043] According to the distribution model of the comprehensive characteristics and the adaptive adjustment threshold, it is determined that the comprehensive characteristics at a certain time point deviate from the normal distribution range and the data at this time point exceeds the adaptive adjustment threshold, then it is determined that the data at this time point has water quality abnormalities.

[0044] Furthermore, setting the adaptive adjustment threshold according to the abnormal index matrix includes:

[0045] Calculate the statistical characteristics of each abnormal index in the abnormal index matrix using a sliding window or a recursive statistical method;

[0046] Using an adaptive filter to smooth the statistical features to obtain smoothed statistical features;

[0047] The Bayesian method is used to set the threshold of each statistical feature based on the historical data distribution to obtain the adaptive adjustment threshold.

[0048] Furthermore, after determining that the data at the time point has abnormal water quality, the method further includes:

[0049] Analyze the causes, types and extent of abnormal water quality and suggest measures to be taken.

[0050] The advantages of the present invention are:

[0051] (1) Based on the anomaly recognition of molecular spectral data of different dimensions, such as three-dimensional fluorescence, two-dimensional fluorescence spectrum, water quality data and other multi-dimensional data, the present invention proposes a dynamic multi-scale time series anomaly data set and an adaptive statistical threshold and outlier detection method. It combines multimodal data and dynamic feature analysis, which can significantly improve the accuracy and robustness of water quality anomaly recognition and realize short-term and long-term water quality anomaly recognition.

[0052] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a method for dynamic identification of abnormal water quality proposed in one embodiment of the present invention;

[0054] Figure 2 It is a schematic diagram of the principle of the dynamic identification process of water quality anomalies in one embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] like Figure 1 to Figure 2 As shown, an embodiment of the present invention provides a method for dynamically identifying abnormal water quality, the method comprising the following steps:

[0057] S10, collecting multimodal molecular spectral data, wherein the multimodal molecular spectral data includes three-dimensional fluorescence spectral data, two-dimensional absorption spectral data and water quality data;

[0058] It should be noted that this embodiment can use conventional instruments to collect multimodal molecular spectral data.

[0059] S20, extracting multi-scale features from the multimodal molecular spectral data, and fusing the extracted multi-scale features to obtain comprehensive features;

[0060] It should be noted that the fusion of multiple data sources is an effective way to solve the limitations of a single data source. This embodiment can construct a more comprehensive representation of water quality characteristics by integrating multi-dimensional information such as three-dimensional fluorescence spectral data, two-dimensional absorption spectral data and water quality data.

[0061] S30, constructing a time series data matrix of different time windows for each modal data in the multimodal molecular spectral data, and calculating the distance of the time series data matrix corresponding to each modal data in different periods;

[0062] It should be noted that this embodiment adopts a multi-scale feature extraction and fusion method to make up for the shortcomings of a single data source and improve the accuracy and comprehensiveness of anomaly detection. At the same time, the fusion of multiple data sources also helps to discover the inherent connections and mutual influences between different data, providing strong support for in-depth analysis and resolution of water quality issues.

[0063] S40, calculating an abnormal index matrix corresponding to the multimodal molecular spectral data according to the distances of the time series data matrices corresponding to each modal data at different periods;

[0064] S50: Based on the comprehensive features and the abnormal index matrix, adaptively adjust the threshold to perform dynamic identification of water quality abnormalities.

[0065] As a further preferred technical solution, the step S10: collecting multimodal molecular spectral data includes:

[0066] Collecting three-dimensional fluorescence spectrum data at different time points, wherein the three-dimensional fluorescence spectrum data includes spatial position (spatial position or detector position), wavelength and time;

[0067] collecting two-dimensional absorption spectrum data at different time points, wherein the two-dimensional absorption spectrum data includes wavelength and time;

[0068] The water quality data at different time points are collected, and the water quality data include temperature, pH value, conductivity, COD, ammonia nitrogen, and total phosphorus.

[0069] As a further preferred technical solution, after the step S10: collecting multimodal molecular spectral data, the method further comprises the following steps:

[0070] performing normalization processing on the multimodal molecular spectral data;

[0071] Accordingly, the multi-scale feature extraction is performed on the multi-modal molecular spectral data, and the extracted multi-scale features are fused to obtain comprehensive features, including:

[0072] Multi-scale feature extraction is performed on the normalized multimodal molecular spectral data, and the extracted multi-scale features are fused to obtain comprehensive features.

[0073] It should be noted that this embodiment ensures data consistency by normalizing the three-dimensional fluorescence spectrum data, the two-dimensional absorption spectrum data and the water quality data; and ensures data integrity by processing missing values ​​in the data, such as using interpolation to fill in the missing data.

[0074] As a further preferred technical solution, the step S20: extracting multi-scale features from the multimodal molecular spectral data and fusing the extracted multi-scale features to obtain comprehensive features, comprises the following steps:

[0075] Using a three-dimensional convolutional neural network to extract features from the three-dimensional fluorescence spectrum data to obtain three-dimensional features including time, space and wavelength dimensions;

[0076] Using a two-dimensional convolutional neural network to extract features from the two-dimensional absorption spectrum data to obtain spectral features;

[0077] A one-dimensional convolutional neural network is used to extract features of the water quality data to obtain features of the time series data;

[0078] The three-dimensional features, the spectral features and the features of the time series data are fused to obtain comprehensive features.

[0079] It should be noted that the one-dimensional convolutional neural network (1D-CNN), two-dimensional convolutional neural network (2D-CNN) and three-dimensional convolutional neural network (3D-CNN) used in this embodiment are all trained in advance using the training set, verification set and test set divided by the collected historical data set.

[0080] Specifically, this embodiment uses multi-scale feature extraction methods, such as convolutional neural networks (MS-CNN), principal component analysis PCA, wavelet transform, and change coding, to extract features of water quality data at different time scales and capture abnormal patterns of data in the short and long term. And by using methods such as splicing, weighted averaging or fusion layers, the feature maps of three-dimensional data, two-dimensional spectral data, and water quality data are fused into a comprehensive feature representation. By dynamically fusing information from different data sources and performing real-time processing on time series data, the accuracy and robustness of anomaly detection are improved, and the deviations that may be encountered in the judgment of a single sample are reduced.

[0081] This embodiment performs dynamic data fusion, combining the characteristic graphs of three-dimensional fluorescence spectrum data, two-dimensional absorption spectrum data and water quality data for analysis. This fusion method can comprehensively consider the information of multiple data sources and improve the system's ability to identify anomalies, especially when there are complex relationships between data sources. It enhances the ability to detect water quality anomalies, improves the robustness and accuracy of the system, and can better cope with the complex relationships between data sources. This multi-scale feature extraction and fusion method can make up for the shortcomings of a single data source and improve the accuracy and comprehensiveness of anomaly detection. At the same time, the fusion of multiple data sources can also help discover the inherent connections and mutual influences between different data, providing strong support for the in-depth analysis and resolution of water quality problems.

[0082] As a further preferred technical solution, the step S30: constructing a time series data matrix of different time windows for each modal data in the multimodal molecular spectral data, and calculating the distance of the time series data matrix corresponding to each modal data in different periods, includes the following steps:

[0083] S31, constructing a three-dimensional time series data matrix at different time points according to the three-dimensional fluorescence spectrum data;

[0084] S32, constructing a two-dimensional time series data matrix at different time points according to the two-dimensional absorption spectrum data;

[0085] S33, constructing a one-dimensional time series data matrix at different time points according to the water quality data;

[0086] Specifically, this embodiment constructs a three-dimensional data matrix A at time point T=n based on the three-dimensional fluorescence spectrum data matrix. T=n; Based on the two-dimensional absorption spectrum data matrix, construct a two-dimensional spectrum matrix B at time point T = n T =n; Based on the water quality data matrix, a one-dimensional data matrix C at time point T = n is constructed T =n.

[0087] S34, respectively calculating the distances of the three-dimensional time series data matrix, the two-dimensional time series data matrix, and the one-dimensional time series data matrix in different periods.

[0088] Specifically, this implementation calculates the distance of each time series data matrix in different periods as follows:

[0089] Three-dimensional fluorescence spectrum data distance: Calculate the three-dimensional data matrix A T =n and A T =n+1, A T =Euclidean distance or Mahalanobis distance between n-1.

[0090] Two-dimensional absorption spectrum data distance: Calculate the two-dimensional data matrix B T =n and B T =n+1, B T =Euclidean distance or Mahalanobis distance between n-1.

[0091] Water quality data distance: Calculate the three-dimensional data matrix C T =n and C T =n+1, C T =Euclidean distance or Mahalanobis distance between n-1.

[0092] As a further preferred technical solution, the step S40: calculating the abnormal index matrix corresponding to the multimodal molecular spectral data according to the distance of the time series data matrix corresponding to each modal data at different periods, comprises the following steps:

[0093] S41, calculating the abnormal index set corresponding to each modal data according to the distance of the time series data matrix corresponding to each modal data in different periods;

[0094] S42. Calculate an abnormal index matrix corresponding to the multimodal molecular spectral data according to an abnormal index set corresponding to each modal data.

[0095] As a further preferred technical solution, the step S41: calculating the abnormal index set corresponding to each modal data according to the distance of the time series data matrix corresponding to each modal data in different periods, includes:

[0096] According to the distance of the time series data matrix corresponding to each modal data in different periods, the abnormal index corresponding to each modal data at different time points is calculated. The formula is expressed as:

[0097]

[0098] In the formula, w n is the abnormal index at time n, D n,n+1 is the distance between the time series data matrix at time n and time n+1, D n-1,n is the distance between the time series data matrix at time n-1 and time n;

[0099] According to the abnormal index corresponding to each modal data at different time points, the abnormal index set corresponding to each modal data is calculated.

[0100] It should be noted here that the time series data matrix in this embodiment can be expressed as a three-dimensional time series data matrix, a two-dimensional time series data matrix and a one-dimensional time series data matrix.

[0101] It should be noted that existing anomaly detection methods may not be able to effectively process complex time series data, or may not be sensitive enough to data changes. The judgment of outliers may lack sufficient accuracy and scientific basis, resulting in reduced reliability of the judgment results. The present invention provides a quantitative anomaly metric by calculating the anomaly index (such as calculation based on Euclidean distance or Mahalanobis distance) to accurately evaluate the anomaly index of data matrix changes between time points, so that the system can more scientifically determine whether there is an abnormal situation. Provided is a scientific and quantitative anomaly detection method, improves the accuracy and reliability of detection.

[0102] As a further preferred technical solution, the step S50: based on the comprehensive features and the abnormal index matrix, using an adaptive threshold to dynamically identify water quality abnormalities includes the following steps:

[0103] S51, setting an adaptive adjustment threshold according to the abnormal index matrix;

[0104] S52, modeling the comprehensive features using a distribution estimation method to obtain a distribution model of the comprehensive features;

[0105] This embodiment uses distribution estimation (such as mixed Gaussian model) to model the data and combines it with dynamic thresholds for anomaly detection. A mixed Gaussian model (GMM) is used to model each indicator in the matrix data. For multidimensional data, GMM can be used to estimate the joint distribution, and the formula is expressed as:

[0106]

[0107] In the formula, π i is the weight of the ith Gaussian distribution, is the PDF of the Gaussian distribution.

[0108] It should be noted that the comprehensive features contain key information extracted from three-dimensional fluorescence spectra, two-dimensional absorption spectra and water quality data. The comprehensive features are modeled by using distribution estimation methods (such as Gaussian mixture model GMM, kernel density estimation, etc.) to capture their potential distribution characteristics. The distribution model will be used for subsequent anomaly detection.

[0109] S53. According to the distribution model of the comprehensive characteristics and the adaptive adjustment threshold, it is determined that the comprehensive characteristics at a certain time point deviate from the normal distribution range and the data at this time point exceeds the adaptive adjustment threshold, and it is determined that the data at this time point has abnormal water quality.

[0110] It should be noted that the comprehensive features obtained in this embodiment include feature information extracted and fused from multiple data sources, which fully reflect the various characteristics and potential anomalies of water quality data. The anomaly index matrix is ​​obtained by calculating the distance and change rate between data matrices, reflecting the stability and variability of data at different time points.

[0111] In the anomaly detection process, comprehensive features are used to calculate dynamic statistical features, while the anomaly index matrix serves as an important reference for anomaly detection. By combining comprehensive features with the anomaly index matrix, the system can more comprehensively evaluate the anomalies of water quality data and improve the accuracy and reliability of anomaly detection. When the comprehensive features indicate that the data at a certain point in time is significantly different from the normal distribution, and the anomaly index matrix also shows that the data at that point in time has significantly changed from the previous and next moments, the system will consider that there is an anomaly at that point in time. This combination makes full use of information from multi-source data and multiple features to improve the ability to dynamically identify water quality anomalies.

[0112] As a further preferred technical solution, the step S51: setting the adaptive adjustment threshold according to the abnormal index matrix includes:

[0113] S511, using a sliding window or a recursive statistical method to calculate the statistical characteristics of each abnormal index in the abnormal index matrix;

[0114] Specifically, this embodiment calculates the dynamic mean, standard deviation, quartiles and other statistical features of the abnormal index through a sliding window or recursive statistical method. An adaptive filter is used to smooth the statistical features to reduce the impact of noise. The dynamic mean and standard deviation are calculated for each indicator in the matrix data. This means that for each column (or each indicator) of the matrix, its dynamic mean and standard deviation are calculated respectively. The sliding window method is used for each indicator to dynamically update the mean and standard deviation. The formula is expressed as:

[0115]

[0116] In the formula, μt,j and σ t,j are the dynamic mean and standard deviation of the index at time point t and the jth index, respectively, x i,j is the data point in the i-th row and j-th column in the matrix, and w is the window size.

[0117] S512, using an adaptive filter to smooth the statistical features to obtain smoothed statistical features;

[0118] S513: Using the Bayesian method to set a threshold for each statistical feature based on historical data distribution, to obtain an adaptive adjustment threshold.

[0119] It should be noted that this embodiment combines the Bayesian method or the dynamic probability model to model the statistical threshold, dynamically adjusts the anomaly detection threshold, calculates the upper and lower limits based on the dynamic statistical characteristics, and sets the anomaly detection threshold.

[0120] Take the Bayesian adaptive threshold as an example: set the threshold for each indicator in the matrix data based on the historical data distribution. Using the Bayesian method can help dynamically adjust the threshold of each indicator, and the formula is expressed as:

[0121] Threshold t,j =μ t,j +k·σ t,j

[0122] Threshold t,j =μ t,j -k·σ t,j

[0123] Among them, Threshold t,j is the threshold value for indicator j at time t, μ t,j is the mean (or expected value) of indicator j at time t, k is a multiple of the standard deviation, usually 2 or 3, σ t,j is the standard deviation of indicator j at time t.

[0124] It should be noted that since the static anomaly detection threshold may not be able to adapt to the dynamic changes of data, the sensitivity of anomaly detection is not enough. The detection of outliers may be affected by noise, thereby increasing the risk of false positives and false negatives. This embodiment introduces an adaptive threshold setting method based on dynamic statistical features (such as mean, standard deviation, etc.). Through sliding windows and recursive statistical methods, the threshold is dynamically adjusted to adapt to long-term and short-term changes in data. The adaptive statistical threshold mechanism reduces the impact of noise and can adjust the detection threshold according to the dynamic changes of data characteristics. The sensitivity and accuracy of anomaly detection are improved, the occurrence of false positives and false negatives is reduced, and the reliability of the system is enhanced.

[0125] Adaptive statistical threshold setting is an important means to improve the robustness of anomaly detection. By setting thresholds based on dynamic statistical features and combining Bayesian methods or dynamic probability models for modeling, the impact of noise and temporary fluctuations on anomaly detection can be reduced. This adaptive statistical threshold setting method can dynamically adjust the anomaly index and improve the stability and reliability of the system. At the same time, it can also reduce the occurrence of false positives and false negatives and improve the accuracy and credibility of the early warning system. The implementation of these improvement plans will significantly improve the performance and application effect of water quality monitoring technology and provide strong support for water quality protection and ecological environment protection.

[0126] As a further preferred technical solution, in step S53, after determining that the data at the time point has water quality abnormality, the method further includes the following steps:

[0127] Analyze the causes, types and extent of abnormal water quality and suggest measures to be taken.

[0128] Specifically, after detecting abnormal water quality, this embodiment can further analyze the cause, type, degree, etc. of the abnormality as needed, and take corresponding measures (such as alarming, recording, adjusting monitoring parameters, etc.).

[0129] This embodiment proposes improvement schemes such as multi-data source fusion, dynamic time series anomaly detection and adaptive statistical threshold setting. These schemes are aimed at improving the comprehensiveness, accuracy and robustness of water quality monitoring to better cope with complex and changeable water quality problems.

[0130] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0131] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0132] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for dynamic identification of abnormal water quality, characterized in that: The method comprises: Collecting multimodal molecular spectral data, wherein the multimodal molecular spectral data includes three-dimensional fluorescence spectral data, two-dimensional absorption spectral data and water quality data; Extracting multi-scale features from the multimodal molecular spectral data, and fusing the extracted multi-scale features to obtain comprehensive features; Constructing a time series data matrix of different time windows for each modal data in the multimodal molecular spectral data, and calculating the distance of the time series data matrix corresponding to each modal data in different periods; Calculating the abnormal index matrix corresponding to the multimodal molecular spectral data according to the distance of the time series data matrix corresponding to each modal data at different periods; Based on the comprehensive features and the abnormal index matrix, an adaptive threshold is used to dynamically identify water quality anomalies.

2. The method for dynamic identification of abnormal water quality according to claim 1, characterized in that: The collecting of multimodal molecular spectral data, wherein the multimodal molecular spectral data includes three-dimensional fluorescence spectral data, two-dimensional absorption spectral data and water quality data, includes: Collecting three-dimensional fluorescence spectrum data at different time points, wherein the three-dimensional fluorescence spectrum data includes spatial position, wavelength and time; collecting two-dimensional absorption spectrum data at different time points, wherein the two-dimensional absorption spectrum data includes wavelength and time; The water quality data at different time points are collected, and the water quality data include temperature, pH value, conductivity, COD, ammonia nitrogen, and total phosphorus.

3. The method for dynamic identification of abnormal water quality according to claim 1, characterized in that: After collecting the multimodal molecular spectroscopy data, the method further comprises: performing normalization processing on the multimodal molecular spectral data; Accordingly, the multi-scale feature extraction is performed on the multi-modal molecular spectral data, and the extracted multi-scale features are fused to obtain comprehensive features, including: Multi-scale feature extraction is performed on the normalized multimodal molecular spectral data, and the extracted multi-scale features are fused to obtain comprehensive features.

4. The method for dynamic identification of abnormal water quality according to claim 1, characterized in that: The method extracts multi-scale features from the multi-modal molecular spectral data and fuses the extracted multi-scale features to obtain comprehensive features, including: Using a three-dimensional convolutional neural network to extract features from the three-dimensional fluorescence spectrum data to obtain three-dimensional features; Using a two-dimensional convolutional neural network to extract features from the two-dimensional absorption spectrum data to obtain spectral features; A one-dimensional convolutional neural network is used to extract features of the water quality data to obtain features of the time series data; The three-dimensional features, the spectral features and the features of the time series data are fused to obtain comprehensive features.

5. The method for dynamic identification of abnormal water quality according to claim 1, characterized in that: The step of constructing a time series data matrix of different time windows for each modal data in the multimodal molecular spectral data, and calculating the distance of the time series data matrix corresponding to each modal data in different periods, includes: Constructing a three-dimensional time series data matrix at different time points according to the three-dimensional fluorescence spectrum data; Constructing a two-dimensional time series data matrix at different time points according to the two-dimensional absorption spectrum data; According to the water quality data, construct a one-dimensional time series data matrix at different time points; The distances of the three-dimensional time series data matrix, the two-dimensional time series data matrix, and the one-dimensional time series data matrix in different periods are calculated respectively.

6. The method for dynamic identification of abnormal water quality according to claim 5, characterized in that: The method of calculating the abnormal index matrix corresponding to the multimodal molecular spectral data according to the distances of the time series data matrix corresponding to each modal data at different periods includes: According to the distance of the time series data matrix corresponding to each modal data in different periods, the abnormal index set corresponding to each modal data is calculated; According to the abnormal index set corresponding to each modal data, the abnormal index matrix corresponding to the multimodal molecular spectral data is calculated.

7. The method for dynamic identification of abnormal water quality according to claim 6, characterized in that: The method of calculating the abnormal index set corresponding to each modal data according to the distance of the time series data matrix corresponding to each modal data in different periods includes: According to the distance of the time series data matrix corresponding to each modal data in different periods, the abnormal index corresponding to each modal data at different time points is calculated. The formula is expressed as: In the formula, w n is the abnormal index at time n, D n,n+1 is the distance between the time series data matrix at time n and time n+1, D n-1,n is the distance between the time series data matrix at time n-1 and time n; According to the abnormal index corresponding to each modal data at different time points, the abnormal index set corresponding to each modal data is calculated.

8. The method for dynamic identification of abnormal water quality according to claim 1, characterized in that: The method of dynamically identifying water quality anomalies by adaptively adjusting thresholds based on the comprehensive features and the anomaly index matrix includes: Modeling the comprehensive features using a distribution estimation method to obtain a distribution model of the comprehensive features; According to the distribution model of the comprehensive characteristics and the adaptive adjustment threshold, it is determined that the comprehensive characteristics at a certain time point deviate from the normal distribution range and the data at this time point exceeds the adaptive adjustment threshold, then it is determined that the data at this time point has water quality abnormalities.

9. The method for dynamic identification of abnormal water quality according to claim 8, characterized in that: The step of setting the adaptive adjustment threshold according to the abnormal index matrix includes: Calculate the statistical characteristics of each abnormal index in the abnormal index matrix using a sliding window or a recursive statistical method; Using an adaptive filter to smooth the statistical features to obtain smoothed statistical features; The Bayesian method is used to set the threshold of each statistical feature based on the historical data distribution to obtain the adaptive adjustment threshold.

10. The method for dynamic identification of abnormal water quality according to claim 8, characterized in that: After determining that the data at the time point has abnormal water quality, the method further includes: Analyze the causes, types and extent of abnormal water quality and suggest measures to be taken.

Images