Sewage treatment abnormality early warning method and system based on big data analysis

Abstract

The present application relates to the field of data processing, and more particularly to a sewage treatment abnormality early warning method and system based on big data analysis. The method comprises the steps of: constructing a causal relationship structure diagram; calculating the theoretical sensitivity between two nodes based on the causal relationship structure diagram, constructing a sewage treatment index prediction network based on the LSTM algorithm, inputting all types of sewage treatment operation data segments in each sample in the historical operation data set into the sewage treatment index prediction network, obtaining the output data corresponding to each node of the type, and calculating the fitting sensitivity of the two nodes based on the change correlation between the output data of the corresponding type of the latter node and the output data of the corresponding type of the former node; constructing a causal correlation loss function, and completing the training of the sewage treatment index prediction network based on the causal correlation loss function and the loss function provided with the LSTM algorithm, so as to realize the sewage treatment abnormality early warning. The accuracy of the sewage treatment abnormality early warning is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing, and in particular to a method and system for early warning of anomalies in wastewater treatment based on big data analysis. Background Technology

[0002] Wastewater treatment is a critical facility for ensuring environmental safety and public health, and its core is to ensure that the effluent quality (such as COD, ammonia nitrogen, etc.) meets the standards. Accurately predicting water quality changes in advance and issuing early warnings is crucial for monitoring the treatment process.

[0003] In recent years, data-driven water quality prediction has become mainstream. Long Short-Term Memory (LSTM) networks are widely used because they are good at capturing long-term dependencies in time series data. They infer future trends by learning historical data of target water quality indicators.

[0004] However, applying LSTM to wastewater treatment scenarios has significant limitations: First, it treats the wastewater treatment process as a "black box," learning only the surface correlations of data while ignoring internal processes and causal mechanisms, failing to distinguish the direct and indirect effects and transmission processes of variables at different levels. Second, it is insensitive to the "dynamic changes" of upstream parameters, resulting in delayed prediction responses. It can only passively capture the consequences of parameter changes and cannot understand the forward-looking information of the changes themselves. For example, if an operator lowers the fan frequency, although the model knows that it will eventually increase ammonia nitrogen, it only makes a judgment after observing a decrease in DO concentration, failing to grasp the quantitative relationship between them, leading to delayed early warnings and a significant decrease in prediction accuracy under dynamic operating conditions.

[0005] Therefore, this field urgently needs new methods that not only understand the inherent causal structure of wastewater treatment, but also clearly establish a quantitative relationship between the "dynamic changes" of upstream parameters and the "dynamic response" of downstream water quality, and use this to constrain model training, so as to fundamentally improve the prediction accuracy under dynamic operating conditions. This is the core problem that this invention aims to solve. Summary of the Invention

[0006] To address the problem of how to enable predictive networks to learn causal information about wastewater treatment processes, thereby improving the accuracy of wastewater treatment predictions, this invention provides a wastewater treatment anomaly early warning method and system based on big data analysis.

[0007] In a first aspect, the present invention provides a wastewater treatment anomaly early warning method based on big data analysis, employing the following technical solution:

[0008] A wastewater treatment anomaly early warning method based on big data analysis includes the following steps:

[0009] Obtain a historical operation dataset, wherein each sample in the historical operation dataset contains several types of wastewater treatment operation data segments for a specific time period;

[0010] Using various types of wastewater treatment operation data as nodes, directed edges and corresponding weights of directed edges are constructed between nodes based on the correlation between different types of wastewater treatment operation data, resulting in a causal relationship structure graph; the theoretical sensitivity between two nodes is calculated based on the causal relationship structure graph, and the theoretical sensitivity is positively correlated with the weight of the directed edge of the path between the two nodes.

[0011] A wastewater treatment index prediction network is constructed based on the LSTM algorithm. All types of wastewater treatment operation data segments from each sample in the historical operation dataset are input into the wastewater treatment index prediction network to obtain the output data of each type corresponding to each node. The fitting sensitivity of the two nodes is calculated based on the variation correlation between the output data of the type corresponding to the latter node and the output data of the type corresponding to the former node. A causal correlation loss function is constructed, which is positively correlated with the difference between the fitting sensitivity and the theoretical sensitivity.

[0012] The wastewater treatment indicator prediction network is trained based on the causal correlation loss function and the loss function built into the LSTM algorithm to achieve early warning of wastewater treatment anomalies.

[0013] This invention designs a causal association loss function to enable a wastewater treatment index prediction network to learn the causal association information of each stage of the wastewater treatment process. This enhances the network's ability to analyze changes in downstream data caused by changes in upstream data, thereby improving the accuracy and robustness of the predictions. Furthermore, in designing the causal association loss function, theoretical sensitivity is introduced to accurately reflect the association between two stages in the wastewater treatment process obtained from empirical information. Similarly, fitting sensitivity is introduced to accurately reflect the association information between two stages in the wastewater treatment process fitted by the prediction network. Furthermore, by analyzing the difference between fitting sensitivity and theoretical sensitivity, the difference between the association between different stages in the wastewater treatment process fitted by the prediction network and the association between two stages in the empirically obtained wastewater treatment process is reflected. This guides the association between different stages in the wastewater treatment process fitted by the prediction network to approach the association between two stages in the empirically obtained wastewater treatment process, thus enabling the wastewater treatment index prediction network to learn the association information between different stages in the wastewater treatment process.

[0014] Preferably, the step of using various types of wastewater treatment operation data as nodes, constructing directed edges between nodes and the weights of corresponding directed edges based on the correlation between different types of wastewater treatment operation data, and obtaining a causal relationship structure graph includes:

[0015] Based on the process flow, directed edges are constructed for nodes corresponding to different types of wastewater treatment data; the time series sequence of wastewater treatment operation data for each node is obtained; the misalignment duration of the time series sequence of wastewater treatment operation data for two adjacent nodes is calculated; based on the time series relationship of the time series sequence of wastewater treatment operation data for two adjacent nodes under the misalignment duration, the change sensitivity of the directed edge corresponding to the two adjacent nodes is calculated, and the change sensitivity of the two adjacent nodes is used as the weight of the directed edge corresponding to the two adjacent nodes; the resulting graph structure is denoted as the causal relationship structure graph.

[0016] This invention uses a cause-and-effect diagram to vividly and accurately reflect the relationships between different stages in the wastewater treatment process.

[0017] Preferably, the calculation of the misalignment duration of the time series of wastewater treatment operation data between two adjacent nodes includes:

[0018] The time series of sewage treatment operation data of the preceding node relative to the following node in two adjacent nodes is shifted by one time step. After each shift, the correlation of the time series of sewage treatment operation data of the two adjacent nodes after the shift is calculated. The number of shifts corresponding to the maximum correlation value is taken as the shift duration of the time series of sewage treatment operation data of the two adjacent nodes.

[0019] This invention uses misalignment correlation analysis to more accurately obtain the misalignment duration.

[0020] Preferably, the calculation of the change sensitivity of the directed edge corresponding to the two adjacent nodes based on the time series relationship of the sewage treatment operation data time series of two adjacent nodes under the misalignment duration includes:

[0021] The difference between each data point and the previous data point in the time series of wastewater treatment operation data at each node is taken as the change in each data point.

[0022] The change in any data point in the time series of wastewater treatment operation data of the latter node among two adjacent nodes is divided by the change in the corresponding time point of the wastewater treatment data time series of the former node under the misalignment duration and the sum of the zero-prevention parameter, which is used as the change sensitivity of that data.

[0023] The average change sensitivity of all data in the time series of wastewater treatment operation data of the latter of two adjacent nodes is taken as the change sensitivity of the two adjacent nodes.

[0024] This invention accurately reflects the sensitivity of changes in two adjacent nodes by using the correlation information of data changes between the two adjacent nodes.

[0025] Preferably, the calculation of the theoretical sensitivity between two nodes based on the causal relationship structure graph includes:

[0026] In the causal relationship structure graph, obtain all paths between two nodes, and take the product of the weights of all directed edges on any path between the two nodes as the theoretical sensitivity of that path.

[0027] The theoretical sensitivity between the two nodes is the sum of the theoretical sensitivities of all paths between them.

[0028] This invention uses theoretical sensitivity to accurately reflect the correlation between nodes obtained based on experience.

[0029] Preferably, the step of calculating the fitting sensitivity of the two nodes based on the correlation between the changes in the output data of the corresponding type of the latter node and the output data of the corresponding type of the former node includes:

[0030]

[0031] in, This represents the partial derivative of the output data of the type corresponding to the j-th node at time h. This represents the partial derivative of the output data of the type corresponding to the i-th node at time k. This represents the time-correspondence weight between the output data of the j-th node at time h and the output data of the i-th node at time k. Indicates the number of moments in the output data. Indicates the zero-prevention parameter. This represents the fitting sensitivity between the i-th node and the j-th node.

[0032] This invention uses fitting sensitivity to accurately reflect the correlation between nodes fitted by the wastewater treatment index prediction network.

[0033] Preferably, the method for obtaining the time-corresponding weight includes:

[0034] The variance of the correlation calculated after all times of misalignment is denoted as the reference variance. A Gaussian distribution function is constructed with the misalignment duration as the mean and the reference variance as the variance. The time interval between the corresponding time of each data type of the i-th node and the corresponding time of each data type of the j-th node is calculated. The value at the time interval length in the Gaussian distribution function is denoted as the distribution weight.

[0035] The normalized value of the distribution weights is used as the weight corresponding to time.

[0036] Preferably, the construction of the causal association loss function includes:

[0037]

[0038] in, This represents the fitting sensitivity between the i-th node and the j-th node. This represents the theoretical sensitivity of the i-th node and the j-th node. This represents the number of successor nodes of the i-th node. Indicates the number of predecessor nodes. This represents the loss function for causal association.

[0039] Preferably, the method for achieving abnormal early warning in wastewater treatment includes:

[0040] The newly collected wastewater treatment operation data segments of several types in the current time period are input into the trained wastewater treatment index prediction network to obtain wastewater treatment operation data of various types at future time. If there is a data in the wastewater treatment operation data of various types at future time that meets the abnormal warning conditions, an abnormal warning is issued.

[0041] If all wastewater treatment operation data of various types do not meet the abnormal warning conditions at any future time, no warning will be issued.

[0042] Secondly, the present invention provides a wastewater treatment anomaly early warning system based on big data analysis, which adopts the following technical solution:

[0043] The wastewater treatment anomaly early warning system based on big data analysis includes a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned wastewater treatment anomaly early warning method based on big data analysis is implemented.

[0044] By adopting the above technical solution, the above-mentioned wastewater treatment anomaly early warning method based on big data analysis is generated into a computer program and stored in a memory so that it can be loaded and executed by a processor. In this way, a terminal device can be made based on the memory and the processor for convenient use.

[0045] The present invention has the following technical effects:

[0046] This invention designs a causal association loss function to enable the wastewater treatment index prediction network to learn the causal association information of each stage of the wastewater treatment process. This enhances the network's ability to analyze changes in downstream data caused by changes in upstream data during the wastewater treatment process, thereby improving the accuracy and robustness of the predictions.

[0047] Furthermore, when designing the causal association loss function, theoretical sensitivity is introduced to accurately reflect the association between the two stages in the wastewater treatment process obtained from empirical information; when designing the causal association loss function, fitting sensitivity is introduced to accurately reflect the association information between the two stages in the wastewater treatment process fitted by the wastewater treatment index prediction network.

[0048] Furthermore, by analyzing the difference between the fitting sensitivity and the theoretical sensitivity, we can reflect the difference between the correlation between different links in the wastewater treatment process fitted by the wastewater treatment index prediction network and the correlation between two links in the wastewater treatment process obtained from experience. This will guide the correlation between different links in the wastewater treatment process fitted by the index prediction network to approach the correlation between two links in the wastewater treatment process obtained from experience, thereby enabling the wastewater treatment index prediction network to learn the correlation information between different links in the wastewater treatment process. Attached Figure Description

[0049] Figure 1 This is a flowchart of a wastewater treatment anomaly early warning method based on big data analysis, according to an embodiment of the present invention. Detailed Implementation

[0050] This invention discloses a wastewater treatment anomaly early warning method based on big data analysis, referring to... Figure 1 This includes steps S1-S4:

[0051] S1: Obtain historical operation datasets, where each sample in the historical operation dataset contains several types of wastewater treatment operation data segments for a specific time period.

[0052] Specifically, wastewater treatment operation data of various types at various times are obtained, and the wastewater treatment operation data of various types at all times are arranged in time sequence to obtain the time sequence of wastewater treatment operation data of various types.

[0053] The time series of various types of wastewater treatment operation data is evenly divided into data segments of a preset length and denoted as wastewater treatment operation data segments. All types of wastewater treatment operation data segments in the next time period of any given time period are taken as the observation data segments of that time period in the future. All types of wastewater treatment operation data segments in that time period with observation data segments are taken as a sample. The dataset composed of all samples is denoted as the historical operation dataset.

[0054] The types of wastewater treatment operation data include, but are not limited to, the following:

[0055] Equipment switch status data, equipment current data, equipment power data, reagent dosing frequency data, reagent dosing amount data, mixed liquid suspended solids concentration, oxidation-reduction potential, pH value, dissolved oxygen data, water temperature, ammonia nitrogen concentration data, and phosphorus concentration data.

[0056] S2: Using various types of wastewater treatment operation data as nodes, construct directed edges between nodes and their corresponding weights based on the correlation between different types of wastewater treatment operation data to obtain a causal relationship structure graph; calculate the theoretical sensitivity between two nodes based on the causal relationship structure graph, whereby the theoretical sensitivity is positively correlated with the weight of the directed edge of the path between the two nodes.

[0057] S20: Using various types of wastewater treatment operation data as nodes, construct directed edges between nodes and the weights of corresponding directed edges based on the correlation between different types of wastewater treatment operation data to obtain a causal relationship structure graph.

[0058] It should be noted that there are a series of causal relationships in the wastewater treatment process. For example, reducing the frequency of aeration blowers → reducing the aeration intensity of the biological treatment tank → decreasing the dissolved oxygen (DO) concentration → inhibiting the activity of aerobic microorganisms → increasing ammonia nitrogen (DO). Reduced degradation efficiency leads to excessive ammonia nitrogen in effluent. Traditional LSTM algorithms often predict wastewater treatment data by fitting patterns of change, failing to effectively learn causal relationships. This results in an inability to predict subsequent data changes when equipment or processes are altered, hindering timely warnings of wastewater treatment anomalies. To enable LSTM algorithms to learn these causal relationships, existing experience needs to be used to extract this information, guiding the algorithm in learning these relationships.

[0059] It should be further explained that graph structures can better reflect the causal relationships between events, so the following will use graph structures to describe this causal relationship information.

[0060] Preferably, as an example, various types of wastewater treatment operation data are used as nodes. Directed edges and their corresponding weights are constructed between nodes based on the correlation between different types of wastewater treatment operation data, resulting in a causal relationship structure graph, including:

[0061] By using various types of wastewater treatment operation data as nodes, directed edges are constructed between nodes corresponding to different types of wastewater treatment data based on the process flow.

[0062] The time series of wastewater treatment operation data of each node at all times corresponding to each node is denoted as the wastewater treatment operation data time series of each node; the misalignment time of the wastewater treatment operation data time series of two adjacent nodes is calculated.

[0063] The difference between each data point and the previous data point in the time series of wastewater treatment operation data for each node is taken as the change of each data point; the change of any data point in the time series of wastewater treatment operation data for the next node among two adjacent nodes is divided by the change of the data point at the corresponding time under the misalignment duration in the time series of wastewater treatment data for the previous node and the sum of the zero-prevention parameter, which is taken as the change sensitivity of that data point; the average change sensitivity of all data points in the time series of wastewater treatment operation data for the next node among two adjacent nodes is taken as the change sensitivity of the two adjacent nodes.

[0064] The change sensitivity of two adjacent nodes is used as the weight of the directed edge corresponding to the two adjacent nodes; the resulting graph structure is denoted as the causal relationship structure graph.

[0065] To make it easier to understand, a specific example will be used below:

[0066] Assume one of the process flow chains is: aeration blower frequency reduced → aeration intensity in biological treatment tank decreased → dissolved oxygen (DO) concentration decreased → aerobic microbial activity inhibited → ammonia nitrogen (DO) concentration decreased. Reduced degradation efficiency leads to excessive ammonia nitrogen levels in effluent.

[0067] The frequency data of the aeration blower, the aeration intensity data of the biological treatment tank, the dissolved oxygen concentration data, the aerobic microbial activity data, and the ammonia nitrogen concentration data are respectively used as nodes 1, 2, 3, 4, and 5. Based on this process flow chain, directed edges are drawn to the five nodes to obtain node 1 → node 2 → node 3 → node 4 → node 5.

[0068] Taking nodes 2 and 3 as examples, assume that the time series sequence of aeration intensity data obtained by node 2 is: [1.7,1.9,1.8,0.8,0.7,0.9,0.8,1.0,0.9,1.1], and the time series sequence of dissolved oxygen concentration data obtained by node 3 is [3.4,3.6,3.5,3.7,3.6,2.2,2.1,2.3,2.2,2.4]. Assume that the calculated misalignment time is 2 moments.

[0069] The changes in aeration intensity data over time are calculated as follows: [0.2, -0.1, -1.0, -0.1, 0.2, -0.1, 0.2, -0.1, 0.2]. The changes in dissolved oxygen concentration data over time are calculated as follows: [0.2, -0.1, 0.2, -0.1, -1.4, -0.1, 0.2, -0.1, 0.2]. The sensitivity of each data point in the dissolved oxygen concentration data over time is calculated as -0.200, 1.010, 0.995, 1.010, -6.965, 1.010, and 0.995. The mean of the sensitivity of all data points in the dissolved oxygen concentration data over time, -0.306, is used as the sensitivity of nodes 2 and 3. Here, the calculation of sensitivity is illustrated using a zero-prevention parameter of 0.001 as an example. Other values ​​for the zero-prevention parameter are also possible, and this embodiment does not impose specific limitations.

[0070] Understandably, directed edges between nodes are constructed based on the propagation relationships of the process flow, allowing the causal relationship structure diagram to contain process flow information. Furthermore, analyzing the duration of misalignment reflects the temporal misalignment of different data, providing a foundation for subsequent analysis of the correlation between different nodes based on the correlation of time-aligned data. For example, the dissolved oxygen concentration decreases only after the aeration fan frequency is reduced by 2 seconds, meaning the characteristic of inhibited aerobic biological activity lags behind the characteristic of reduced aeration fan frequency. Therefore, time-alignment processing of the dissolved oxygen concentration data and the aeration fan frequency data is necessary to accurately analyze the correlation between these two characteristic data. Further, the sensitivity between two nodes is reflected by analyzing the change in data at the corresponding time moment of the next node caused by a unit change in the data of the previous node. The larger this value, the greater the change in data at the corresponding time moment of the next node caused by a unit change in the data of the previous node, and thus the greater the sensitivity between the two nodes.

[0071] It should be added that the calculation of the misalignment duration of the time series of wastewater treatment operation data between two adjacent nodes includes:

[0072] The time series of sewage treatment operation data of the preceding node relative to the following node in two adjacent nodes is shifted by one time step. After each shift, the correlation of the time series of sewage treatment operation data of the two adjacent nodes after the shift is calculated. The number of shifts corresponding to the maximum correlation value is taken as the shift duration of the time series of sewage treatment operation data of the two adjacent nodes.

[0073] To make it easier to understand, here is a specific example:

[0074] Taking nodes 2 and 3 as examples, assuming the aeration intensity data time series obtained by node 2 is [1.7, 1.9, 1.8, 0.8, 0.7, 0.9, 0.8, 1.0, 0.9, 1.1], and the dissolved oxygen concentration data time series obtained by node 3 is [3.4, 3.6, 3.5, 3.7, 3.6, 2.2, 2.1, 2.3, 2.2, 2.4], the absolute values ​​of the Pearson correlation coefficients of the two overlapping time series obtained by shifting the aeration intensity data time series obtained by node 2 by 1, 2, 3, 4, 5, 6, and 7 time points are 0.12, 0.98, 0.05, 0.18, 0.23, 0.11, and 0.08, respectively. The correlation is greatest after shifting the aeration intensity data time series by 2 time points, so 2 is taken as the shift duration between nodes 2 and 3.

[0075] It should be noted that after the misalignment, there may be some time intervals where the data does not overlap. Therefore, it is only necessary to calculate the correlation of the sequence formed by the data at the overlapping time intervals. Furthermore, since the data is limited, the calculated correlation value will be less accurate. Therefore, if the number of data at the overlapping time intervals is less than a preset number, the calculated correlation value will no longer be considered.

[0076] S21: Calculate the theoretical sensitivity between two nodes based on the causal relationship structure graph. The theoretical sensitivity is positively correlated with the weight of the directed edge of the path between the two nodes.

[0077] Preferably, as an example, the theoretical sensitivity between two nodes is calculated based on the causal relationship structure graph, including:

[0078] In the causal relationship structure graph, obtain all paths between two nodes, and take the product of the weights of all directed edges on any path between the two nodes as the theoretical sensitivity of that path.

[0079] The theoretical sensitivity between the two nodes is the sum of the theoretical sensitivities of all paths between them.

[0080] It is understandable that there are multiple paths connecting two nodes, and each path can correlate the change information of the two nodes. Therefore, it is necessary to analyze the correlation information on all the connection paths between the two nodes to analyze the theoretical sensitivity of the two nodes.

[0081] S3: Construct a wastewater treatment index prediction network based on the LSTM algorithm. Input all types of wastewater treatment operation data segments from each sample in the historical operation dataset into the wastewater treatment index prediction network, obtain the output data of the corresponding type of each node, and calculate the fitting sensitivity of the two nodes based on the change correlation between the output data of the type corresponding to the latter node and the output data of the type corresponding to the former node; causal correlation loss function, which is positively correlated with the difference between the fitting sensitivity and the theoretical sensitivity.

[0082] It should be noted that in order to predict the sewage treatment status of the sewage treatment system in the future, it is necessary to first construct a sewage treatment indicator prediction network for predicting the future status.

[0083] Preferably, as an example, a wastewater treatment indicator prediction network is constructed based on the LSTM algorithm, including:

[0084] The network constructed using the LSTM algorithm is used as a prediction network for wastewater treatment indicators.

[0085] S30: Construct a wastewater treatment index prediction network based on the LSTM algorithm.

[0086] It should be noted that, in order for the wastewater treatment index prediction network to learn the causal relationship information of each stage in the wastewater treatment process, a loss function can be designed to supervise the wastewater treatment index prediction network to learn the relationship information between nodes in the causal relationship network.

[0087] S31: Input all types of wastewater treatment operation data segments from each sample in the historical operation dataset into the wastewater treatment index prediction network, obtain the output data of each type corresponding to each node, and calculate the fitting sensitivity of the two nodes based on the correlation between the output data of the type corresponding to the latter node and the output data of the type corresponding to the former node.

[0088] It should be noted that when designing the loss function, if the causal relationships between the nodes fitted by the wastewater treatment index prediction network are similar to the causal relationships obtained based on experience, it indicates that the wastewater treatment index prediction network has learned the correlation information of each stage in the wastewater treatment process. Therefore, the loss function can be designed based on this logic. First, we analyze the causal relationships between the nodes fitted by the wastewater treatment index prediction network.

[0089] Preferably, as an example, all types of wastewater treatment operation data segments from each sample in the historical operation dataset are input into the wastewater treatment index prediction network. Output data for each node is obtained, and the fitting sensitivity of the two nodes is calculated based on the correlation between the output data of the latter node and the output data of the former node. This includes:

[0090]

[0091] in, This represents the partial derivative of the output data of the type corresponding to the j-th node at time h. This represents the partial derivative of the output data of the type corresponding to the i-th node at time k. This represents the time-correspondence weight between the output data of the j-th node at time h and the output data of the i-th node at time k. Indicates the number of moments in the output data. Indicates the zero-prevention parameter. This represents the fitting sensitivity between the i-th node and the j-th node.

[0092] To make it easier to understand, a specific example will be used below:

[0093] The wastewater treatment operation data segments of all types for the corresponding time period are input into the wastewater treatment index prediction network. Assume the i-th node corresponds to aeration intensity, with the output data segment for aeration intensity being [1.7, 1.9, 1.8, 0.8, 0.7], and the j-th node corresponds to dissolved oxygen concentration, with the output data segment for dissolved oxygen concentration being [3.6, 1.8, 1.4, 1.6, 1.5]. The partial derivatives of all time-time data in the output data segment [1.7, 1.9, 1.8, 0.8, 0.7] for the i-th node are calculated to be 0.2, -0.1, -1.0, and -0.1, respectively. The partial derivatives of all time-time data in the output data segment [3.6, 1.8, 1.4, 1.6, 1.5] for the j-th node are calculated to be -1.8, -0.4, 0.2, and -0.1, respectively. Assume the time weights for every two time-time intervals between the i-th and j-th nodes are as follows: , , , , , , , , , , , , , , , Based on the partial derivatives of the output data segments of the corresponding types of the two nodes at each time step and the time weights corresponding to each pair of time steps, the fitting sensitivity calculated according to the fitting sensitivity calculation formula is 0.757.

[0094] Understandable This reflects the correlation between the data of the j-th node corresponding to the type at time h and the data of the i-th node corresponding to the type at time k, as fitted by the wastewater treatment index prediction network. A larger value indicates that the change in the data of the i-th node at time k significantly influences the change in the j-th node at time t, thus indicating a higher sensitivity between the i-th and j-th nodes. Since there is a temporal misalignment between the data of the i-th and j-th node corresponding types, the sensitivity of their changes is also related to the temporal correspondence. Therefore, when analyzing the sensitivity of changes between different nodes fitted by the wastewater treatment index prediction network, the temporal correspondence must be considered. Therefore, by introducing... This reflects the correspondence between time periods. This reflects the sensitivity of changes between the i-th node and the j-th node fitted by the wastewater treatment index prediction network.

[0095] It should be added that the methods for obtaining the time-related weights include:

[0096] The variance of the correlation calculated after all times of misalignment is denoted as the reference variance. A Gaussian distribution function is constructed with the misalignment duration as the mean and the reference variance as the variance. The time interval between the corresponding time of each data type of the i-th node and the corresponding time of each data type of the j-th node is calculated. The value at the time interval length in the Gaussian distribution function is denoted as the distribution weight.

[0097] The normalized value of the distribution weights is used as the weight corresponding to time.

[0098] To make it easier to understand, a specific example will be used below:

[0099] Assuming the misalignment time between node i and node j is 2, and the variance of the correlation calculated after all subsequent misalignments of node i and node j by one time step is 0.1073, a Gaussian function with a mean of 2 and a variance of 0.1073 is obtained. Based on the time intervals between the corresponding time steps of data of the type corresponding to node i and the corresponding time steps of data of the type corresponding to node j (0, 1, 2, 3, -1, 0, 1, 2, -2, -1, 0, 1, -3, -2, -1, 0), the normalized values ​​of the data at the corresponding positions in the Gaussian function for each time interval length are: , , , , , , , , , , , , , , , .

[0100] S31: Causal correlation loss function.

[0101] It should be noted that in order for the wastewater treatment index prediction network to learn the causal relationship information between various stages of wastewater treatment, the fitting sensitivity generated by the wastewater treatment index network should be close to the theoretical sensitivity. This will ensure that the data fitted by the wastewater treatment index network conforms to this causal relationship information, thereby promoting the wastewater treatment index prediction network to learn the causal relationship information between various stages of wastewater treatment.

[0102] Preferably, as an example, the causal association loss function includes:

[0103]

[0104] in, This represents the fitting sensitivity between the i-th node and the j-th node. This represents the theoretical sensitivity of the i-th node and the j-th node. This represents the number of successor nodes of the i-th node. Indicates the number of predecessor nodes. This represents the loss function for causal association.

[0105] It is understandable that by analyzing the difference between the theoretical sensitivity and the fitted sensitivity, the loss function is set so that it reflects the difference between the theoretical sensitivity and the fitted sensitivity. Furthermore, by minimizing the value of the loss function, the data fitted by the wastewater treatment index network can be made to conform to the causal relationship information obtained based on experience, thereby enabling the wastewater treatment index network to learn the causal relationship information obtained based on experience.

[0106] It should be noted that the predecessor node is the node before each node, and the successor node is the node after each node.

[0107] S4: The wastewater treatment indicator prediction network is trained based on the causal correlation loss function and the loss function built into the LSTM algorithm to achieve early warning of wastewater treatment anomalies.

[0108] S40: The wastewater treatment index prediction network is trained based on the causal correlation loss function and the loss function built into the LSTM algorithm.

[0109] Preferably, as an example, the training of the wastewater treatment indicator prediction network is completed based on the causal association loss function and the loss function built into the LSTM algorithm, including:

[0110] The sum of the causal correlation loss function and the loss function built into the LSTM algorithm is used as the comprehensive loss function.

[0111] Each sample is input into the wastewater treatment index prediction network. The learning path loss value is calculated based on the method in steps S2 to S3. The predicted loss value is calculated based on the loss function of the LSTM algorithm. The learning path loss value and the predicted loss value are summed to obtain the comprehensive loss value.

[0112] Based on the comprehensive loss value, the parameters in the wastewater treatment index prediction network are updated in reverse using the gradient descent method. The training of the wastewater treatment index prediction network is completed until the comprehensive loss value converges.

[0113] It should be noted that the methods of using the loss function built into the LSTM algorithm to calculate the predicted loss value and using gradient descent to update the parameters in the wastewater treatment index prediction network based on the loss value are existing techniques, which will not be elaborated here.

[0114] S41: To achieve early warning of abnormalities in wastewater treatment.

[0115] Preferably, as an example, to implement abnormal early warning in wastewater treatment, the following are included:

[0116] The newly collected wastewater treatment operation data segments of several types in the current time period are input into the trained wastewater treatment index prediction network to obtain wastewater treatment operation data of various types at future time. If there is a data in the wastewater treatment operation data of various types at future time that meets the abnormal warning conditions, an abnormal warning is issued.

[0117] If all wastewater treatment operation data of various types do not meet the abnormal warning conditions at any future time, no warning will be issued.

[0118] It should be noted that the abnormal warning condition can be that the wastewater treatment operation data is not within the preset normal range. Other conditions are also acceptable, but no specific restrictions are imposed here.

[0119] This invention also discloses a wastewater treatment anomaly early warning system based on big data analysis, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the wastewater treatment anomaly early warning method based on big data analysis according to this invention is implemented.

[0120] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0121] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (DRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (DRAM), high-bandwidth memory, hybrid memory cube, etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device.

Claims

1. A sewage treatment abnormality early warning method based on big data analysis, characterized in that, The method comprises the steps of: obtaining a historical operation data set, wherein each sample in the historical operation data set comprises a plurality of types of sewage treatment operation data segments of a time period; taking each type of sewage treatment operation data as a node, and constructing a directed edge between the nodes and a weight corresponding to the directed edge according to the relevance of different types of sewage treatment operation data, to obtain a causal relationship structure diagram; calculating a theoretical sensitivity between two nodes based on the causal relationship structure diagram, wherein the theoretical sensitivity is positively correlated with the weight of the directed edge of the path between the two nodes; constructing a sewage treatment index prediction network based on an LSTM algorithm, inputting all types of sewage treatment operation data segments in each sample in the historical operation data set into the sewage treatment index prediction network, obtaining output data of each node corresponding to the type, and calculating a fitting sensitivity of the two nodes based on the change relevance of the output data of the latter node corresponding to the type and the output data of the former node corresponding to the type; constructing a causal correlation loss function, wherein the causal correlation loss function is positively correlated with the difference between the fitting sensitivity and the theoretical sensitivity; completing training of the sewage treatment index prediction network based on the causal correlation loss function and a loss function provided by the LSTM algorithm, to realize sewage treatment anomaly early warning; taking each type of sewage treatment operation data as a node, and constructing a directed edge between the nodes and a weight corresponding to the directed edge according to the relevance of different types of sewage treatment operation data, to obtain a causal relationship structure diagram, calculating a misplacement time length of the sewage treatment operation data time sequence of two adjacent nodes, comprising: sequentially misplacing the sewage treatment operation data time sequence of the former node relative to the latter node in the two adjacent nodes by one time point, calculating the correlation of the sewage treatment operation data time sequence of the two adjacent nodes after misplacing each time point by one time point, and obtaining the misplacement time length of the sewage treatment operation data time sequence of the two adjacent nodes as the number of misplacement time points corresponding to the maximum correlation. calculating a change sensitivity of the directed edge corresponding to the two adjacent nodes based on the time sequence relationship of the sewage treatment operation data time sequence of the two adjacent nodes under the misplacement time length, comprising: taking the difference between each data and the previous data in the sewage treatment operation data time sequence of each node as the change amount of each data; dividing the change amount of any data in the sewage treatment operation data time sequence of the latter node in the two adjacent nodes by the cumulative sum of the change amount of the corresponding time point data in the sewage treatment data time sequence of the former node under the misplacement time length and the anti-zero parameter as the change sensitivity of the data; and taking the average of the change sensitivities of all data in the sewage treatment operation data time sequence of the latter node as the change sensitivity of the two adjacent nodes.

2. The method of claim 1, wherein the method is characterized by, The method comprises the steps of: obtaining a historical operation data set, wherein each sample in the historical operation data set comprises a plurality of types of sewage treatment operation data segments of a time period; taking each type of sewage treatment operation data as a node, and constructing a directed edge between the nodes and a weight corresponding to the directed edge according to the relevance of different types of sewage treatment operation data, to obtain a causal relationship structure diagram, calculating a theoretical sensitivity between two nodes based on the causal relationship structure diagram, wherein the theoretical sensitivity is positively correlated with the weight of the directed edge of the path between the two nodes; constructing a sewage treatment index prediction network based on an LSTM algorithm, inputting all types of sewage treatment operation data segments in each sample in the historical operation data set into the sewage treatment index prediction network, obtaining output data of each node corresponding to the type, and calculating a fitting sensitivity of the two nodes based on the change relevance of the output data of the latter node corresponding to the type and the output data of the former node corresponding to the type; constructing a causal correlation loss function, wherein the causal correlation loss function is positively correlated with the difference between the fitting sensitivity and the theoretical sensitivity; completing training of the sewage treatment index prediction network based on the causal correlation loss function and a loss function provided by the LSTM algorithm, to realize sewage treatment anomaly early warning; taking each type of sewage treatment operation data as a node, and constructing a directed edge between the nodes and a weight corresponding to the directed edge according to the relevance of different types of sewage treatment operation data, to obtain a causal relationship structure diagram, calculating a misplacement time length of the sewage treatment operation data time sequence of two adjacent nodes, comprising: sequentially misplacing the sewage treatment operation data time sequence of the former node relative to the latter node in the two adjacent nodes by one time point, calculating the correlation of the sewage treatment operation data time sequence of the two adjacent nodes after misplacing each time point by one time point, and obtaining the misplacement time length of the sewage treatment operation data time sequence of the two adjacent nodes as the number of misplacement time points corresponding to the maximum correlation. calculating a change sensitivity of the directed edge corresponding to the two adjacent nodes based on the time sequence relationship of the sewage treatment operation data time sequence of the two adjacent nodes under the misplacement time length, comprising: taking the difference between each data and the previous data in the sewage treatment operation data time sequence of each node as the change amount of each data; dividing the change amount of any data in the sewage treatment operation data time sequence of the latter node in the two adjacent nodes by the cumulative sum of the change amount of the corresponding time point data in the sewage treatment data time sequence of the former node under the misplacement time length and the anti-zero parameter as the change sensitivity of the data; and taking the average of the change sensitivities of all data in the sewage treatment operation data time sequence of the latter node as the change sensitivity of the two adjacent nodes. The method comprises the steps of: obtaining a historical operation data set, wherein each sample in the historical operation data set comprises a plurality of types of sewage treatment operation data segments of a time period; taking each type of sewage treatment operation data as a node, and constructing a directed edge between the nodes and a weight corresponding to the directed edge according to the relevance of different types of sewage treatment operation data, to obtain a causal relationship structure diagram, calculating a theoretical sensitivity between two nodes based on the causal relationship structure diagram, wherein the theoretical sensitivity is positively correlated with the weight of the directed edge of the path between the two nodes; constructing a sewage treatment index prediction network based on an LSTM algorithm, inputting all types of sewage treatment operation data segments in each sample in the historical operation data set into the sewage treatment index prediction network, obtaining output data of each node corresponding to the type, and calculating a fitting sensitivity of the two nodes based on the change relevance of the output data of the latter node corresponding to the type and the output data of the former node corresponding to the type; constructing a causal correlation loss function, wherein the causal correlation loss function is positively correlated with the difference between the fitting sensitivity and the theoretical sensitivity; completing training of the sewage treatment index prediction network based on the causal correlation loss function and a loss function provided by the LSTM algorithm, to realize sewage treatment anomaly early warning; 3.The method of claim 1, wherein, The fitting sensitivity of the two nodes is calculated based on the change correlation of the output data of the corresponding type of the latter node and the output data of the corresponding type of the former node, and the fitting sensitivity of the two nodes is calculated, comprising: wherein, represents the partial derivative of the output data of the jth node corresponding to the type at the hth time point, represents the partial derivative of the output data of the ith node corresponding to the type at the kth time point, represents the time corresponding weight of the output data of the jth node corresponding to the type at the hth time point and the output data of the ith node corresponding to the type at the kth time point, represents the number of time points of the output data, represents the zero prevention parameter, represents the fitting sensitivity of the ith node and the jth node. 4.The method of claim 3, wherein, The method for obtaining the time corresponding weight comprises: The variance of the correlation calculated at all times after the wrong position is calculated as a reference variance, a Gaussian distribution function is constructed with the wrong position time length as the mean and the reference variance as the variance, the time interval length of the corresponding time of each data of the i th node and the corresponding time of each data of the j th node is calculated, and the value of the Gaussian distribution function is obtained as the distribution weight; The normalized value of the distribution weight is taken as the time corresponding weight. 5.The method of early warning of sewage treatment abnormality based on big data analysis according to claim 1, characterized in that, The construction of the causal correlation loss function comprises: wherein, represents the fitted sensitivity of the i-th node to the j-th node, represents the theoretical sensitivity of the i-th node to the j-th node, represents the number of successor nodes of the i-th node, represents the number of predecessor nodes, represents the causal association loss function. 6.The method of early warning of sewage treatment abnormality based on big data analysis according to claim 1, characterized in that, The implementation of the sewage treatment abnormality early warning comprises: The newly collected sewage treatment operation data of several types of the current time period is input into the trained sewage treatment index prediction network to obtain various types of sewage treatment operation data at future times, if there is a data in the various types of sewage treatment operation data at future times that meets the abnormality early warning condition, an abnormality early warning is issued; If all the various types of sewage treatment operation data at future times do not meet the abnormality early warning condition, no early warning is performed.

7. A sewage treatment abnormality early warning system based on big data analysis, characterized by Comprise: A processor and a memory, the memory stores computer program instructions, when the computer program instructions are executed by the processor, the method for early warning of sewage treatment abnormality based on big data analysis according to any one of claims 1-6 is realized.

Images