Water source water quality monitoring method and system based on K-means clustering and BP neural network

Abstract

The invention discloses a K-means clustering and BP neural network-based water source water quality monitoring method and system. The method comprises the steps of obtaining historical water quality data of all water source regions in a monitoring region; processing the historical water quality data by using a K-means clustering algorithm, dividing feature categories of abnormal water quality data and normal water quality data according to a clustering processing result, and respectively marking the feature categories of the abnormal water quality data and the normal water quality data; inputting the historical water quality data with marks after K-means clustering processing into a BP neural network for training to obtain a water quality detection model; current water quality data is collected and input into the water quality detection model, and the water quality detection model outputs the feature category of the current water quality data. The characteristic that the K-means clustering algorithm can accurately classify the data is combined with the BP neural network, repeated clustering processing does not need to be carried out on the water quality data, and rapid detection of the subsequent water quality data can be directly achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of water source monitoring, in particular to a water quality monitoring method and system based on K-means clustering and BP neural network. BACKGROUND

[0002] Traditional water quality detection is generally realized through manual sampling, laboratory analysis and other steps, which is low in efficiency and time-consuming and laborious. The rise of wireless sensor network technology provides a new way for water quality detection, and the abnormal data detection method based on wireless sensor network provides technical support for water quality anomaly detection. However, the existing wireless sensor abnormal water quality data detection algorithm often needs the prior distribution condition of data, and cannot directly detect unknown data collected by the water area to be measured. In addition, many sensor nodes will be deployed in large-area water detection, which will sense and collect many water quality data corresponding to parameters, which requires higher timeliness of the abnormal data detection algorithm. SUMMARY

[0003] To solve the above technical problems, the present application provides a water quality monitoring method and system based on K-means clustering and BP neural network, which aims to solve the technical problem of low water quality detection accuracy caused by complex and variable water source distribution and difficulty in obtaining stable prior water quality data.

[0004] In order to achieve the above-mentioned purpose of the application, the present application provides a water quality monitoring method based on K-means clustering and BP neural network, which comprises:

[0005] S101, obtaining historical water quality data of all water source areas in a monitoring area;

[0006] S102, processing the historical water quality data by using a K-means clustering algorithm, and dividing and marking the feature categories of abnormal water quality data and normal water quality data according to the clustering processing result;

[0007] S103, inputting the historical water quality data with marks after K-means clustering processing into a BP neural network for training to obtain a water quality detection model;

[0008] S104, collecting current water quality data and inputting them into the water quality detection model, and the water quality detection model outputs the feature category of the current water quality data.

[0009] Further, the method further comprises:

[0010] Arranging a water quality wireless sensor network in all water source areas in the monitoring area for collecting historical water quality data and current water quality data of all water source areas.

[0011] Further, the historical water quality data is processed by using the K-means clustering algorithm, and the characteristic categories of the abnormal water quality data and the normal water quality data are divided and marked according to the clustering processing result, and specifically includes:

[0012] The historical water quality data with the time sequence length h is intercepted, and the historical water quality data is processed by using the missing data smoothing method and the normalization processing;

[0013] The distance of each historical water quality data to the initial clustering center is calculated, each historical water quality data is divided into the cluster with the minimum distance, and iteration is performed;

[0014] According to the clustering processing result, the cluster with the minimum data amount is defined as an abnormal cluster, and the normal cluster and the abnormal cluster are marked respectively, so that the characteristic categories of the abnormal water quality data and the normal water quality data are obtained.

[0015] Specifically, the method includes:

[0016] Supposing that the historical water quality data X collected by the water quality wireless sensor network of all water source areas in a monitoring area m×n , wherein X m×n = [X 1,n , X 2,n , …, X m,n ]; X m×n is the water quality data collected by m sensor nodes at n time points, and the historical water quality data X m×n is collected and transmitted in the form of a matrix;

[0017] The step of processing the historical water quality data by using the K-means clustering algorithm specifically includes the following steps:

[0018] Supposing that the historical water quality data X m×n is selected, k historical water quality data are selected as initial clustering centers C, C = [C1, C2, …, C k ]; if k classes are formed, each C s is an n-dimensional vector;

[0019] The Euclidean distance of all historical water quality data to the initial clustering center is calculated:

[0020]

[0021] X i,n is the n-time water quality data of the i-th sensor node;

[0022] C s,n is the n-time water quality data of the s-th initial clustering center;

[0023] dist(.) is the Euclidean distance;

[0024] The water quality data X collected by each sensor node i at time n i,n According to the distance size, the historical water quality data in the cluster is divided into a corresponding category, and the average value of the Euclidean distance of the historical water quality data in the cluster is calculated According to The initial clustering center is updated;

[0025]

[0026] Wherein, S s is the historical water quality data set of the s-th cluster, |S s | is the cluster size;

[0027] The above steps are repeatedly repeated until the clustering center point is no longer offset, and the iteration is stopped; for each sensor node i, the distance of the sensor node i to all C s is calculated, and the sensor node i is divided into the nearest cluster;

[0028] The historical water quality data of each category after K-means clustering processing is marked respectively, such as formula:

[0029]

[0030] The historical water quality data is expressed as X m×n =[X 1，n , X 2，n , …, X m，n , y ，i ]; y i is the label of the i-th water quality sample.

[0031] Preferably, the historical water quality data with a time series length of h is intercepted, and after the missing data smoothing method processing and normalization processing of the historical water quality data, it further includes:

[0032] The selection of the initial clustering center C is optimized, and the specific process is as follows:

[0033] S201, for each historical water quality data x m×n in the historical water quality data X i , the space-parameter joint distance is calculated:

[0034]

[0035] Wherein, x i is the historical water quality data of the i-th sensor node; ω is the parameter distance weight; d 空间 is the geographical Euclidean distance of the sensor node i; d 参数 is the parameter Euclidean distance of two historical water quality data;

[0036] Let two historical water quality data x i = (x i1 , x i2 ,..., x id ), x j = (x j1 , x j2 ,..., x jd ), each historical water quality data x i contains d-dimensional data, then the parameter Euclidean distance of two historical water quality data is the square root of the sum of squares of the differences of each dimension, and the formula is:

[0037]

[0038] Let the sensor node coordinates (lon i , lat i ), and the spatial parameters include 2-dimensional data, then the geographical Euclidean distance of two historical water quality data is the square root of the sum of squares of the differences of two dimensions, and the formula is:

[0039]

[0040] S202, density peak screening:

[0041] S301, calculate the local density p i of each point:

[0042] For historical water quality data x i , its local density p i represents the density of data points in the neighborhood, and is obtained by weighted summation of joint distance d ij through Gaussian kernel function:

[0043]

[0044] Where, σ is the bandwidth parameter of Gaussian kernel;

[0045] S302, calculate the distance parameter d i :

[0046] The d i of high-density points: take the minimum joint distance from this point to all high-density points whose local density is greater than p i ;

[0047] d i = min{d ij | p j > p i , i≠j}

[0048] The d i of low-density points: take the maximum joint distance from this point to all data points;

[0049] δ i =max{d ij |j=1,2,3,...,m×n,i≠j}

[0050] With the horizontal axis as ρ i The vertical axis is δ i Construct a decision graph and select K high-ρ points based on the decision graph. i And high δ i The data is used as the initial cluster centers.

[0051] Furthermore, before inputting the labeled historical water quality data after K-means clustering into the BP neural network for training to obtain the water quality detection model, the method further includes:

[0052] Set the parameters of the BP neural network and build the BP neural network.

[0053] Furthermore, the step of inputting labeled historical water quality data after K-means clustering into a BP neural network for training to obtain a water quality detection model specifically includes:

[0054] Historical water quality data X obtained through K-means clustering m×n =[X 1，n X 2，n , ..., X m，n y ，i Rewritten as a water quality dataset D = [(X 1，n y i ), (X 2，n y i ), ..., (X m，n y i The labeled water quality dataset D is split into a training set, a validation set, and a test set; where X∈R p y i ∈R q p is the dimension of the input vector, where the input feature vector X belongs to a p-dimensional real space; q is the dimension of the output vector, where the output label vector y is the dimension of the output vector. i If the value belongs to the q-dimensional real space, then the calculation steps of the water quality detection model are as follows:

[0055] (1) Input layer → Hidden layer

[0056] The labeled historical water quality data samples are imported into the initial BP network model. The input representation of the historical water quality data vector at the j-th neuron is as follows:

[0057]

[0058] Where, ωi,j is the weight of the input layer i-th neuron, the hidden layer j-th neuron; b j is the bias of the hidden layer j-th neuron;

[0059] a j is the input of the hidden layer, after the activation function sigmoid function processing to get the output of the hidden layer, sigmoid function compression output (value range (0, 1)):

[0060] As formula:

[0061]

[0062] (2) hidden layer → output layer

[0063] The output layer neuron is passed to the output layer neuron after the hidden layer gets the output result, at this time the input value of the output layer neuron, as formula:

[0064]

[0065] Among them, v j,s is the weight of the hidden layer j-th neuron, the output layer s-th neuron; b s is the bias of the output layer s-th neuron;

[0066] The predicted value of the output layer is calculated, and sigmoid (output abnormal probability) is used for binary classification:

[0067]

[0068] The weighted sum of the output layer and β s is compressed by the sigmoid function to get the predicted value of water quality detection (such as abnormal probability).

[0069] (3) back propagation optimization (parameter adjustment)

[0070] The process of BP neural network forward transmission, the predicted value and the mean square error of the real value D s is as follows:

[0071]

[0072] BP neural network adjusts the weights ω i,j , v j,s and the threshold b j , b s by gradient descent in reverse, the goal is to minimize the error E; when the error E is less than the set threshold, stop iteration, get the final water quality detection model.

[0073] Calculate the error E on the output layer weight vj,s and bias b s are adjusted with learning rate η:

[0074]

[0075] Error E is back-propagated to the hidden layer, and the gradients of ω i,j and b j are calculated and updated:

[0076]

[0077] Based on the same inventive purpose, the application further provides a water source water quality monitoring system based on K-means clustering and BP neural network, which comprises:

[0078] a monitoring server, and a plurality of water quality wireless sensors arranged in all water source areas in a monitoring area and connected with the monitoring server; the monitoring server and the plurality of water quality wireless sensors form a water quality wireless sensor network, and are used for executing the following steps:

[0079] S101, acquiring historical water quality data of all water source areas in a monitoring area;

[0080] S102, processing the historical water quality data by using a K-means clustering algorithm, and marking the feature categories of abnormal water quality data and normal water quality data according to a clustering processing result;

[0081] S103, inputting the historical water quality data after K-means clustering processing and marking into a BP neural network for training to obtain a water quality detection model;

[0082] S104, collecting current water quality data and inputting into the water quality detection model, and the water quality detection model outputs the feature category of the current water quality data.

[0083] Technical effects of the application:

[0084] The water source water quality monitoring method and system based on K-means clustering and BP neural network of the embodiments of the application combine the characteristics that the K-means clustering algorithm can accurately classify data with the BP neural network, first classify the historical water quality data by using the K-means clustering algorithm, mark the data of different categories with feature tags, realize the classification and marking of abnormal data and normal data, then train the BP neural network by using the data set with tags, and obtain the final abnormal water quality data detection model, which does not need to repeatedly cluster the water quality data, and can directly realize the rapid detection of subsequent water quality data. BRIEF DESCRIPTION OF DRAWINGS

[0085] In order to make the technical solutions of the embodiments of the present application or the prior art clearer, the accompanying drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the accompanying drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor.

[0086] Figure 1 A flow chart of a water quality monitoring method based on K-means clustering and BP neural network of an embodiment of the present application;

[0087] Figure 2 A flow chart of a BPNN abnormal water quality data detection algorithm based on K-means clustering of an embodiment of the present application;

[0088] Figure 3 A K-means clustering algorithm processing process (a)-(f) of an embodiment of the present application;

[0089] Figure 4 A BP neural network structure diagram of an embodiment of the present application;

[0090] Figure 5 An initial cluster center selection decision diagram of an embodiment of the present application;

[0091] Figure 6 A hardware structure flow chart of a water quality monitoring system based on K-means clustering and BP neural network of an embodiment of the present application. DETAILED DESCRIPTION

[0092] The specific embodiments of the present application will be further described below in conjunction with the accompanying drawings. It should be noted that the description of the embodiments is used to help understand the present application, but does not constitute a limitation on the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0093] Before the embodiments of the present application are further described in detail, the terms and terms involved in the embodiments of the present application are explained, which are applicable to the following explanations.

[0094] The K-means clustering algorithm is an unsupervised learning algorithm, which divides data points into K clusters by iteration, so that the similarity (such as Euclidean distance) of data points in the cluster is maximized, and the similarity between clusters is minimized. For example, Figure 3As shown, randomly select K initial cluster centers in water quality data, compare the distance of water quality data to the centroid and classify it, then update the cluster center point by calculating the average distance of water quality data in the cluster, and iterate until the final classification result meets the minimum distance of the data in the cluster and the maximum distance between clusters. As an unsupervised learning method in machine learning, K-means clustering does not require prior knowledge of data, and the calculation is simple, which meets the needs of abnormal water quality data detection. According to the definition of abnormal water quality data in wireless sensor network, the proportion of abnormal water quality data is small compared with normal water quality data, so when K-means clustering algorithm is applied to abnormal water quality data detection, we can identify the cluster with the least data in the cluster after classification as the abnormal cluster, and all water quality data in this cluster are abnormal water quality data.

[0095] Water Quality Wireless Sensor Network (WQ-WSN) is an intelligent monitoring system that collects, transmits and processes water quality parameters in real time through distributed sensor nodes, combining sensor technology, wireless communication technology and data processing technology, mainly used for water environment monitoring and management.

[0096] BPNN (Back Propagation Neural Network) is a supervised learning algorithm based on artificial neural network, also one of the most classic neural network models. It adjusts network weights and biases through back propagation algorithm to realize modeling of complex nonlinear relationship.

[0097] Although K-means clustering algorithm solves the problem of abnormal water quality data detection with unknown distribution, according to its calculation principle, direct application of this method to real-time sampling water quality WSN system will cause repeated iteration calculation of data, and further lead to low timeliness of detection. Therefore, the combination of K-means clustering and BP neural network will be used to realize efficient detection of water quality data. K-means clustering is mainly used to realize the normal and abnormal marking of historical water quality data, and the results of clustering are used as sample data set to effectively realize the modeling analysis of BP neural network.

[0098] Missing data smoothing method processing: missing data refers to the missing of part of the index value of some samples in the data set (such as the "dissolved oxygen" value of a sample in water quality monitoring is not recorded), if the sample containing missing value is directly deleted, it will lead to the reduction of data quantity, and the smoothing method fills the missing value reasonably, makes the data distribution more continuous and reduces the deviation. Based on the assumption of "local continuity of data", the effective data around the missing value (such as the adjacent samples of the same index, the mean value of the same category) is used for estimation, so as to avoid the "fault" of data distribution and make the filled data closer to the real distribution. For example, the adjacent value smoothing method fills the missing value with the mean / median of the adjacent samples before and after the missing value (for example, in time series, the missing value at t time is filled with the mean of the values at t-1 and t+1 times).

[0099] Normalization processing is to convert indexes with different dimensions and different numerical ranges to a unified order of magnitude (such as [0, 1] or [-1, 1]), so as to eliminate the "numerical dominant deviation" caused by the unit difference of indexes (such as the COD value range is 0-500 mg / L, and the pH value range is 0-14, so the weight of COD will be enlarged in direct calculation). Through linear or nonlinear conversion, the numerical distribution of all indexes is in the same interval, so as to ensure that all indexes have the same weight in subsequent analysis (such as distance calculation and model training), and avoid that a certain index affects the result due to the too large numerical range.

[0100] Embodiment 1

[0101] As shown in Figure 1 The embodiment of the present application provides a water quality monitoring method based on K-means clustering and BP neural network, which comprises the following steps:

[0102] S101, acquiring historical water quality data of all water source areas in a monitoring area;

[0103] S102, processing the historical water quality data by using a K-means clustering algorithm, dividing and marking the characteristic categories of abnormal water quality data and normal water quality data according to the clustering processing result;

[0104] S103, inputting the historical water quality data with marks after K-means clustering processing into a BP neural network for training to obtain a water quality detection model;

[0105] S104, collecting current water quality data and inputting the current water quality data into the water quality detection model, and the water quality detection model outputs the characteristic category of the current water quality data.

[0106] Optionally, the method further comprises:

[0107] Arranging a water quality wireless sensor network in all water source areas in the monitoring area, which is used for collecting historical water quality data and current water quality data of all water source areas.

[0108] Water quality data generally includes conventional five parameters (turbidity, dissolved oxygen, water temperature, conductivity, pH value), ammonia nitrogen, nitrate nitrogen, chlorophyll, total organic carbon (TOC) and the like. These parameters are divided into physical indicators, chemical indicators and biological indicators, wherein the physical indicators are sensory physical characteristics, including water turbidity, water temperature and color, etc. The chemical indicators are the content of chemical substances in the water area, which generally include pH value, organic / inorganic toxic substances, total organic carbon and chemical oxygen demand, etc. The biological indicators refer to the content of microorganisms contained in normal water quality. Bacteria and viruses present in the intestinal tract of humans and animals can spread with water flow, and many parasites can also multiply through water. Microorganisms in wastewater can also damage infrastructure life and cause immeasurable loss of life and property.

[0109] As shown in Figure 2 , 3 , the historical water quality data is processed using the K-means clustering algorithm, and the characteristic categories of abnormal water quality data and normal water quality data are divided according to the clustering processing result and are labeled respectively. Specifically, the method comprises:

[0110] The historical water quality data with a time sequence length of h is intercepted, and the historical water quality data is processed by a missing data smoothing method and normalized;

[0111] The distance of each historical water quality data to the initial clustering center is calculated, each historical water quality data is divided into the cluster with the smallest distance, and iteration is performed;

[0112] According to the clustering processing result, the cluster with the smallest data amount is defined as an abnormal cluster, and the normal cluster and the abnormal cluster are labeled respectively, so as to obtain the characteristic categories of abnormal water quality data and normal water quality data.

[0113] Specifically, the method comprises:

[0114] Suppose that the historical water quality data X collected by the water quality wireless sensor network of all water source areas in the monitoring area is m×n , wherein X m×n = [X 1，n , X 2，n , …, X m，n ]; X m×n is the water quality data collected by m sensor nodes at n time points, and the historical water quality data X m×n is collected and transmitted in the form of a matrix;

[0115] The step of processing the historical water quality data using the K-means clustering algorithm specifically comprises the following steps:

[0116] Suppose that the historical water quality data Xm×n Select k historical water quality data as the initial clustering center C, C = [C1, C2,..., Ck]; k If it is clustered into k classes, each C s is an n-dimensional vector (representing the "typical time series water quality characteristics" of a certain class of sensor nodes);

[0117] Calculate the Euclidean distance of all historical water quality data to the initial clustering center:

[0118]

[0119] X i,n is the n-time water quality data of the i-th sensor node, a 1-row n-column vector (such as the pH and dissolved oxygen sequence of sensor node i at time 1-n);

[0120] C s,n is the n-time water quality data of the s-th initial clustering center, a 1-row n-column "typical time series feature" (such as the average time series pattern of a certain class of sensor nodes);

[0121] dist(.) is the Euclidean distance: measures the "time series pattern difference" between the time series data of sensor node i and the initial clustering center s, the smaller the value, the more similar;

[0122] Each sensor node i collects n-time water quality data X i,n is divided into the corresponding category according to the distance size and the average value of the cluster historical water quality data Euclidean distance is calculated According to the initial clustering center is updated;

[0123]

[0124] Where S s is the historical water quality data set of the s-th cluster, |S s | is the cluster size; repeat "assignment-update" until the clustering center is stable (change less than threshold).

[0125] Repeat the above steps until the clustering center point no longer shifts, stop iteration; for each sensor node i, calculate its distance to all C s , and divide sensor node i into the nearest cluster;

[0126] Assume you want to do K-means clustering on the time series water quality data of sensor node i, as follows:

[0127] The historical water quality data X i,n of each sensor node i is a time series curve (n-time water quality indicators);

[0128] Initial cluster center C s,n is the typical time series curve of a certain type of sensor node i (such as the average pH change of "stable sensor" or the dissolved oxygen pattern of "fluctuating sensor").

[0129] Calculate dist(X i,n ,C s,n ), which is to quantify the difference between the "time series pattern of sensor node i" and the "typical pattern":

[0130] Small distance: the time series change of this sensor node is highly similar to the center of this cluster, and should be divided into this cluster.

[0131] Large distance: the time series pattern is quite different, and belongs to other clusters.

[0132] After K-means clustering processing, the historical water quality data of each category is marked, such as:

[0133]

[0134] The historical water quality data is expressed as X m×n =[X 1，n , X 2，n , …, X m，n , y ，i ]; y i is the label of the i-th water quality sample.

[0135] In the water quality monitoring wireless sensor network (WSN), the K-means clustering algorithm is prone to unstable clustering results or large errors due to random selection of initial clustering centers, insufficient use of spatial and temporal characteristics, and other problems. The specific reasons are as follows:

[0136] Spatial dimension: adjacent sensor nodes have similar geographical distances, and water quality parameters (such as pH and dissolved oxygen) have spatial correlation (such as parameter values converging in the same water area).

[0137] Temporal dimension: the water quality parameters of the same node change slowly over time (such as the influence of temperature difference on water temperature), but may be affected by sudden pollution events and show abnormal fluctuations.

[0138] Multi-parameter correlation: water quality monitoring usually includes temperature, turbidity, conductivity, ammonia nitrogen concentration, and other multi-dimensional data, and single parameter clustering cannot reflect the true water quality conditions.

[0139] Noise and outliers: sensor failure, water flow disturbance, or communication interference may cause data to jump (such as sudden increase in turbidity value), and K-means is sensitive to noise.

[0140] Therefore, the current K-means clustering has the following defects in the selection of initial clustering centers:

[0141] Initial cluster center randomness: Randomly selecting centers can lead to "empty clusters" or "class cluster imbalance". Water sources in monitoring areas are usually concentrated and uneven, mainly natural rivers, glacial meltwater, and groundwater, with uneven spatial distribution.

[0142] Ignoring spatial and temporal priors: Ignoring the geographical location and time series information of sensor nodes can misclassify data that is spatially close but has small parameter differences.

[0143] Low iteration efficiency: Noise data participates in cluster center updating, which can cause slow convergence or fall into local optimum.

[0144] Preferably, the historical water quality data with a time series length of h is intercepted, and after the missing data smoothing method and normalization processing, it further includes:

[0145] The selection of the initial clustering center C is optimized as follows:

[0146] S201, for each historical water quality data x m×n in the historical water quality data X i , the spatial-parameter joint distance is calculated:

[0147]

[0148] where x i is the historical water quality data of the i-th sensor node (including water quality parameters and sensor node coordinates (lon i , lat i )), ω is the parameter distance weight (determined by cross-validation, such as ω = 0.7 when the water quality parameter is important),

[0149] d 空间 is the geographical Euclidean distance of sensor node i.

[0150] Let two historical water quality data x i = (x i1 , x i2 ,..., x id ), x j = (x j1 , x j2 ,..., x jd ), each historical water quality data x i contains temperature, turbidity, conductivity, ammonia nitrogen concentration, etc. d-dimensional data, then the parameter Euclidean distance of the two historical water quality data is the square root of the square sum of the difference of each dimension, the formula is:

[0151]

[0152] Let sensor node coordinates (lon i , lat i ), spatial parameters include 2-dimensional data, the geographical Euclidean distance of two historical water quality data is the square root of the square sum of 2-dimensional difference, the formula is:

[0153]

[0154] S202, density peak screening:

[0155] S301, calculate the local density p i of each point: use the joint distance-based Gaussian kernel function to measure the density of data points in the neighborhood.

[0156] For historical water quality data x i , its local density p i represents the density of data points in the neighborhood, and the joint distance d ij is weighted and summed by the Gaussian kernel function to obtain:

[0157]

[0158] Where, sigma is the bandwidth parameter of the Gaussian kernel, which controls the neighborhood range (usually take the median of data distance or determine by cross-validation). If the data distribution is dense, reduce sigma; if sparse, increase sigma.

[0159] S302, calculate distance parameter d i : for high-density points, take the distance to the nearest high-density point; for low-density points, take the maximum distance to all points.

[0160] High-density points: if the local density p i of historical water quality data x i is greater than or equal to the median of p i in the data set (or a custom threshold), it is considered a high-density point.

[0161] Low-density points: otherwise, it is a low-density point.

[0162] The d i of high-density points: take the minimum joint distance of the point to all high-density points with local density greater than p i .

[0163] d i = min{d ij | p j > p i , i≠j}

[0164] The d i of low-density points: take the maximum joint distance of the point to all data points.

[0165] delta i = max{d ij |j = 1, 2, 3,..., m x n, i ≠ j}

[0166] As Figure 5 shown, a decision graph is constructed with the horizontal axis as p i and the vertical axis as d i , and K data with high p i and high d i are selected as initial clustering centers according to the decision graph. The value of K determines the final classification accuracy, and after repeated experiments, the value of K is finally set to 4.

[0167] Compared with random selection, the above K initial clustering centers can ensure that the initial clustering centers are located in the data-intensive area (reflecting water quality similarity) and are uniformly distributed in space (covering different monitoring areas).

[0168] As Figure 4 shown, before the water quality detection model is obtained by training the historical water quality data with labels after K-means clustering processing in the BP neural network, the method further comprises:

[0169] Setting the BP neural network parameters and establishing the BP neural network.

[0170] Specifically, the water quality data set D with labels is divided into a training set, a validation set, and a test set, and is divided into a training set (training model), a validation set (parameter tuning to avoid overfitting), and a test set (evaluating the final effect) in proportion (such as 7:2:1);

[0171] Setting the BP neural network parameters, designing the input layer (n-dimensional), the hidden layer (h neurons), and the output layer (q-dimensional):

[0172] Input layer: dimension = number of water quality features n (such as 3 time indicators → 3 neurons in the input layer).

[0173] Hidden layer: usually 1-2 layers (to avoid overfitting), and the number of neurons is optimized by empirical method (such as h = 2n + 1) or grid search.

[0174] Output layer: dimension = label dimension q (binary classification q = 1, multi-classification q = k).

[0175] Its characteristics are to inversely adjust the weights in the network between the predicted value and the true value, and the steps of the BP neural network anomaly data detection are as follows:

[0176] 1. Assign a random number of (1, 1) to the weight between each neuron, set the error function, error accuracy, and maximum learning times;

[0177] 2. Calculate the input-output error;

[0178] 3. Calculate the input and output of each neuron in the hidden layer;

[0179] 4. Calculate the partial derivative of the error function with respect to the output of each neuron in the output layer by the expected output and the actual output of the network;

[0180] 5. Modify the weights of each neuron using the error function;

[0181] 6. Calculate the global error to determine whether the network error meets the requirements, and end the algorithm when the error reaches the preset precision or the number of learning reaches the maximum value;

[0182] Given the good binary classification performance of BPNN, a BP neural network is used as the abnormal water quality data detection module. However, using BPNN alone to detect complex water quality data can cause the model to fall into a local optimal solution, resulting in low accuracy of the final detection result. In addition, BP neural network requires a data sample set with prior knowledge for modeling. Therefore, we use the labeled data after K-means clustering as the training sample set of BP neural network, and then use the neural network to establish an abnormal data classification model for subsequent data classification detection. This method solves the problem of low timeliness caused by repeated iteration of K-means clustering, and also solves the problem of prior knowledge required for training data samples of BP neural network.

[0183] Further, the water quality detection model obtained by inputting the labeled historical water quality data after K-means clustering into the BP neural network for training, specifically includes:

[0184] The historical water quality data X obtained by K-means clustering m×n = [X 1，n , X 2，n , …, X m，n , y ，i ] is rewritten as a water quality data set D = [(X 1，n , y i ), (X 2，n , y i ), …, (X m，n , y i ), and the labeled water quality data set D is divided into a training set, a validation set, and a test set; wherein X ∈ R p , y i ∈ R q , p is the dimension of the input vector, and the input feature vector X belongs to a p-dimensional real number space; q is the dimension of the output vector, and the output label vector y i belongs to a q-dimensional real number space; then the calculation steps of the water quality detection model are as follows:

[0185] (1) Input layer→Hidden layer

[0186] The labeled historical water quality data sample is imported into the initial BP network model, and the input of the historical water quality data vector at the jth neuron is represented as follows:

[0187]

[0188] Where ω i,j is the weight of the ith neuron in the input layer and the jth neuron in the hidden layer; b j is the bias of the jth neuron in the hidden layer;

[0189] Then a j is taken as the input of the hidden layer, and the output of the hidden layer is obtained after processing by the activation function sigmoid function. The sigmoid function is a commonly used nonlinear activation function in neural networks, which compresses the output (value range (0, 1)):

[0190] As follows:

[0191]

[0192] (2) Hidden layer→Output layer

[0193] The output of the hidden layer is then transmitted to the output layer neuron, and the input value of the output layer neuron at this time is as follows:

[0194]

[0195] Where v j,s is the weight of the jth neuron in the hidden layer and the s th neuron in the output layer; b s is the bias of the s th neuron in the output layer;

[0196] The predicted value of the output layer is calculated, and sigmoid (output abnormal probability) is used for binary classification:

[0197]

[0198] The weighted sum of the output layer β s is compressed by the sigmoid function to obtain the predicted value of water quality detection (such as abnormal probability).

[0199] (3) Back propagation optimization (parameter adjustment)

[0200] The process of forward transmission of the BP neural network, the predicted value and the mean square error of the true value D s are calculated as follows:

[0201] ​

[0202] The BP neural network adjusts the weight ω by gradient descent in reverse i,j 、v j,s and the threshold b j 、b s The target is to minimize the error E; when the error E is less than the set threshold, stop iteration, and get the final water quality detection model.

[0203] The gradient of the error E to the output layer weight v j,s and the bias b s is calculated, and the learning rate η is adjusted:

[0204]

[0205] The error E is propagated to the hidden layer, the gradient of the error E to the hidden layer weight ω i,j and the bias b j is calculated and updated:

[0206]

[0207] The learning rate η is tried in the interval of 0.0001-0.1, the range of stable loss reduction is found, and the maximum iteration number (such as 1000 times) is set. The validation set error is monitored, and if the error reduction of continuous iterations is less than the threshold (such as 1e-5) for several times, the training is stopped.

[0208] Embodiment 2

[0209] As shown in Figure 6 , the embodiment of the application also proposes a water source water quality monitoring system based on K-means clustering and BP neural network, which comprises:

[0210] A monitoring server 100 and a plurality of water quality wireless sensors 200 arranged in all water source areas in the monitoring area; the monitoring server 100 and the plurality of water quality wireless sensors 200 form a water quality wireless sensor network, which is used to perform the following steps:

[0211] S101, obtaining historical water quality data of all water source areas in the monitoring area;

[0212] S102, processing the historical water quality data by using a K-means clustering algorithm, and dividing and marking the feature categories of abnormal water quality data and normal water quality data according to the clustering processing result;

[0213] S103, inputting the historical water quality data with marks after K-means clustering processing into a BP neural network for training to obtain a water quality detection model;

[0214] S104. Collect current water quality data and input it into the water quality detection model. The water quality detection model outputs the feature category of the current water quality data.

[0215] The main sources of water pollution currently include industrial wastewater, domestic pollution, and agricultural fertilizer pollution, indicating that there are many factors contributing to water pollution. Different pollution sources lead to different anomaly characteristics in polluted water quality data, which requires anomaly detection algorithms to achieve high-precision detection in water quality monitoring areas with unknown pollution sources. However, existing anomaly detection algorithms have low detection accuracy on water quality datasets with unknown prior knowledge. Directly obtaining labeled current water quality data in actual aquatic environments is very difficult, and existing detection algorithms struggle to directly detect water quality data of unknown categories.

[0216] like Figure 2 As shown, the detection process of this system is as follows: The first stage is the water quality data classification stage, which uses the K-means clustering algorithm to accurately classify historical water quality data and obtain the feature categories of abnormal and normal water quality data. The second stage is the water quality data anomaly detection stage, which uses the BP neural network algorithm to directly detect current water quality data without the need for repeated clustering processing. This algorithm achieves accurate detection of abnormal water quality data with unknown prior knowledge.

[0217] The water quality monitoring method and system based on K-means clustering and BP neural network of this invention combines the accurate data classification capability of K-means clustering algorithm with BP neural network. First, historical water quality data is classified by K-means clustering, and feature labels are assigned to different categories of data to classify abnormal and normal data. Then, the BP neural network is trained using the labeled dataset to obtain the final abnormal water quality data detection model. This detection model does not require repeated clustering of water quality data and can directly realize the rapid detection of subsequent water quality data.

[0218] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A water quality monitoring method based on K-means clustering and BP neural network, characterized in that, The method comprises: S101, acquiring historical water quality data of all water source areas in a monitoring area; S102, processing the historical water quality data by using a K-means clustering algorithm, and marking feature categories of abnormal water quality data and normal water quality data according to a clustering processing result; S103, inputting the historical water quality data with the mark after K-means clustering processing into a BP neural network for training to obtain a water quality detection model; S104, collecting current water quality data and inputting the current water quality data into the water quality detection model, and the water quality detection model outputs a feature category of the current water quality data. 2.The water quality monitoring method based on K-means clustering and BP neural network according to claim 1, wherein, The method further comprises: Arranging a water quality wireless sensor network in all water source areas in the monitoring area, for collecting historical water quality data and current water quality data of all water source areas. 3.The water quality monitoring method based on K-means clustering and BP neural network according to claim 1, wherein, The step S102 specifically comprises: Cutting historical water quality data with a time series length of h, and performing missing data smoothing processing and normalization processing on the historical water quality data; Calculating distances of all historical water quality data to initial clustering centers, and dividing the historical water quality data into clusters with minimum distances, and performing iteration; Defining a cluster with minimum data quantity as an abnormal cluster according to a clustering processing result, and marking the normal cluster and the abnormal cluster respectively, so as to obtain feature categories of abnormal water quality data and normal water quality data.

4. The water quality monitoring method based on K-means clustering and BP neural network according to claim 3, characterized in that, The method comprises: History water quality data X collected by water quality wireless sensor network of all water source areas in monitoring area m×n , wherein X m×n = [X 1，n , X 2，n , …, X m，n ]; X m×n is water quality data collected by m sensor nodes in n time; history water quality data X m×n is collected and transmitted in matrix form The step of processing the historical water quality data by using the K-means clustering algorithm specifically comprises: Let X be historical water quality data m×n Select k historical water quality data as initial clustering centers C, C = [C1, C2, …, Ck]; k If k clusters are formed, each C s is an n-dimensional vector; Calculating Euclidean distances of all historical water quality data to initial clustering centers: X i,n Xi(n) is the water quality data of the ith sensor node at time n; C s,n is the water quality data at time n of the s-th initial cluster center; dist(.) is the Euclidean distance; The water quality data X collected by each sensor node i at time n i,n According to the distance, the water quality data is divided into corresponding categories, and the average value of the Euclidean distance of the historical water quality data in the cluster is calculated According to Update the initial clustering center; where S s is the historical water quality data set of the s-th cluster, |S s | is the cluster size; The above steps are repeated until the cluster center points no longer shift, and the iteration is stopped; for each sensor node i, the distance of it to all C s is calculated, and the sensor node i is divided into the nearest cluster; Marking historical water quality data of each category after K-means clustering processing, as follows: The historical water quality data is represented as X m×n = [X 1，n , X 2，n , …, X m，n , y ，i ]; y i is the label of the ith water quality sample.

5. The water quality monitoring method based on K-means clustering and BP neural network according to claim 3, characterized in that, After cutting historical water quality data with a time series length of h, and performing missing data smoothing processing and normalization processing on the historical water quality data, the method further comprises: Optimizing selection of the initial clustering center C, specifically as follows: S201, Historical water quality data X m×n Each historical water quality data x i Computational space-parameter joint distance: wherein x i is the historical water quality data of the i-th sensor node; ω is the parameter distance weight; d 参数 is the parameter Euclidean distance of two historical water quality data; d 空间 is the geographical Euclidean distance of the sensor node i; S202, density peak value screening: S301、calculating the local density p of each point i : For historical water quality data x i Its local density p i Indicates the degree of density of data points in the neighborhood, and the joint distance d ij Weighted sum: Wherein, σ is a bandwidth parameter of a Gaussian kernel; S302、Calculate the distance parameter δ i : delta of high density points i : delta i = min{d ij | p j > p i , i≠j} delta of low density points i : delta i = max {d ij | j = 1, 2, 3,..., m x n, i ≠ j} With the horizontal axis as p i , the vertical axis as d i , a decision graph is constructed, and K data with high p i and high d i are selected as initial clustering centers according to the decision graph.

6. The water source water quality monitoring method based on K-means clustering and BP neural network according to claim 5, characterized in that, Let two historical water quality data x i =(x i1 , x i2 ,..., x id ), x j =(x j1 , x j2 ,..., x jd ), each historical water quality data x i contains d dimensions of data, then the parameter Euclidean distance of the two historical water quality data is the square root of the square sum of the difference of each dimension, the formula is: Let sensor node coordinate (lon i , lat i ), spatial parameters include 2-dimension data, then the Euclidean distance of two historical water quality data in geography is the square root of the square sum of 2-dimension difference, the formula is:

7. The water quality monitoring method based on K-means clustering and BP neural network according to claim 1, characterized in that, Before the step S103, the method further comprises: Setting a BP neural network parameter, and establishing a BP neural network.

8. The water quality monitoring method based on K-means clustering and BP neural network according to claim 7, characterized in that, The step S103 specifically comprises: The historical water quality data X obtained by K-means clustering processing m×n = [X 1,n , X 2,n , …, X m,n , y ,i ] is rewritten as a water quality data set D = [(X 1,n , y i ), (X 2,n , y i ), …, (X m,n , y i ), and the water quality data set D with labels is split into a training set, a validation set and a test set; wherein X ∈ R p , y i ∈ R q , p is the dimension of the input vector, the input feature vector X belongs to a p-dimensional real number space; q is the dimension of the output vector, the output label vector y i belongs to a q-dimensional real number space; and the calculation steps of the water quality detection model are as follows: (1) Input layer→hidden layer Introducing historical water quality data samples with labels into an initial BP network model, and input of the historical water quality data vector at the jth neuron is represented as follows: Where, ω i,j These are the weights of the i-th neuron in the input layer and the j-th neuron in the hidden layer; b j It is the bias of the j-th neuron in the hidden layer; Again, a j As the input of the hidden layer, the output of the hidden layer is obtained after processing by the activation function sigmoid function, as follows: (2) Hidden layer→output layer After the hidden layer obtains an output result, the output result is transmitted to output layer neurons, and at this time, input values of the output layer neurons are as follows: where v j,s is the weight of the jth neuron of the hidden layer to the st neuron of the output layer; b s is the bias of the st neuron of the output layer. Calculating a prediction value of the output layer: sigmoid function to the output layer weighted sum β s compressing, obtaining the predicted value of water quality detection ; (3) Back propagation optimization The process of forward transmission of BP neural network, the predicted value The mean square error with the true value D s is calculated as follows: The BP neural network adjusts the weight ω by gradient descent in reverse i,j , j,s and threshold b j , b s When the error E is less than the set threshold, stop iteration and get the final water quality detection model. 9.The water quality monitoring method based on K-means clustering and BP neural network according to claim 8, characterized in that, The BP neural network adjusts the weight ω by gradient descent in reverse i,j 、 j,s and threshold b j 、 b s Specifically includes: The gradient of the error E with respect to the output layer weights v j,s and bias b s is adjusted with a learning rate η: The error E is backpropagated to the hidden layer, the gradients of ω i,j and b j are computed and updated:

10. A water source quality monitoring system based on K-means clustering and BP neural network, characterized in that, The system comprises: A monitoring server, and a plurality of water quality wireless sensors arranged in all water source areas in a monitoring area and connected with the monitoring server; the monitoring server and the plurality of water quality wireless sensors form a water quality wireless sensor network, and are used for performing the following steps: S101, acquiring historical water quality data of all water source areas in a monitoring area; S102, the historical water quality data is processed by using a K-means clustering algorithm, and characteristic categories of abnormal water quality data and normal water quality data are divided according to a clustering processing result and are marked respectively; S103, the historical water quality data with the mark after K-means clustering processing is input into a BP neural network for training to obtain a water quality detection model; S104, current water quality data is collected and input into the water quality detection model, and the water quality detection model outputs a characteristic category of the current water quality data.

Images