A water-out BOD prediction method based on self-organizing interconnection modular neural network

Abstract

The application relates to a water-out BOD prediction method based on a self-organizing interconnection modular neural network, which is directly applied to the field of sewage treatment. The modular neural network is a network model constructed by the modular properties of a 'brain-like' module, and there is a certain connection between the modules of the 'brain-like' modular network. When processing complex tasks, the connection or node is adjusted according to the task. Therefore, the application designs a sewage treatment water-out BOD prediction method based on a self-organizing interconnection modular neural network, which realizes accurate prediction of the water-out BOD concentration in the mode of simulating the 'brain-like' modular information processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and is directly applied to the field of sewage treatment, and particularly relates to a water outlet BOD prediction method based on a self-organizing interconnected modular neural network. BACKGROUND

[0002] Biochemical Oxygen Demand (BOD) is the amount of dissolved oxygen consumed in the biochemical reaction process of microbial decomposition of biodegradable organic matter existing in water under certain conditions. It is an important indicator of organic pollutants in water. The current methods for measuring the BOD concentration of the outlet water include microbial electrode method, dilution inoculation method, instrument method, etc. However, these methods have the defects of long time and low precision. Therefore, how to predict the BOD concentration of the outlet water at low cost and high efficiency is a difficult problem faced by the sewage treatment process.

[0003] Modular neural network is a network model constructed by "brain-like" modular properties, and there is a certain connection between each module of the "brain-like" modular network. When processing complex tasks, the connection or node is adjusted according to the task. Therefore, the present application designs a sewage treatment outlet water BOD prediction method based on a self-organizing interconnected modular neural network, which realizes accurate prediction of the BOD concentration of the outlet water by simulating the "brain-like" modular information processing method. SUMMARY

[0004] 1. The technical problems that need to be solved and can be solved by the present application.

[0005] The present application provides a sewage treatment outlet water BOD prediction method based on a self-organizing interconnected modular neural network. The input variables are divided into several sub-sets by soft clustering, each sub-set is processed by different sub-networks, at the same time, all sub-networks are connected with each other, and the structure of the sub-networks is adjusted by using a gradient-based growth strategy and a threshold-based pruning strategy to predict the BOD of the outlet water of the sewage treatment, so as to improve the prediction accuracy.

[0006] 2. The specific technical scheme of the present application.

[0007] 1. A water outlet BOD prediction method based on a self-organizing interconnected modular neural network, mainly comprising:

[0008] Step 1: water quality variable subset division;

[0009] Collect the actual water quality variable data of the sewage treatment plant, and record O as the BOD concentration of the outlet water, o n is the nth sample value of the BOD concentration of the outlet water, wherein n=1,2,…,N; M fis the fth input water quality variable, where f = 1, 2, …, F; F is the number of water quality variables, and N is the number of samples of the water quality variables, is the nth sample value of the fth water quality variable;

[0010] Step 1.1: Set the soft clustering algorithm indicators as follows:

[0011]

[0012] where u kf is the membership degree of the fth variable to the kth cluster, a = 1.5 is a fuzzy coefficient, w kn is the weight of the nth sample in the kth cluster, v kn is the center of the nth sample in the kth cluster, and g = 0.5 is a reinforcement parameter;

[0013] Step 1.2: Set i = 1, the maximum number of iterations Maxc = 20 of the soft clustering algorithm, set the total number of clusters K = 2, and randomly initialize the cluster centers v kn and the weights w kn = 1 / N, where N is the number of samples of the water quality variables, k = 1, 2, …, K, and n = 1, 2, …, N;

[0014] Step 1.3: Calculate the membership degree u kf of the fth water quality variable to the kth cluster according to the following formula:

[0015]

[0016] Step 1.4: Calculate the cluster center v kf of the nth sample in the kth cluster according to the following formula using the u kn of step 1.3:

[0017]

[0018] Step 1.5: Calculate the weight w kn of the nth sample in the kth cluster according to the following formula using the membership degree u kf of the fth water quality variable to the kth cluster of step 1.3 and the cluster center v kn of the nth sample in the kth cluster of step 1.4:

[0019]

[0020] where e(·) is an exponential function with base e;

[0021] Step 1.6: Calculate the value of formula (1) according to the results of steps 1.3, 1.4 and 1.5, denoted as J(i), when i = 1, i = i + 1 return to step 1.3, when i > 1, go to step 1.7;

[0022] Step 1.7: When |J(i)-J(i-1)|<0.001 or i>Maxc, the soft clustering algorithm ends, where J is the soft clustering algorithm index defined by formula (1), execute step 1.8, otherwise i = i + 1 return to step 1.3;

[0023] Step 1.8: Use the membership generated by iteration u kf Subsets are divided for water quality variables, and the division threshold δ = 0.3 is set. For the kth cluster, if the membership u kf > δ of the fth water quality variable, the water quality variable is divided into the cluster, until all water quality variables are divided;

[0024] After step 1, K subsets Z k (k = 1, 2,..., K) containing different water quality variables are obtained;

[0025] Step 2: Design the interconnection modular neural network model structure for predicting effluent BOD;

[0026] Step 2.1: Normalize the water quality variables in the K subsets Z k obtained by step 1 to the interval -1 to 1 according to formula (5), where k = 1, 2,..., K; normalize the output variable effluent BOD to the interval 0 to 1 according to formula (6):

[0027] j∈Z k and k = 1, 2,..., K (5)

[0028]

[0029] Wherein, m j represents the jth water quality variable in the set Z k , O represents the output variable, x j and d represent the normalized jth water quality variable and output variable respectively; then divide the samples in each subset into three parts of training set, validation set and test set;

[0030] Step 2.2: Design the subnetwork structure of interconnection modular neural network, the model is composed of K subnetworks, each subnetwork adopts a multilayer perceptron with double hidden layers, including input layer, first hidden layer, second hidden layer and output layer four layer structure. The initial topology of each subnetwork is z k -H1-H2-1; k = 1, 2,..., K, wherein z kThe number of input neurons of the kth subnetwork is the same as the number of water quality variables in the set Z k H1 and H2 are the number of neurons in the first and second hidden layers, respectively, and the number of output layer neurons is 1. At the same time, the second hidden layer nodes of each subnetwork are connected to the output nodes of other subnetworks;

[0031] The sparse rate Spa is set to 0.3, and the weights between all adjacent layers of each subnetwork are processed. Initially, the number of weights from the input layer to the first hidden layer is z k ×H1, the number of weights from the first to the second hidden layer is H1×H2, and the number of weights from the second hidden layer to the output layer is H2×K. Then, the weights in each adjacent layer are sorted in ascending order, and the rth weight after sorting is recorded as the pruning threshold λ=S r , where r = the number of weights in the layer × (1-Spa). The weights smaller than λ in the layer are deleted, and the deleted weights are considered as dormant weights. The number of deleted nodes in the first and second hidden layers of the kth subnetwork after sparsification is and The structure is k = 1, 2, …, K, where and are the remaining number of neurons in the first and second hidden layers of the kth subnetwork, respectively, and satisfy and k = 1, 2, …, K;

[0032] Step 2.3: Assuming that for the kth subnetwork Z k , k = 1, 2, …, K, the nth input sample is n = 1, 2, …, N; At this time, the output of the h1th neuron in the first hidden layer of the kth subnetwork is:

[0033]

[0034] where is the connection weight between the jth node of the input layer and the h1th neuron of the first hidden layer in the kth subnetwork, B() is the activation function of the h1th hidden layer neuron in the subnetwork, and is defined as formula (8):

[0035]

[0036] Step 2.4: Calculate the output of the h2th neuron in the second hidden layer of the kth subnetwork:

[0037]

[0038] where is the connection weight between the h1th node of the first hidden layer and the h2th neuron of the second hidden layer in the kth subnetwork;

[0039] Step 2.5: Calculate the output of the kth subnetwork:

[0040]

[0041] wherein, is the connection weight between the h2th node of the second hidden layer and the output node of the kth subnetwork in the kth subnetwork;

[0042] Step 2.6: Calculate the output of the interconnected modular neural network:

[0043]

[0044] wherein is the output of the nth sample of the kth subnetwork, and K is the number of subnetworks.

[0045] Step 2.7: Select the mean square error function as the objective function of the interconnected modular neural network:

[0046]

[0047] wherein d n is the expected output of the nth sample of the output variable;

[0048] Step 3: Self-organizing design of the interconnected modular neural network structure

[0049] Step 3.1: Set the number of structure self-organization l max = 8, and let l = 1;

[0050] Step 3.2: Calculate the output error of the current nth sample:

[0051] e n = d n - Y n n = 1, 2,..., N (13)

[0052] Step 3.3: Under the current neural network structure, let the vector Δ contain all the weights in all subnetworks:

[0053] Δ = [Δ1, Δ2,..., Δ K ] (14)

[0054] wherein Δ k is the parameter in the kth subnetwork, including the connection weights between layers and

[0055]

[0056] The parameter update formula is as follows:

[0057] Δ(t+1) = Δ(t) - (Q(t) + μ(t)I) -1 g(t) (16)

[0058] where t represents the iteration step number, Q is a quasi-Hessian matrix, g is a gradient vector, I is a unit matrix, and μ is a learning rate. The quasi-Hessian matrix and the gradient vector are calculated according to formulas (17) and (18), respectively.

[0059]

[0060]

[0061] where e n is the neural network output error of the nth sample, calculated according to formula (13), and j n is the Jacobian matrix row vector of the corresponding sample, defined as follows:

[0062] j n = [j n,1 ,j n,2 ,...,j n,K ] (19)

[0063] where j n,k is the Jacobian row vector of the kth subnetwork, defined as follows:

[0064]

[0065] According to formulas (7)-(11) and (13), the following is obtained:

[0066]

[0067]

[0068]

[0069] Through formulas (21)-(23), the Jacobian matrix row vector j n,k is obtained. After all the training samples are traversed, the quasi-Hessian matrix Q and the gradient vector g are obtained, and then the connection weights in all subnetworks contained in the vector Δ are updated according to the update formula (16);

[0070] In the training process, the training objective function values E tr (t) and E tr (t+1) of the neural network at time t and t+1 are calculated according to formula (12). When E tr(t+1) < E tr (t), the learning rate μ(t+1) = μ(t) / 10, all weight parameters in the network are reserved; otherwise, the learning rate μ(t+1) = μ(t) *10, all weight parameters in the network are restored to their values before updating, and the neural network parameters are updated based on the current μ, while recording the objective function value E of the neural network on the validation set at each time during training va ; let the maximum iteration step number be T max =40, and the expected error value be E d =0.08; the parameter learning process is iterated continuously, when the iteration step number t=T max , if the training error E tr >E d , go to step 3.4; if the current training error E tr ≤E d but |E tr -E va |>0.01, |·| is the absolute value, then go to step 3.5; if the current training error E tr ≤E d and |E tr -E va |≤0.01, the training is stopped;

[0071] Step 3.4: calculate the gradient of all dormant weight values between adjacent layers according to the following formula:

[0072]

[0073]

[0074]

[0075] wherein, and represent the dormant connection weight values between neurons of each layer at this time;

[0076] Calculate the gradient between neurons across layers, the gradient across one layer of neurons is:

[0077]

[0078] The gradient across two layers of neurons is:

[0079]

[0080] wherein represents the connection weight value between the jth input neuron of the input layer and the h2th neuron of the second hidden layer, represents the connection weight value between the h1th input neuron of the first hidden layer and the kth neuron of the output layer;

[0081] If l>l max Jump to step 4; if l<l max , find the maximum value of the gradient of the dormant weight between adjacent layers in all sub-networks and the gradient between cross-layer neurons through the calculation of step 3.4, if the maximum value appears in and , the dormant weight corresponding to the maximum gradient is reactivated, and a random value in the interval of 0 to 1 is assigned, l=l+1, return to step 3.2; if the maximum gradient appears in , and the number of neurons in the first hidden layer is less than , add 1 neuron in the first hidden layer of the kth sub-network, the input weight and the output weight are set to a random number between 0 and 1 respectively, l=l+1, return to step 3.2, otherwise if the number of neurons in the first hidden layer is equal to , jump to step 4; if the maximum gradient appears in , and the number of neurons in the second hidden layer is less than , add 1 neuron in the second hidden layer of the kth sub-network, the input weight and the output weight are set to a random number between 0 and 1 respectively, l=l+1, return to step 3.2, otherwise if the number of neurons in the second hidden layer is equal to , jump to step 4;

[0082] Step 3.5: set the weight reduction threshold Thw=0.1, and judge whether there is a value less than Thw between the connection weights of each layer of each sub-network and , if yes, delete, l=l+1, return to step 3.2, otherwise end.

[0083] Step 4: take the test sample data as the input of the self-organizing interconnection modular neural network trained, get the output of the self-organizing interconnection modular neural network, and get the predicted value of the effluent BOD concentration by inverse normalization.

[0084] 3、Compared with the prior art, the present application has the following obvious advantages and beneficial effects:

[0085] The present application aims at the low measurement accuracy of effluent BOD in the current sewage treatment process, divides the auxiliary variables into several subsets by using a soft clustering algorithm, and proposes an effluent BOD prediction method based on a self-organizing interconnection modular neural network, so as to realize the accurate measurement of effluent BOD concentration and improve the prediction accuracy of effluent BOD to a certain extent. DRAWINGS:

[0086] Figure 1A structural schematic diagram of a self-organizing interconnection modular neural network of the present application;

[0087] Figure 2 A training mean square error change graph of the self-organizing interconnection modular neural network of the present embodiment;

[0088] Figure 3 A prediction result graph of the effluent BOD prediction model of the present embodiment;

[0089] Figure 4 An effluent BOD test error graph of the present embodiment;

[0090] Figure 5 A self-organizing interconnection modular neural network hidden layer node quantity change graph of the present embodiment;

[0091] Figure 6 A self-organizing interconnection modular weight quantity change graph of the present embodiment; DETAILED DESCRIPTION:

[0092] The present application provides an effluent BOD prediction method based on a self-organizing interconnection modular neural network, realizes accurate measurement of effluent BOD, and improves the prediction accuracy of effluent BOD of a municipal sewage treatment plant.

[0093] The present embodiment adopts water quality analysis data of a certain sewage plant in 2011, which contains 360 groups of data and 23 water quality variables, including (1) influent PH; (2) effluent PH; (3) influent suspended solid concentration (SS); (4) effluent SS; (5) influent BOD concentration; (6) influent chemical oxygen demand (COD) concentration; (7) effluent COD concentration; (8) biochemical pool sludge settling ratio (SV); (9) biochemical pool mixed liquid suspended solid concentration (MLSS); (10) biochemical pool dissolved oxygen concentration (DO); (11) influent oil; (12) effluent oil; (13) influent ammonia nitrogen concentration; (14) effluent ammonia nitrogen concentration; (15) influent color; (16) effluent color; (17) influent total nitrogen concentration; (18) effluent total nitrogen concentration; (19) influent phosphate concentration; (20) effluent phosphate concentration; (21) influent water temperature; (22) effluent water temperature; (23) effluent BOD concentration; 219 groups of data are randomly selected as training samples, 73 groups of data are selected as test samples, and 73 groups of data are selected as verification samples;

[0094] An effluent BOD prediction method based on a self-organizing interconnection modular neural network includes the following steps:

[0095] Step 1: water quality variable subset division;

[0096] The actual water quality variable data of the sewage treatment plant is collected, denoted as O for the effluent BOD concentration, o n is the nth sample value of the effluent BOD concentration, where n = 1, 2, …, N; M f is the fth input water quality variable, where f = 1, 2, …, F; F is the number of water quality variables, N is the number of samples of water quality variables, m fn is the nth sample value of the fth water quality variable;

[0097] Step 1.1: Set the soft clustering algorithm indicators as follows:

[0098]

[0099] where u kf is the membership degree of the fth variable to the kth cluster, α = 1.5 is the fuzzy coefficient, w kn is the weight of the nth sample in the kth cluster, v kn is the center of the nth sample in the kth cluster, γ = 0.5 is the reinforcement parameter;

[0100] Step 1.2: Set i = 1, the maximum number of iterations of the soft clustering algorithm Maxc = 20, set the total number of clusters K = 2, and randomly initialize the cluster centers v kn and the weights w kn = 1 / N, where N is the number of samples of water quality variables, k = 1, 2, …, K; n = 1, 2, …, N;

[0101] Step 1.3: Calculate the membership degree u kf of the fth water quality variable to the kth cluster as follows:

[0102]

[0103] Step 1.4: Calculate the cluster center v kf of the nth sample in the kth cluster using the u kn of step 1.3 as follows:

[0104]

[0105] Step 1.5: Calculate the weight w kn of the nth sample in the kth cluster using the membership degree u kf of the fth water quality variable to the kth cluster of step 1.3 and the cluster center v kn of the nth sample in the kth cluster of step 1.4 as follows:

[0106]

[0107] where e(·) is the exponential function with base e;

[0108] Step 1.6: Calculate the value of formula (30) according to the results of steps 1.3, 1.4 and 1.5, and record it as J(i), when i = 1, i = i + 1 return to step 1.3, when i > 1, go to step 1.7;

[0109] Step 1.7: When |J(i) - J(i - 1)| < 0.001 or i > Maxc, the soft clustering algorithm ends, where J is the soft clustering algorithm index defined by formula (30), execute step 1.8, otherwise i = i + 1 return to step 1.3;

[0110] Step 1.8: Use the membership generated by iteration u kf Subsets are divided for water quality variables, and the division threshold δ = 0.3 is set. For the kth cluster, if the membership u kf > δ for the fth water quality variable, the water quality variable is divided into the cluster, until all water quality variables are divided;

[0111] After step 1, K subsets Z k , where k = 1, 2, …, K are obtained, which contain different water quality variables;

[0112] In this embodiment, two subsets of water quality variables are obtained. The first subset includes: (1) effluent PH; (2) influent suspended solid (SS) concentration; (3) influent oil; (4) influent color; (5) influent total nitrogen concentration; (6) influent phosphate concentration; and the second subset includes: (1) influent PH; (2) effluent SS; (3) influent BOD concentration; (4) influent chemical oxygen demand (COD) concentration; (5) effluent COD concentration; (6) biochemical pool settling velocity (SV); (7) biochemical pool mixed liquid suspended solids (MLSS) concentration; (8) biochemical pool dissolved oxygen (DO) concentration; (9) effluent oil; (10) influent ammonia nitrogen concentration; (11) effluent ammonia nitrogen concentration; (12) effluent color; (13) influent total nitrogen concentration; (14) effluent total nitrogen concentration; (15) influent phosphate concentration; (16) effluent phosphate concentration; (17) influent water temperature; (18) effluent water temperature; 219 groups of data are randomly selected as training samples, 73 groups of data are selected as test samples, and 73 groups of data are selected as verification samples;

[0113] Step 2: design the interlinked modular neural network model structure for predicting effluent BOD;

[0114] Step 2.1: normalize the water quality variables in each of the K sub-sets Z k acquired in Step 1 to the interval -1 to 1 according to formula (34), where k = 1, 2, …, K; normalize the effluent BOD output variable to the interval 0 to 1 according to formula (35):

[0115] j∈Z k and k = 1, 2, …, K (34)

[0116]

[0117] wherein m j represents the jthwater quality variable in the set Z k , O represents the output variable, x j and d respectively represent the normalized jthwater quality variable and output variable; then divide the samples in each sub-set into three parts: training set, validation set, and test set;

[0118] Step 2.2: design the sub-network structure of the interlinked modular neural network, which is composed of K sub-networks. Each sub-network uses a multilayer perceptron with double hidden layers, including an input layer, a first hidden layer, a second hidden layer, and an output layer. The initial topology of each sub-network is z k -H1-H2-1; k = 1, 2, …, K, wherein z k is the number of input neurons of the kthsub-network, which is the same as the number of water quality variables in the set Z k , H1and H2are the numbers of neurons in the first hidden layer and the second hidden layer respectively, and the number of neurons in the output layer is 1. At the same time, the nodes in the second hidden layer of each sub-network are connected to the output nodes of other sub-networks;

[0119] Set the sparsity Spa=0.3, and perform sparse processing on the weights between all adjacent layers of each sub-network. Initially, the number of weights from the input layer to the first hidden layer is z k ×H1, the number of weights from the first to the second hidden layer is H1×H2. The number of weights from the second hidden layer to the output layer is H2×K. Then, sort the weights in each adjacent layer in ascending order, and record the rthweight after sorting as the pruning threshold λ=S r , where r = the number of weights in the layer × (1-Spa), and delete the weights in the layer that are smaller than λ. The deleted weights are considered as dormant weights. Record the number of deleted nodes in the first hidden layer and the second hidden layer of the kthsub-network after sparsification as and The structure is k = 1, 2, …, K, where and are the remaining number of neurons of the first and second hidden layers of the kth subnetwork, respectively, and satisfy and k = 1, 2, …, K;

[0120] Step 2.3: Assuming that the kth subset Zk= {z1, z2, …, zK} is given, the input of the kth subnetwork is k , k = 1, 2, …, K, the nth input sample is n = 1, 2, …, N; at this time, the output of the first hidden layer of the kth subnetwork is:

[0121]

[0122] where is the connection weight between the jth node of the input layer and the h1th neuron of the first hidden layer in the kth subnetwork, B() is the activation function of the h1th hidden layer neuron of the subnetwork, defined as shown in equation (37):

[0123]

[0124] Step 2.4: Calculate the output of the h2th neuron of the second hidden layer of the kth subnetwork:

[0125]

[0126] where is the connection weight between the h1th node of the first hidden layer and the h2th neuron of the second hidden layer in the kth subnetwork;

[0127] Step 2.5: Calculate the output of the kth subnetwork:

[0128]

[0129] where is the connection weight between the h2th node of the second hidden layer and the output node of the kth subnetwork;

[0130] Step 2.6: Calculate the output of the interconnected modular neural network:

[0131]

[0132] where is the output of the nth sample of the kth subnetwork, and K is the number of subnetworks.

[0133] Step 2.7: Select the mean square error function as the objective function of the interconnected modular neural network:

[0134]

[0135] where d n is the expected output of the n-th sample of the output variable;

[0136] Step 3: Interconnected modular neural network structure self-organizing design

[0137] Step 3.1: Set the structure self-organizing times l max = 8, let l = 1;

[0138] Step 3.2: Calculate the output error of the current n-th sample:

[0139] e n = d n - y n n = 1, 2, …, N (42)

[0140] Step 3.3: Under the current neural network structure, let the vector Δ contain all the weights in all subnetworks:

[0141] Δ = [Δ1, Δ2, …, Δ K ] (43)

[0142] where Δ k is the parameter in the k-th subnetwork, including the connection weights between layers and

[0143]

[0144] The parameter update formula is as follows:

[0145] Δ(t + 1) = Δ(t) - (Q(t) + μ(t)I) -1 g(t) (45)

[0146] where t represents the iteration step number, Q is the quasi-Hessian matrix, g is the gradient vector, I is the identity matrix, and μ is the learning rate. The quasi-Hessian matrix and the gradient vector are calculated according to formulas (46) and (47) respectively:

[0147]

[0148]

[0149] where e n is the neural network output error of the n-th sample, calculated according to formula (42), j n is the Jacobian matrix row vector corresponding to the sample, defined as follows:

[0150] j n = [j n,1 , jn,2 ...,j n,K ] (48)

[0151] where j n,k is the Jacobian row vector of the kth subnetwork, defined as follows:

[0152]

[0153] According to formulas (37)-(40) (42), we obtain:

[0154]

[0155]

[0156]

[0157] Through formulas (50)-(52), the row vector j n,k of the Jacobian matrix can be obtained. After all the training samples are traversed, the quasi-Hessian matrix Q and the gradient vector g are obtained, and then the connection weights contained in the vector Δ in all subnetworks are updated according to the update formula (45);

[0158] In the training process, the training objective function values E tr (t) and E tr (t+1) of the neural network at time t and t+1 are calculated by formula (41). When E tr (t+1)≤E tr (t), the learning rate μ(t+1) = μ(t) / 10, and all weight parameters in the network are retained; otherwise, the learning rate μ(t+1) = μ(t) x 10, and all weight parameters in the network are restored to their values before updating. The neural network parameters are updated based on the current μ, and the objective function value E va of the neural network on the validation set at each time is recorded during training. Let the maximum iteration step number be T max = 40, and the expected error value be E d = 0.08. During the parameter learning process, when the iteration step number t = T max , if the training error E tr >E d , go to step 3.4; if the current training error E tr ≤E d but |E tr -E va | > 0.01, |·| is the absolute value, then go to step 3.5; if the current training error E tr ≤E d and |E tr -E va|≤0.01, the training stops;

[0159] Step 3.4: Calculate the gradient of all dormant weights between adjacent layers according to the following formula:

[0160]

[0161]

[0162]

[0163] wherein, and At this time, respectively represent the dormant connection weights between neurons of each layer;

[0164] Calculate the gradient between neurons across layers, the gradient across neurons of a layer is:

[0165]

[0166] The gradient across neurons of two layers is:

[0167]

[0168] wherein represents the connection weight between the jth input neuron of the input layer and the h2th neuron of the second hidden layer, represents the connection weight between the h1th input neuron of the first hidden layer and the kth neuron of the output layer;

[0169] If l>l max jump to step 4; if l max , find the maximum value of the gradient of the dormant weights between adjacent layers and the gradient across neurons of all subnetworks through the calculation of step 3.4, if the maximum value appears in and , re-activate the dormant weight corresponding to the maximum gradient and assign a random value in the interval of 0 to 1, l=l+1, return to step 3.2; if the maximum gradient appears in , and the number of neurons in the first hidden layer is less than , add 1 neuron to the first hidden layer of the kth subnetwork, the input weight and the output weight are set to a random number between 0 and 1, l=l+1, return to step 3.2, otherwise if the number of neurons in the first hidden layer is equal to , jump to step 4; if the maximum gradient appears in , and the number of neurons in the second hidden layer is less than , add 1 neuron to the second hidden layer of the kth subnetwork, Set the input and output weights to random numbers between 0 and 1, l = l + 1, and return to step 3.2; otherwise, if the number of neurons in the second hidden layer is equal to... If so, skip to step 4;

[0170] Step 3.5: Set the weight reduction threshold Thw = 0.1, and sequentially determine the connection weights between each layer of each sub-network. and Does a value less than Thw exist? If it does, delete it, set l = l + 1, and return to step 3.2; otherwise, end the process.

[0171] In this embodiment, the variation graph of the training mean squared error (MSE) of the effluent BOD prediction model is shown below. Figure 2 As shown, the X-axis represents the total number of training iterations, and the Y-axis represents the training MSE.

[0172] Step 4: Use the test sample data as input to the trained self-organizing interconnected modular neural network to obtain the output of the self-organizing interconnected modular neural network, and then perform inverse normalization to obtain the predicted value of the effluent BOD concentration.

[0173] In this embodiment, the prediction results of the effluent BOD prediction model are as follows: Figure 3 As shown, the X-axis represents the number of test samples (in units), and the Y-axis represents the predicted effluent BOD concentration (in mg / L). The solid line represents the predicted output value of the effluent BOD concentration, and the dashed line represents the expected output value of the effluent BOD concentration. Test errors are shown in the figure. Figure 4 As shown, the X-axis represents the number of test samples (in units), and the Y-axis represents the effluent BOD prediction error (in mg / L). The results demonstrate the effectiveness of the effluent BOD prediction method based on a self-organizing interconnected modular neural network.

Claims

1. A self-organizing interconnection modular neural network-based effluent BOD prediction method, characterized by The method comprises the following steps: Step 1: water quality variable subset division; The actual water quality variable data of the sewage treatment plant is collected, and O is the effluent BOD concentration, o n is the nth sample value of the effluent BOD concentration, wherein n = 1, 2, …, N; M f is the fth input water quality variable, wherein f = 1, 2, …, F; F is the number of water quality variables, N is the number of samples of the water quality variable, is the nth sample value of the fth water quality variable; Step 1.1: set the soft clustering algorithm index as follows: where u kf is the membership degree of the fth variable belonging to the kth cluster, a = 1.5 is a fuzzy coefficient, w kn is the weight of the nth sample in the kth cluster, v kn is the center of the nth sample in the kth cluster, and g = 0.5 is a reinforcement parameter. Step 1.2: Set i = 1, the maximum iteration number of soft clustering algorithm Maxc = 20, set the total number of clusters K = 2, and initialize the cluster center v randomly in the interval of -1 to 1 kn and the weight w kn = 1 / N, where N is the sample number of water quality variables, k = 1, 2, …, K; n = 1, 2, …, N; Step 1.3: Calculate the membership of the fth water quality variable to the kth cluster u kf : Step 1.4: Utilize u from Step 1.3 kf Compute the cluster center v for the nth sample in the kth cluster kn As follows: Step 1.5: Calculate the membership u of the kth cluster with the fth water quality variable for the nth sample kf and the cluster center v of the kth cluster with the nth sample from step 1.4 kn Calculate the weight w of the nth sample in the kth cluster kn as follows: Wherein e(·) is the exponential function with base e; Step 1.6: according to the results of steps 1.3, 1.4 and 1.5, calculate the value of formula (1), denoted as J(i), when i=1, i=i+1 return to step 1.3, when i>1, go to step 1.7; Step 1.7: when |J(i)-J(i-1)|<0.001 or i>Maxc, the soft clustering algorithm ends, wherein J is the soft clustering algorithm index defined by formula (1), step 1.8 is executed, otherwise i=i+1 return to step 1.3; Step 1.8: Using the iteratively generated membership u kf The water quality variables are subsetted, and the subsetting threshold δ = 0.3 is set. For the kth cluster, if the membership u kf > δ of the fth water quality variable, the water quality variable is classified into the cluster, until all water quality variables are classified. K subsets Z containing different water quality variables are obtained by step 1 k where k = 1, 2, …, K; Step 2: design the interconnection modular neural network model structure for predicting effluent BOD; Step 2.1: The water quality variables in the K subsets Z k obtained from Step 1 are normalized to the interval -1 to 1 according to equation (5) for k = 1, 2,..., K; the output variable effluent BOD is normalized to the interval 0 to 1 according to equation (6): where m j represents the jth water quality variable in the set Z k , O represents the output variable, x j and d represent the normalized jth water quality variable and output variable, respectively; then the samples in each subset are divided into three parts: training set, validation set, and test set; Step 2.2: design the subnetwork structure of the interconnected modular neural network, the model is composed of K subnetworks, each subnetwork adopts a multilayer perceptron with double hidden layers, including an input layer, a first hidden layer, a second hidden layer and an output layer; the initial topology structure of each subnetwork is z k -H1-H2-1; k = 1, 2, …, K; wherein z k is the number of input neurons of the kth subnetwork, which is the same as the number of water quality variables in the set Z k , H1 and H2 are the numbers of neurons of the first hidden layer and the second hidden layer respectively, and the number of output layer neurons is 1; at the same time, the second hidden layer nodes of each subnetwork are connected to the output nodes of other subnetworks; Set the sparsity Spa=0.3, and perform sparse processing on the weights between all adjacent layers of each subnetwork. Initially, the number of weights between the input layer and the first hidden layer is z k ×H1, the number of weights between the first and second hidden layers is H1×H2, and the number of weights between the second hidden layer and the output layer is H2×K. Then, the weights in each adjacent layer are sorted in ascending order, and the rth sorted weight is recorded as the pruning threshold λ=S r , where r=(1-Spa)×(the number of weights in the layer). The weights in the layer that are smaller than λ are deleted, and the deleted weights are considered to be dormant weights. Let the number of deleted nodes in the first hidden layer and the second hidden layer of the kth subnetwork after sparsification be and respectively, and satisfy and are the number of remaining neurons in the first hidden layer and the second hidden layer of the kth subnetwork, respectively, and satisfy ​ Step 2.3: Assume the output of the first hidden layer of the kth subnetwork Zk k , for the nth input sample is At this time, the output of the h1th neuron of the first hidden layer of the kth subnetwork is: wherein, is the connection weight of the jth node of the input layer and the h1th neuron of the first hidden layer in the kth subnetwork, B() is the activation function of the h1th hidden layer neuron of the subnetwork, defined as shown in equation (8): Step 2.4: calculate the output of the h2th neuron in the second hidden layer of the kth subnetwork: wherein, is the connection weight of the h1th node of the first hidden layer and the h2th neuron of the second hidden layer in the kth subnetwork; Step 2.5: calculate the output of the kth subnetwork: wherein, is the connection weight of the h2th node of the second hidden layer in the kth subnetwork and the output node of the kth subnetwork. Step 2.6: calculate the output of the interconnection modular neural network: wherein Yk,nis the output of the kth subnetwork for the nth sample, and K is the number of subnetworks. Step 2.7: select the mean square error function as the objective function of the interconnection modular neural network: where d n is the expected output for the n-th sample of the output variable; Step 3: self-organizing design of interconnection modular neural network structure Step 3.1: Set the number of structure self-organization times l max = 8, let l = 1; Step 3.2: calculate the output error of the current nth sample: e n = d n - Y n n = 1, 2,..., N (13) Step 3.3: under the current neural network structure, let the vector Δ contain all the weights in all subnetworks: Δ = [Δ1, Δ2,..., Δ K ](14) wherein, Δ k is the parameter in the kth subnetwork, including the connection weight between layers and The parameter update formula is as follows: Δ(t + 1) = Δ(t) - (Q(t) + μ(t)I) -1 g(t) (16) Wherein t represents the iteration step number, Q is the quasi-Hessian matrix, g is the gradient vector, I is the unit matrix, and μ is the learning rate; The quasi-Hessian matrix and the gradient vector are calculated according to formulas (17) and (18) respectively: where e n is the neural network output error for the nth sample, computed according to equation (13), j n is the Jacobian matrix row vector for the corresponding sample, defined as follows: j n = [j n,1 , j n,2 ..., j n,K ](19) where j n,k is the Jacobian row vector of the kth subnetwork, defined as follows: According to formulas (7)-(11)(13), we get: Through formulas (21)-(23), the row vector j of the Jacobian matrix can be obtained n,k After all the training samples are traversed, the quasi-Hessian matrix Q and the gradient vector g are obtained, and then the connection weights in all sub-networks contained in the vector Δ are updated according to the update formula (16). During the training process, the training objective function values ​​E of the neural network at time t and time t+1 are calculated using formula (12). tr (t) and E tr (t+1), when E tr (t+1)≤E tr When μ(t) = μ(t), the learning rate μ(t+1) = μ(t) / 10, and all mean parameters in the network are preserved; conversely, when μ(t+1) = μ(t) × 10, all mean parameters in the network are restored to their state before the update, and the neural network parameters are updated based on the current μ. At the same time, the objective function value E of the neural network on the validation set at each time step is recorded during training. va Let the maximum number of iterations be T. max =40, the expected error value is E d =0.08; The parameter learning process involves continuous iteration, and when the number of iterations t = T max When the training error E tr >E d Skip to step 3.4; if the current training error E tr ≤E d But |E tr -E va If |>0.01, and |·| represents the absolute value, then jump to step 3.5; if the current training error E tr ≤E d And |E tr -E va If |≤0.01, then training stops; Step 3.4: calculate the gradient of all dormant weights between adjacent layers according to the following formula: wherein, and Here, respectively represent the dormant connection weights between neurons in each layer. Calculate the gradient between cross-layer neurons, the gradient across one layer of neurons is: The gradient across two layers of neurons is: wherein represents a connection weight between the jth input neuron of the input layer and the h2nd neuron of the second hidden layer, represents a connection weight between the h1st input neuron of the first hidden layer and the kth neuron of the output layer; If l>l max Go to step 4; If l<l max , find the maximum value of the gradient of the dormant weights between adjacent layers and the gradient between inter-layer neurons in all sub-networks through the calculation of step 3.4, if the maximum value appears in and , then re-activate the dormant weights corresponding to the maximum gradient value and assign a random value in the interval of 0 to 1, l=l+1, return to step 3.2; if the maximum gradient value appears in , and the number of neurons in the first hidden layer is less than , add 1 neuron to the first hidden layer of the kth sub-network, set the input weight and output weight to a random number between 0 and 1 respectively, l=l+1, return to step 3.2, otherwise if the number of neurons in the first hidden layer is equal to , go to step 4; if the maximum gradient value appears in , and the number of neurons in the second hidden layer is less than , add 1 neuron to the second hidden layer of the kth sub-network, set the input weight and output weight to a random number between 0 and 1 respectively, l=l+1, return to step 3.2, otherwise if the number of neurons in the second hidden layer is equal to , go to step 4; Step 3.5: Set the weight reduction threshold Thw = 0.1, and judge the connection weight between each layer of each sub-network in turn and whether there is a value less than Thw, if there is, delete, l = l + 1, return to step 3.2, otherwise end; Step 4: input the test sample data into the self-organizing interconnection modular neural network after training to get the output of the self-organizing interconnection modular neural network, and perform inverse normalization to get the predicted value of effluent BOD concentration.

Images