A method for predicting effluent ammonia nitrogen concentration based on fast multi-layer feedforward small-world neural network

Abstract

A method for predicting effluent ammonia and nitrogen based on a fast, multi-layer feedforward, small-world neural network, achieving accurate prediction of effluent ammonia and nitrogen concentration, is a key branch of advanced manufacturing technology, encompassing both the control and water treatment fields. By designing a fast, multi-layer feedforward, small-world neural network, this method accurately predicts effluent ammonia and nitrogen concentration based on data collected during the sewage treatment process. This solves the problem of difficult real-time measurement of effluent ammonia and nitrogen concentration during sewage treatment, improving real-time monitoring of water quality and decision-making flexibility in urban sewage treatment plants.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method for predicting effluent ammonia and nitrogen concentration based on a fast multi-layer feedforward small-world neural network. Predicting effluent ammonia and nitrogen concentration is an important branch of advanced manufacturing technology, encompassing both the control and water treatment fields. Background Art

[0002] With the acceleration of urbanization and industrialization, water pollution is also increasing, significantly impacting human survival and development, as well as society's ecological balance. In the urban sewage treatment industry, ammonia nitrogen in wastewater is a nutrient that can cause eutrophication. As sewage treatment plants expand in construction and operation, they face the crucial task of reducing the total amount of ammonia nitrogen in nature. Therefore, strictly limiting ammonia nitrogen emissions can effectively address this issue. Predicting effluent ammonia nitrogen concentrations can improve control and decision-making flexibility in sewage treatment plants.

[0003] By extracting historical data and enabling early prediction, this approach provides an efficient and rapid solution for predicting key water quality parameters during sewage treatment. Neural networks, due to their universal approximation properties, have become a mainstream prediction model. This paper designs a fast, multi-layer, feedforward, small-world neural network method for predicting effluent ammonia and nitrogen, enabling accurate prediction of effluent ammonia and nitrogen concentrations. Summary of the Invention

[0004] The present invention obtains a method for predicting effluent ammonia nitrogen concentration based on a fast multi-layer feedforward small-world neural network. It realizes accurate prediction of effluent ammonia nitrogen based on data collected during the sewage treatment process, solves the problem of difficulty in real-time and accurate measurement of effluent ammonia nitrogen during the sewage treatment process, and improves the flexibility of control and decision-making of urban sewage treatment plants.

[0005] A method for predicting effluent ammonia nitrogen concentration based on a fast multi-layer feedforward small-world neural network, the method comprising the following steps:

[0006] Step 1: Select historical data of effluent ammonia nitrogen concentration and preprocess the data;

[0007] First, the historical data of effluent ammonia nitrogen concentration are normalized to [0,1] according to formula (1).

[0008]

[0009] Where y is a time series consisting of historical data of effluent ammonia nitrogen concentration, y(t) is the effluent ammonia nitrogen concentration at time t, min(y) and max(y) represent the minimum and maximum values ​​in the sequence y, respectively;

[0010] Secondly, the effluent ammonia nitrogen concentration y(t-τ)~y(t) before the time τ is selected as the network input, τ is set to [3,8], and the effluent ammonia nitrogen concentration y(t+1) at the next moment is predicted. That is, the input of the model is y(t-τ)~y(t), and y(t+1) is the expected output of the model;

[0011] Step 2: Design the topology of a fast multi-layer feedforward small-world neural network model;

[0012] Step 2.1: Design a fast multi-layer feedforward small-world neural network model wiring method;

[0013] Randomly disconnect the regular connections in the multi-layer feedforward neural network and construct a feedforward small-world neural network according to the reconnection rule of prioritizing long-distance connections. The specific construction process is as follows: construct a regularly connected L-layer feedforward neural network, randomly select a connection between adjacent layers, and then disconnect it; assume that the connection is from neuron i in layer l to neuron j in layer l+1, and its weight is is set to zero; secondly, a cross-layer connection is introduced with a certain probability That is, the connection from neuron i in the lth layer to neuron c in the sth layer, the probability value is calculated as follows,

[0014]

[0015] Where, when l = 1, D is equal to L-1, otherwise D is equal to L, and s is the number of layers where cross-layer connections occur. is the cumulative sum of all possible values ​​of s;

[0016] Repeat the above steps until the number of rewired cross-layer connections reaches the set value V max , take [10,100];

[0017] Step 2.2: Design neuron activation functions for a fast multi-layer feedforward small-world neural network.

[0018] The designed feedforward small-world neural network topology has a total of L layers, including input layer, hidden layer, and output layer; the neuron activation function of each layer is designed as follows:

[0019] ① Input layer: This layer has n neurons, representing n inputs, expressed as x = [x1, x2, x3…x n ] T , where x n Represents the nth input variable of the input layer. The activation function of the neurons in this layer is set to linear. Then the output of neuron i in the input layer is for,

[0020]

[0021] Among them, xi Represents the i-th input variable of the input layer;

[0022] ② Hidden layer: This layer has multiple layers and implements nonlinear mapping through its nonlinear activation function. The output of neuron j in layer l is,

[0023]

[0024] Among them, n s represents the number of neurons in the sth layer of the neural network, represents the connection weight between neuron i in the sth layer of the neural network and neuron j in the lth layer, is the output of neuron i in layer s, and f(·) is the Sigmoid function;

[0025] ③ Output layer: The output layer contains one neuron, and the activation function is linear, so its output is,

[0026]

[0027] Among them, n s represents the number of neurons in the sth layer of the neural network, Represents the connection weight between neuron i in the sth layer of the neural network and the neurons in the output layer, is the output of neuron i in layer s;

[0028] Step 3: Initialize the weights of the fast multi-layer feedforward small-world neural network;

[0029] Step 3.1: Initialize the connection weights of the hidden layer neurons;

[0030] Use the ridge regression algorithm to initialize the connection weights of the hidden layer neurons to ensure that the hidden layer neurons are in an activated state;

[0031] Use ridge regression to initialize the neuron connection weights and define the loss function as,

[0032]

[0033] in, is a vector representing the weights of all neurons connected to neuron j in layer l, H is the output matrix storing these neurons, μ1 is the regularization factor, μ1 takes [0.01, 001], and t is a vector sampled uniformly in the activation interval [-4.36, 4.36]; The calculation method is as follows,

[0034]

[0035] Among them, I is the identity matrix, H is the output matrix that stores these neurons, and HT is the transposed matrix of H, t is a vector sampled uniformly in the activation interval [-4.36, 4.36], μ1 is the regularization factor, μ1 takes [0.01, 001];

[0036] Repeat the above method until the connection weights of all hidden layer neurons are initialized;

[0037] Step 3.2: Initialize the connection weights of the output layer neurons;

[0038] The activation function of the output layer neurons is linear and there is no activation dead zone, so its connection weights are uniformly sampled in the interval [-1, 1];

[0039] Step 4: Design a fast multi-layer feedforward small-world neural network training algorithm;

[0040] Based on the characteristics of neuron activation functions, the fast multi-layer feedforward small-world neural network can be viewed as a cascade of two sub-networks: a deep sub-network consisting of the input layer and hidden layers, and a wide sub-network consisting of the output layer and the neurons directly connected to it. The weight connections of the wide sub-network are linear and trained using ridge regression. The weight connections of the deep sub-network are nonlinear and subsequently trained using gradient descent. The details are described as follows:

[0041] Step 4.1: Ridge regression training width sub-network;

[0042] The output layer of the fast multilayer feedforward small-world neural network is a linear summation of its input, so the ridge regression method is used to learn its output weight w out The loss function is defined as,

[0043] L1(w out )=||H o w out -y d || 2 +||μ2w out || 2 (8)

[0044] Then w at time t+1 out The calculation method is as follows,

[0045]

[0046] Where t is the number of iterations, H o (t) is the matrix that stores the output of the neuron connected to the output layer at time t, It is H o The transposed matrix of (t), y d is the expected output of the network, I is the identity matrix, μ2 is the regularization factor, and μ2 is [0.1×10-3 ,0.1×10 -5 ], which not only makes the inversion operation feasible but also prevents the overfitting problem;

[0047] Step 4.2: Gradient descent training of the deep sub-network;

[0048] When obtaining the output weight w out After that, we use gradient descent to train the deep sub-network; first, we define the loss function as,

[0049]

[0050] Among them, y d is the expected output of the network, y L is the actual output vector of the network, then the actual output vector y of the network at time t+1 L (t+1) is,

[0051]

[0052] in, It is the transposed matrix that stores the output of the neuron connected to the output layer at time t. Then, the weights of the deep sub-network are updated by formula (12).

[0053] W(t+1)=W(t)+△W(t) (12)

[0054] Among them, W(t) is the weight matrix at time t, W(t+1) is the weight matrix at time t+1, and △W(t) is the weight change matrix at time t. Its elements are calculated as follows:

[0055]

[0056] in, represents the change in the connection weight between the lth layer neuron i and the sth layer neuron j at time t, L2(W) is the loss function, is the connection weight between neuron i in layer l and neuron j in layer s at time t, λ is the learning rate, is the output of neuron i in layer l at time t, is the error of neuron j in layer s at time t, which is calculated as follows,

[0057]

[0058] Among them, n d is the number of neurons in the dth layer, is the connection weight between neuron j in the sth layer and neuron c in the dth layer at time t, is the error of neuron c in layer d at time t, is the connection weight between the neuron j in the sth layer and the neuron in the output layer at time t+1, is the output of neuron j in layer s at time t, δ L (t+1) is the output error of the network, which is calculated as follows,

[0059]

[0060] Where M is the total sample size, is the true output of the network corresponding to the mth sample point at time t+1, is the expected output corresponding to the mth sample point;

[0061] Step 4.3: Calculate the training RMSE. If the RMSE is less than the expected training RMSE or the number of iterations reaches the maximum number of iterations, stop the calculation. The maximum number of iterations is in the range of [5000, 10000]. Otherwise, jump to step 4.1. The definition of RMSE is shown in formula (16).

[0062]

[0063] Where M is the total sample size, is the true output of the network corresponding to the mth sample point, is the expected output corresponding to the mth sample point;

[0064] Step 5: Prediction of effluent ammonia nitrogen;

[0065] The test sample data is used as the input of the trained fast multi-layer feedforward small-world neural network. After obtaining the output of the neural network, it is denormalized to obtain the predicted value of the effluent ammonia nitrogen concentration.

[0066] Compared with the prior art, the present invention has the following obvious advantages and beneficial effects:

[0067] (1) Aiming at the problem that ammonia nitrogen concentration, a key water quality parameter in the current sewage treatment process, is difficult to measure, the present invention designs a fast multi-layer feedforward small-world neural network model, which can achieve accurate prediction of effluent ammonia nitrogen concentration with high precision, good stability and strong generalization ability.

[0068] (2) To address the high training time cost of traditional multi-layer feedforward small-world neural networks, the present invention proposes a construction method that prioritizes long-distance connections, weight initialization based on ridge regression, and a hybrid training algorithm based on ridge regression and gradient descent to improve network training efficiency, thereby greatly shortening the training time. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a neural network topology diagram of the present invention;

[0070] Figure 2 This is a graph showing changes in the root mean square error (RMSE) of the training method for predicting effluent ammonia nitrogen concentration according to the present invention;

[0071] Figure 3 This is a diagram showing the prediction results of the effluent ammonia nitrogen concentration according to the present invention; DETAILED DESCRIPTION

[0072] The present invention has obtained a method for predicting effluent ammonia nitrogen concentration based on a fast multi-layer feedforward small-world neural network. The method realizes the prediction of effluent ammonia nitrogen concentration based on data collected during the sewage treatment process, solves the problem of difficulty in real-time measurement of effluent ammonia nitrogen concentration during the sewage treatment process, and improves the level of real-time monitoring of water quality in urban sewage treatment plants.

[0073] The experimental data comes from the water quality analysis data of a sewage treatment plant from September 16 to September 22, 2014. There are a total of 1064 effluent ammonia nitrogen concentration samples. After data reconstruction, the first 740 data were selected as training samples, and the remaining 317 data were used as test samples.

[0074] A method for predicting effluent ammonia nitrogen concentration based on a fast multi-layer feedforward small-world neural network, the method comprising the following steps:

[0075] Step 1: Select historical data of effluent ammonia nitrogen concentration and preprocess the data;

[0076] First, the historical data of effluent ammonia nitrogen concentration are normalized to [0,1] according to formula (1).

[0077]

[0078] Where y is a time series consisting of historical data of effluent ammonia nitrogen concentration, y(t) is the effluent ammonia nitrogen concentration at time t, min(y) and max(y) represent the minimum and maximum values ​​in the sequence y, respectively;

[0079] Secondly, the effluent ammonia nitrogen concentration y(t-τ)~y(t) before the time τ is selected as the network input, τ is generally taken as [3,8], and the effluent ammonia nitrogen concentration y(t+1) at the next moment is predicted, that is, the input of the model is y(t-τ)~y(t), and y(t+1) is the expected output of the model;

[0080] In this embodiment, τ=6 is set based on experience, that is, the sample input dimension is 7;

[0081] Step 2: Design the topology of a fast multi-layer feedforward small-world neural network model;

[0082] Step 2.1: Design a fast multi-layer feedforward small-world neural network model wiring method;

[0083] Randomly disconnect the regular connections in the multi-layer feedforward neural network and construct a feedforward small-world neural network according to the reconnection rule of prioritizing long-distance connections. The specific construction process is as follows: construct a regularly connected L-layer feedforward neural network, randomly select a connection between adjacent layers, and then disconnect it; assume that the connection is from neuron i in layer l to neuron j in layer l+1, and its weight is is set to zero; secondly, a cross-layer connection is introduced with a certain probability That is, the connection from neuron i in the lth layer to neuron c in the sth layer, the probability value is calculated as follows,

[0084]

[0085] Where, when l = 1, D is equal to L-1, otherwise D is equal to L, and s is the number of layers where cross-layer connections occur. is the cumulative sum of all possible values ​​of s, p s increases as s increases, so long-distance connections are rewired with a higher probability;

[0086] Repeat the above steps until the number of rewired cross-layer connections reaches the set value V max , take [10,100];

[0087] In this embodiment, V max is 35;

[0088] Step 2.2: Design neuron activation functions for a fast multi-layer feedforward small-world neural network.

[0089] The designed feedforward small-world neural network topology has a total of L layers, including input layer, hidden layer, and output layer; the neuron activation function of each layer is designed as follows:

[0090] ① Input layer: This layer has n neurons, representing n inputs, expressed as x = [x1, x2, x3…x n ] T , where x n Represents the nth input variable of the input layer. The activation function of the neurons in this layer is set to linear. Then the output of neuron i in the input layer is for,

[0091]

[0092] Among them, x i Represents the i-th input variable of the input layer;

[0093] ② Hidden layer: This layer has multiple layers and implements nonlinear mapping through its nonlinear activation function. The output of neuron j in layer l is,

[0094]

[0095] Among them, n s represents the number of neurons in the sth layer of the neural network, represents the connection weight between neuron i in the sth layer of the neural network and neuron j in the lth layer, is the output of neuron i in layer s, and f(·) is the Sigmoid function;

[0096] ③ Output layer: The output layer contains one neuron, and the activation function is linear, so its output is,

[0097]

[0098] Among them, n s represents the number of neurons in the sth layer of the neural network, Represents the connection weight between neuron i in the sth layer of the neural network and the neurons in the output layer, is the output of neuron i in layer s;

[0099] In this embodiment, the number of network layers L is set to 6, the number of neurons in the input layer and the hidden layer is 6, and the number of neurons in the output layer is 1;

[0100] Step 3: Initialize the weights of the fast multi-layer feedforward small-world neural network;

[0101] Step 3.1: Initialize the connection weights of the hidden layer neurons;

[0102] Use the ridge regression algorithm to initialize the connection weights of the hidden layer neurons to ensure that the hidden layer neurons are in an activated state;

[0103] Use ridge regression to initialize the neuron connection weights and define the loss function as,

[0104]

[0105] in, is a vector representing the weights of all neurons connected to neuron j in layer l, H is the output matrix storing these neurons, μ1 is the regularization factor, μ1 takes [0.01, 001], and t is a vector sampled uniformly in the activation interval [-4.36, 4.36]; The calculation method is as follows,

[0106]

[0107] Among them, I is the identity matrix, H is the output matrix that stores these neurons, and H Tis the transposed matrix of H, t is the vector sampled uniformly in the activation interval of [-4.36, 4.36], μ1 is the regularization factor, μ1 takes [0.01, 001], and its regularization effect can effectively avoid the Too large a value can lead to gradient explosion during training;

[0108] Repeat the above method until the connection weights of all hidden layer neurons are initialized;

[0109] In this embodiment, μ1 is set to 0.06;

[0110] Step 3.2: Initialize the connection weights of the output layer neurons;

[0111] The activation function of the output layer neurons is linear and there is no activation dead zone, so its connection weights are uniformly sampled in the interval [-1, 1];

[0112] Step 4: Design a fast multi-layer feedforward small-world neural network training algorithm;

[0113] Based on the characteristics of neuron activation functions, the fast multi-layer feedforward small-world neural network can be viewed as a cascade of two sub-networks: a deep sub-network consisting of the input layer and hidden layers, and a wide sub-network consisting of the output layer and the neurons directly connected to it. The weight connections of the wide sub-network are linear and trained using ridge regression. The weight connections of the deep sub-network are nonlinear and subsequently trained using gradient descent. The details are described as follows:

[0114] Step 4.1: Ridge regression training width sub-network;

[0115] The output layer of the fast multilayer feedforward small-world neural network is a linear summation of its input, so the ridge regression method is used to learn its output weight w out The loss function is defined as,

[0116] L1(w out )=||H o w out -y d || 2 +||μ2w out || 2 (8)

[0117] Then w at time t+1 out The calculation method is as follows,

[0118]

[0119] Where t is the number of iterations, H o (t) is the matrix that stores the output of the neuron connected to the output layer at time t, It is H o The transposed matrix of (t), y d is the expected output of the network, I is the identity matrix, μ2 is the regularization factor, and μ2 is [0.1×10 -3 ,0.1×10 -5 ], which not only makes the inversion operation feasible but also prevents the overfitting problem; in this embodiment, μ2 is set to 0.5×10 -4 .

[0120] Step 4.2: Gradient descent training of the deep sub-network;

[0121] When obtaining the output weight w out After that, we use gradient descent to train the deep sub-network; first, we define the loss function as,

[0122]

[0123] Among them, y d is the expected output of the network, y L is the actual output vector of the network, then the actual output vector y of the network at time t+1 L (t+1) is,

[0124]

[0125] in, It is the transposed matrix that stores the output of the neuron connected to the output layer at time t. Then, the weights of the deep sub-network are updated by formula (12).

[0126] W(t+1)=W(t)+△W(t) (12)

[0127] Among them, W(t) is the weight matrix at time t, W(t+1) is the weight matrix at time t+1, and △W(t) is the weight change matrix at time t. Its elements are calculated as follows:

[0128]

[0129] in, represents the change in the connection weight between the lth layer neuron i and the sth layer neuron j at time t, L2(W) is the loss function, is the connection weight between neuron i in layer l and neuron j in layer s at time t, and λ is the learning rate. In this embodiment, λ is set to 0.3×10 -3 , is the output of neuron i in layer l at time t, is the error of neuron j in layer s at time t, which is calculated as follows,

[0130]

[0131] Among them, n d is the number of neurons in the dth layer, is the connection weight between neuron j in the sth layer and neuron c in the dth layer at time t, is the error of neuron c in layer d at time t, is the connection weight between the neuron j in the sth layer and the neuron in the output layer at time t+1, is the output of neuron j in layer s at time t, δ L (t+1) is the output error of the network, which is calculated as follows,

[0132]

[0133] Where M is the total sample size, is the true output of the network corresponding to the mth sample point at time t+1, is the expected output corresponding to the mth sample point;

[0134] Step 4.3: Calculate the training RMSE. If the RMSE is less than the expected training RMSE or the number of iterations reaches the maximum number of iterations, stop the calculation. The maximum number of iterations is in the range of [5000, 10000]. Otherwise, jump to step 4.1. The definition of RMSE is shown in formula (16).

[0135]

[0136] Where M is the total sample size, is the true output of the network corresponding to the mth sample point, is the expected output corresponding to the mth sample point;

[0137] In this embodiment, the maximum number of iterations is set to 10000, the expected training RMSE is set to 0.031, and the training RMSE change diagram is as follows: Figure 2 As shown, X-axis: training steps, Y-axis: training RMSE.

[0138] Step 5: Prediction of effluent ammonia nitrogen;

[0139] The test sample data is used as the input of the trained fast multi-layer feedforward small-world neural network. After obtaining the output of the neural network, it is denormalized to obtain the predicted value of the effluent ammonia nitrogen concentration.

[0140] In this embodiment, the prediction results are as follows: Figure 3As shown in the figure, the X-axis represents the test sample (unit: sample / sample), and the Y-axis represents the effluent ammonia nitrogen concentration (unit: mg / L). The solid line represents the actual output value of the effluent ammonia nitrogen concentration, and the dotted line represents the predicted output value of the effluent ammonia nitrogen concentration. The results demonstrate the effectiveness of the effluent ammonia nitrogen concentration prediction method based on a fast multilayer feedforward small-world neural network.

Claims

1. A method for predicting effluent ammonia nitrogen concentration based on a fast multi-layer feedforward small-world neural network, the method comprising the following steps: Step 1: Select historical data of effluent ammonia nitrogen concentration and preprocess the data; First, the historical data of effluent ammonia nitrogen concentration are normalized to [0,1] according to formula (1). Where y is a time series consisting of historical data of effluent ammonia nitrogen concentration, y(t) is the effluent ammonia nitrogen concentration at time t, min(y) and max(y) represent the minimum and maximum values ​​in the sequence y, respectively; Secondly, the effluent ammonia nitrogen concentration y(t-τ)~y(t) before the time τ is selected as the network input, τ is set to [3,8], and the effluent ammonia nitrogen concentration y(t+1) at the next moment is predicted. That is, the input of the model is y(t-τ)~y(t), and y(t+1) is the expected output of the model; Step 2: Design the topology of a fast multi-layer feedforward small-world neural network model; Step 2.1: Design a fast multi-layer feedforward small-world neural network model wiring method; Randomly disconnect the regular connections in the multi-layer feedforward neural network and construct a feedforward small-world neural network according to the reconnection rule of prioritizing long-distance connections. The specific construction process is as follows: construct a regularly connected L-layer feedforward neural network, randomly select a connection between adjacent layers, and then disconnect it; assume that the connection is from neuron i in layer l to neuron j in layer l+1, and its weight is is set to zero; secondly, a cross-layer connection is introduced with a certain probability That is, the connection from neuron i in the lth layer to neuron c in the sth layer, the probability value is calculated as follows, Where, when l = 1, D is equal to L-1, otherwise D is equal to L, and s is the number of layers where cross-layer connections occur. is the cumulative sum of all possible values ​​of s, p s Increases as s increases; Repeat the above steps until the number of rewired cross-layer connections reaches the set value V max , take [10,100]; Step 2.2: Design neuron activation functions for a fast multi-layer feedforward small-world neural network. The designed feedforward small-world neural network topology has a total of L layers, including input layer, hidden layer, and output layer; the neuron activation function of each layer is designed as follows: ① Input layer: This layer has n neurons, representing n inputs, expressed as x = [x1, x2, x3…x n ] T , where x n Represents the nth input variable of the input layer. The activation function of the neurons in this layer is set to linear. Then the output of neuron i in the input layer is for, Among them, x i Represents the i-th input variable of the input layer; ② Hidden layer: This layer has multiple layers and implements nonlinear mapping through its nonlinear activation function. The output of neuron j in layer l is, Among them, n s represents the number of neurons in the sth layer of the neural network, represents the connection weight between neuron i in the sth layer of the neural network and neuron j in the lth layer, is the output of neuron i in layer s, and f(·) is the Sigmoid function; ③ Output layer: The output layer contains one neuron, and the activation function is linear, so its output is, Among them, n s represents the number of neurons in the sth layer of the neural network, Represents the connection weight between neuron i in the sth layer of the neural network and the neurons in the output layer, is the output of neuron i in layer s; Step 3: Initialize the weights of the fast multi-layer feedforward small-world neural network; Step 3.1: Initialize the connection weights of the hidden layer neurons; Use the ridge regression algorithm to initialize the connection weights of the hidden layer neurons to ensure that the hidden layer neurons are in an activated state; Use ridge regression to initialize the neuron connection weights and define the loss function as, in, is a vector representing the weights of all neurons connected to neuron j in layer l, H is the output matrix storing these neurons, μ1 is the regularization factor, μ1 takes [0.01, 001], and t is a vector sampled uniformly in the activation interval [-4.36, 4.36]; The calculation method is as follows, Among them, I is the identity matrix, H is the output matrix that stores these neurons, and H T is the transposed matrix of H, t is a vector sampled uniformly in the activation interval [-4.36, 4.36], μ1 is the regularization factor, μ1 takes [0.01, 001]; Repeat the above method until the connection weights of all hidden layer neurons are initialized; Step 3.2: Initialize the connection weights of the output layer neurons; The activation function of the output layer neurons is linear and there is no activation dead zone, so its connection weights are uniformly sampled in the interval [-1, 1]; Step 4: Design a fast multi-layer feedforward small-world neural network training algorithm; Based on the characteristics of neuron activation functions, the fast multi-layer feedforward small-world neural network can be viewed as a cascade of two sub-networks: a deep sub-network consisting of the input layer and hidden layers, and a wide sub-network consisting of the output layer and the neurons directly connected to it. The weight connections of the wide sub-network are linear and trained using ridge regression. The weight connections of the deep sub-network are nonlinear and subsequently trained using gradient descent. The details are described as follows: Step 4.1: Ridge regression training width sub-network; The output layer of the fast multilayer feedforward small-world neural network is a linear summation of its input, so the ridge regression method is used to learn its output weight w out； The loss function is defined as, L1(w out )=||H o w out -y d || 2 +||μ2w out || 2 (8) Then w at time t+1 out The calculation method is as follows, Where t is the number of iterations, H o (t) is the matrix that stores the output of the neuron connected to the output layer at time t, It is H o The transposed matrix of (t), y d is the expected output of the network, I is the identity matrix, μ2 is the regularization factor, and μ2 is [0.1×10 -3 ,0.1×10 -5 ]; Step 4.2: Gradient descent training of the deep sub-network; When obtaining the output weight w out After that, we use gradient descent to train the deep sub-network; first, we define the loss function as, Among them, y d is the expected output of the network, y L is the actual output vector of the network, then the actual output vector y of the network at time t+1 L (t+1) is, in, It is the transposed matrix that stores the output of the neuron connected to the output layer at time t. Then, the weights of the deep sub-network are updated by formula (12). W(t+1)=W(t)+△W(t) (12) Among them, W(t) is the weight matrix at time t, W(t+1) is the weight matrix at time t+1, and △W(t) is the weight change matrix at time t. Its elements are calculated as follows: in, represents the change in the connection weight between the lth layer neuron i and the sth layer neuron j at time t, L2(W) is the loss function, is the connection weight between neuron i in layer l and neuron j in layer s at time t, λ is the learning rate, is the output of neuron i in layer l at time t, is the error of neuron j in layer s at time t, which is calculated as follows, Among them, n d is the number of neurons in the dth layer, is the connection weight between neuron j in the sth layer and neuron c in the dth layer at time t, is the error of neuron c in layer d at time t, is the connection weight between the neuron j in the sth layer and the neuron in the output layer at time t+1, is the output of neuron j in layer s at time t, δ L (t+1) is the output error of the network, which is calculated as follows, Where M is the total sample size, is the true output of the network corresponding to the mth sample point at time t+1, is the expected output corresponding to the mth sample point; Step 4.3: Calculate the training RMSE. If the RMSE is less than the expected training RMSE or the number of iterations reaches the maximum number of iterations, stop the calculation. The maximum number of iterations is in the range of [5000, 10000]. Otherwise, jump to step 4.

1. The definition of RMSE is shown in formula (16). Where M is the total sample size, is the true output of the network corresponding to the mth sample point, is the expected output corresponding to the mth sample point; Step 5: Prediction of effluent ammonia nitrogen; The test sample data is used as the input of the trained fast multi-layer feedforward small-world neural network. After obtaining the output of the neural network, it is denormalized to obtain the predicted value of the effluent ammonia nitrogen concentration.

Images