Atmospheric pollutant concentration prediction method of self-organizing recurrent neural network based on Bayesian optimization

Abstract

The invention discloses an atmospheric pollutant concentration prediction method of a self-organizing recurrent neural network based on Bayesian optimization, and belongs to the technical field of intelligent environmental protection. At present, a method for predicting the concentration of atmospheric pollutants has some problems including complex models, low prediction precision, insufficient space-time correlation, poor real-time performance and the like. Therefore, the invention designs a self-organizing recurrent neural network model based on Bayesian optimization. According to the model, the correlation between various meteorological factors and atmospheric pollutants is deeply mined by using the advantages of a gated circulation unit neural network; an intelligent tuning mechanism based on a Bayesian optimization algorithm is designed to realize preliminary optimization of model parameters and structures, so that the efficiency of the model is improved; and then the contribution degree of each neuron of a hidden layer to output is calculated by adopting sensitivity analysis, so that the network structure and parameters are adaptively adjusted, and the robustness and accuracy of the model are further improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent environmental protection and information prediction, and specifically relates to a method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network model with Bayesian optimization. Background Art

[0002] There are six main types of common air pollutants: particles with diameters equal to or less than 10 μm and 2.5 μm (PM 10 and PM 2.5 ), nitrogen oxides (NO X ), sulfur dioxide (SO2), ozone (O3), carbon monoxide (CO). Long-term exposure to air pollution can have serious effects on human health. For children, this may hinder the development of the heart and lungs and lead to limited lung function; for adults, it may cause respiratory infections, and in severe cases may develop into cardiovascular diseases or lung infections. Therefore, reducing the concentration of air pollutants in the environment has become one of the key issues that my country urgently needs to solve. An important prerequisite for improving air quality is to predict and analyze air quality. The prediction of air pollutant concentrations is crucial for air pollution prevention and control. It can guide the formulation of effective prevention and control measures to reduce harm to health and the environment.

[0003] The more traditional air quality prediction method is based on numerical models, which simulate the emission, diffusion, transformation and removal of air pollutants through meteorological principles and statistical methods, thereby predicting the concentration of pollutants. Although the numerical model fully considers the physical and chemical changes and transmission processes of pollutants in the atmosphere, there are still uncertainties in pollution sources, meteorological conditions and transformation processes. In addition, the complexity and large amount of calculation of the numerical model are also one of the challenges it faces.

[0004] Compared with the numerical model prediction method, the prediction method based on the data-driven model is simpler, more efficient, and has a wide range of applications. It makes predictions by learning and analyzing historical pollutant data. Among them, the deep learning model can obtain better robustness through deeper hidden layers and excellent self-learning ability, explore high-order nonlinear mapping relationships, and through the multi-level structure of the deep neural network, the model can extract abstract features from the data step by step, and perform information fusion at different levels, thereby capturing the complex relationship between pollutant concentrations and various influencing factors. Therefore, there is great potential in constructing a deep learning model for pollutant concentration prediction. The present invention designs a method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization. Summary of the invention

[0005] In view of the shortcomings of existing methods, the present invention proposes a method for predicting atmospheric pollutants based on a gated recurrent unit (GRU) neural network model. The GRU neural network is used to fully explore the correlation characteristics between various meteorological factors and atmospheric pollutants, and the Bayesian optimization algorithm is used to automatically adjust the hyperparameters of the model to improve the learning and generalization capabilities of the model. Then, the sensitivity analysis is used to dynamically adjust the network structure of the model to further improve the robustness and accuracy of the model.

[0006] A method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization, characterized in that it comprises the following steps:

[0007] Step 1: Data Preprocessing

[0008] In the process of model prediction, in order to ensure the uniformity between data indicators, it is necessary to standardize the data and use normalization to map the data values ​​to [0,1], eliminate the order of magnitude differences between data in each dimension, and avoid the influence of excessive network prediction errors caused by large order of magnitude differences between input and output data. The calculation formula is:

[0009]

[0010] Where, X * is the normalized data; X is the sample data; X min , X max are the minimum and maximum values ​​of the samples respectively.

[0011] Step 2: Design the initial network structure

[0012] This paper uses the GRU neural network as the basic network, which consists of an input layer, a double hidden layer, and a fully connected output layer. The input layer consists of 6 nodes, corresponding to the input vector x = (x1, x2, ..., x6) T , which represents the five meteorological factors of temperature, air pressure, humidity, wind speed and wind direction at time t and the concentration of pollutants to be predicted; the output layer has 1 node, which is used to predict the concentration of pollutants to be predicted one hour later.

[0013] Step 3: Bayesian optimization network structure

[0014] The Bayesian optimization algorithm is used to optimize and determine the number of hidden layer neurons (units), number of iterations (epochs), and optimizer hyperparameters of the prediction model. The search range of the number of hidden layer neurons is set to 10-100, the search range of the number of iterations is set to 50-100, and the optimizer search range is set to stochastic gradient descent (SGD), Adam, and AdaGrad. The root mean square error is selected as the loss function, the GRU model with different parameter combinations is used as the independent variable, and the root mean square error is used as the output of the Bayesian framework. The combination of the above hyperparameters is continuously trained and adjusted until the loss function of the model no longer decreases significantly, that is, when the loss function decreases by less than 0.001 each time in 5 consecutive iterations, the training is stopped to determine the optimal hyperparameter combination.

[0015] Step 3.1: Probabilistic surrogate model modification

[0016] First, we select a probabilistic proxy model. In this paper, we use Gaussian Process (GP) as the probabilistic proxy model. Initially, we use the Gaussian process model to establish a prior estimate of the objective function based on the existing training data. The Gaussian distribution is expressed as:

[0017] f(x)~GP(μ(x),k(x,x′)) (2)

[0018] In the formula, f(x) is a Gaussian process; μ(x) is the mean function, which is set to 0; k(x, x′) is the covariance function of x. Then, based on the existing Gaussian process model, the mean and variance of the predicted value at the current unknown point are calculated. The formula is as follows:

[0019]

[0020]

[0021] Where: x * represents the input point corresponding to the predicted value of the objective function; μ(x * ) means that the function is in x * The mean value at c ov (x * ) means that the function is in x * The covariance at the test point; c is the covariance of the test point itself; c * is the covariance vector between the test point and the training data point; C is the covariance matrix between the training data points; is the variance of Gaussian noise; I is the unit matrix; b is the target value vector of the training data point, that is, the true output vector.

[0022] Finally, based on the newly observed objective function value, the posterior distribution of the Gaussian process model is updated, and the mean and covariance are updated using Bayes’ theorem. The mean represents the final expected effect of this point, and the larger the mean, the better the effect. The variance represents the uncertainty of this point’s effect, and the larger the variance, the greater the uncertainty. This further refines the estimate of the objective function.

[0023] Step 3.2: Use the acquisition function to select the next sample point

[0024] First, an acquisition function should be selected in Bayesian optimization to correspond to the balance between exploration and exploitation strategies. The acquisition function used in this paper is the expected improvement (EI) function. This function can evaluate the expected improvement of each sampling point.

[0025] Then, for the expected increment (EI) function, the expected increment value of each candidate point is calculated. The expected increment represents the expected increase in the value of the objective function after sampling the point. The formula is as follows:

[0026]

[0027] x * = argmax x EI(x) (6)

[0028] After conversion, we can get:

[0029] EI(x)=σ(x)φ(z)+(μ(x)-f(x * ))Φ(z)(7)

[0030]

[0031] in, is the expected value, f(x * ) is the current maximum value, μ is the mean, σ is the standard deviation, Φ(z) and φ(z) represent the normal cumulative distribution function and probability density function of the standard normal random variable, respectively. The expected increment function shows that some points are greater than f(x * ), that is, the degree of improvement of selecting this point as the next sampling point compared to the current sampling point. This acquisition function integrates both the improvement probability and the different improvement amounts.

[0032] According to the value of the acquisition function, select the candidate point that maximizes the acquisition function as the next sample point. Add the new sample point to the training data and update the Gaussian process model. Repeat steps 3.1-3.2 until the loss function decreases by less than 0.001 each time in 5 consecutive iterations, then stop training, thereby effectively finding the global optimal solution or the configuration parameters close to the optimal solution.

[0033] Step 4: Construct the self-organizing mechanism of the neural network

[0034] The sensitivity analysis method based on Fourier transform is used to evaluate the contribution of neurons. First, the SA algorithm is used to calculate the contribution of hidden layer neurons to the network output. The input factor of SA is the input value of the output layer, which is expressed as:

[0035]

[0036] in, is the output of the last hidden layer, is its connection weight. f represents the calculation rule of the output layer, and the sensitivity analysis model is expressed as:

[0037]

[0038] in, are m input factors, is the model output; the input factor I j (t) is independent of the contribution of other input parameters to the output and is denoted by The calculation formula is as follows:

[0039]

[0040] in, yes The variance of The input variable is hour expectations, is the variance of Y(t);

[0041] The sensitivity analysis model is transferred to the frequency domain for study and combined The independent effect on the output and the synergistic effect with other input parameters are recorded as the total sensitivity The calculation method is as follows:

[0042]

[0043] in, Indicated in The Fourier coefficients of the cosine signal at frequency, Indicated in The Fourier coefficients of a sinusoidal signal at a frequency are calculated as follows:

[0044]

[0045]

[0046] Where s represents the complex field, is the specified frequency of the jth hidden layer neuron, which is determined by the output of the jth hidden layer neuron and is calculated as follows:

[0047]

[0048] in, is the maximum output of the jth neuron that has been trained in the previous moment, is the minimum output value of the j-th neuron trained in the previous moment;

[0049] Then, the cumulative contribution of the jth neuron in the hidden layer for k consecutive moments is calculated as ST j (t):

[0050]

[0051] Finally, the obtained ST j (t) is used to guide the calculation of neuron splitting and deletion.

[0052] Step 5: Network structural parameter adjustment algorithm

[0053] The sensitivity of each neuron in the hidden layer is the contribution to the output, and the corresponding processing is performed according to the different situations of its contribution. At the same time, the implementation of the network algorithm is often related to the setting of the threshold. First, the mean and standard deviation of the contribution data set are calculated. Then a multiple is selected, where the highest threshold is equal to the mean plus the multiple times the standard deviation, and the lowest threshold is equal to the mean minus the multiple times the standard deviation. The multiple is set to 1.5-3, depending on the distribution of the data and the needs of the application.

[0054] 5.1 Neuron Splitting Algorithm

[0055] When ST j When (t)≥ε1, it indicates that the contribution of the jth neuron in the hidden layer exceeds the set maximum threshold, and the neuron has a greater impact on the output of the neural network, and the system will perform specific operations. In this case, the neuron needs to be split to increase the impact of the neuron on the output and make it more significant. At the same time, in order to ensure the convergence speed of the neural network, the newly added neurons are required to be relatively uniform, so the initial setting of the weights of the new neurons after the split is as follows:

[0056] θ new1=τθ j (t) (17)

[0057] θ new2 =(1-τ)θ j (t) (18)

[0058] Among them, θ new1 is the weight of the first newly added neuron, θ new2 is the weight of the second newly added neuron, θ j (t) is the weight of neuron j before the network structure adjustment at the tth step of training, and τ follows the standard normal distribution. After executing the neuron splitting algorithm, the number of neurons in the hidden layer increases by one.

[0059] 5.2 Neurons remain unchanged

[0060] When ε1 ≥ ST j (t)≥ε2, indicating that the contribution of the jth neuron in the hidden layer is less than the set maximum threshold and greater than the set minimum threshold. The neuron has a moderate impact on the output of the neural network. The system will keep this neuron unchanged and there is no need to perform neuron splitting or neuron deletion operations.

[0061] 5.3 Neuron Reduction Algorithm

[0062] When ST j (t)≤ε2, indicating that the contribution of the jth neuron in the hidden layer is less than the set minimum threshold, and the neuron has little effect on the output of the neural network, and the system will perform specific operations. In this case, it is necessary to prune the neuron to further reduce its influence on the output. Delete the neuron and delete all the values ​​connected to it. After executing the neuron pruning algorithm, the number of corresponding neurons in the hidden layer is reduced by one.

[0063] Step 6: Network parameter learning algorithm

[0064] The network selects the Adam optimizer to adjust the network parameters. The relevant algorithm is defined as follows:

[0065] First, initialize the exponentially weighted average m0 of the first-order gradient and the exponentially weighted average v0 of the second-order gradient of each parameter, and the deviation correction coefficient is For the mini-batch sample at the tth time step, calculate its gradient g t . Then the exponentially weighted average of the first-order and second-order gradients is updated as follows:

[0066] m t =β1·m t-1 +(1-β1)·g t (19)

[0067]

[0068] where β1 and β2 are average coefficients, set to 0.9 and 0.999 respectively. Then calculate and Bias-corrected gradient estimate:

[0069]

[0070]

[0071] in and are the t-th power of β1 and β2 respectively.

[0072] Finally update the weights:

[0073]

[0074] Where η is the learning rate and ∈ is a constant added to increase numerical stability and is set to 10 -8 .

[0075] Step 7: Network training samples and test samples

[0076] Input the training sample data and repeat steps 4-6 until all training samples are trained; input the test sample data and the output of the model is the predicted value of the atmospheric pollutant concentration.

[0077] Compared with the prior art, the present invention has the following obvious advantages and beneficial effects:

[0078] 1. The present invention uses the recurrent neural network GRU as a key technology, which can comprehensively utilize the data information of the current and historical moments, fully explore the correlation characteristics between various meteorological factors and atmospheric pollutants, and significantly improve the performance of the model in terms of spatiotemporal correlation. In addition, the Bayesian optimization algorithm is introduced to optimize the model parameters and structure, and a preliminary atmospheric pollutant concentration prediction model with strong real-time performance, high stability, and excellent prediction accuracy is established, providing effective technical support for solving complex air quality management problems.

[0079] 2. The present invention realizes the self-organizing design of the hidden layer structure of the neural network. The contribution of each neuron in the hidden layer to the network output is calculated through sensitivity analysis, and the number of neurons is dynamically adjusted according to the contribution, thereby further optimizing the neural network structure. At the same time, the present invention also introduces a network parameter learning algorithm to realize adaptive adjustment of parameters, effectively solving the problem that the traditional neural network structure is difficult to determine, while ensuring the generalization performance of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1The present invention is a flow chart of the atmospheric pollutant concentration prediction method.

[0081] Figure 2 This is the prediction result diagram of the atmospheric pollutant concentration prediction method of the present invention

[0082] Figure 3 is the prediction error diagram of the atmospheric pollutant concentration prediction method of the present invention DETAILED DESCRIPTION

[0083] The following is a detailed description of an implementation case of the present invention. This implementation case is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and specific operation process are given, but the protection scope of the present invention is not limited to the following implementation case.

[0084] The present invention adopts the air pollutant PM from March 2022 to May 2022 in a city in Shandong Province, my country 2.5 The experiment was conducted with hourly data of meteorological factors, and a total of 2,000 sets of data were obtained and arranged in chronological order. The first 1,400 sets of data were selected as training samples, and the remaining 600 sets of data were selected as test samples. The five meteorological factors that mainly affect the concentration of air pollutants in the correlation analysis are: temperature, air pressure, humidity, wind speed, and wind direction. Therefore, these five meteorological factors at time t and PM 2.5 The concentration of pollutants is used as input, and the PM 2.5 The concentration of pollutants is used as output. The specific steps are as follows:

[0085] Step 1: Data Preprocessing

[0086] In the process of model prediction, in order to ensure the uniformity between data indicators, it is necessary to standardize the data and use normalization to map the data values ​​to [0,1], eliminate the order of magnitude differences between data in each dimension, and avoid the influence of excessive network prediction errors caused by large order of magnitude differences between input and output data. The calculation formula is:

[0087]

[0088] Where, X * is the normalized data; X is the sample data; X min , X max are the minimum and maximum values ​​of the samples respectively.

[0089] Step 2: Design the initial network structure

[0090] This paper uses the GRU neural network as the basic network, which consists of an input layer, a double hidden layer, and a fully connected output layer. The input layer consists of 6 nodes, corresponding to the input vector x = (x1, x2, ..., x6) T, which represents the five meteorological factors of temperature, air pressure, humidity, wind speed and wind direction at time t and the concentration of pollutants to be predicted; the output layer has 1 node, which is used to predict the concentration of pollutants to be predicted one hour later.

[0091] Step 3: Bayesian optimization network structure

[0092] The Bayesian optimization algorithm is used to optimize and determine the number of hidden layer neurons (units), number of iterations (epochs), and optimizer hyperparameters of the prediction model. The search range of the number of hidden layer neurons is set to 10-100, the search range of the number of iterations is set to 50-100, and the optimizer search range is set to stochastic gradient descent (SGD), Adam, and AdaGrad. The root mean square error is selected as the loss function, the GRU model with different parameter combinations is used as the independent variable, and the root mean square error is used as the output of the Bayesian framework. The combination of the above hyperparameters is continuously trained and adjusted until the loss function of the model no longer decreases significantly, that is, when the loss function decreases by less than 0.001 each time in 5 consecutive iterations, the training is stopped to determine the optimal hyperparameter combination.

[0093] Step 3.1: Probabilistic surrogate model modification

[0094] First, we select a probabilistic proxy model. In this paper, we use Gaussian Process (GP) as the probabilistic proxy model. Initially, we use the Gaussian process model to establish a prior estimate of the objective function based on the existing training data. The Gaussian distribution is expressed as:

[0095] f(x)~GP(μ(x),k(x,x′)) (2)

[0096] In the formula, f(x) is a Gaussian process; μ(x) is the mean function, which is set to 0; k(x, x′) is the covariance function of x. Then, based on the existing Gaussian process model, the mean and variance of the predicted value at the current unknown point are calculated. The formula is as follows:

[0097]

[0098]

[0099] Where: x * represents the input point corresponding to the predicted value of the objective function; μ(x * ) means that the function is in x * The mean value at c ov (x * ) means that the function is in x * The covariance at the test point; c is the covariance of the test point itself; c *is the covariance vector between the test point and the training data point; C is the covariance matrix between the training data points; is the variance of Gaussian noise; I is the unit matrix; b is the target value vector of the training data point, that is, the true output vector.

[0100] Finally, based on the newly observed objective function value, the posterior distribution of the Gaussian process model is updated, and the mean and covariance are updated using Bayes’ theorem. The mean represents the final expected effect of this point, and the larger the mean, the better the effect. The variance represents the uncertainty of this point’s effect, and the larger the variance, the greater the uncertainty. This further refines the estimate of the objective function.

[0101] Step 3.2: Use the acquisition function to select the next sample point

[0102] First, an appropriate acquisition function should be selected in Bayesian optimization to correspond to the balance between exploration and exploitation strategies. The acquisition function used in this paper is the expected improvement (EI) function. This function can evaluate the expected improvement of each sampling point.

[0103] Then, for the expected increment (EI) function, the expected increment value of each candidate point is calculated. The expected increment represents the expected increase in the value of the objective function after sampling the point. The formula is as follows:

[0104]

[0105] x * = argmax x EI(x)(6)

[0106] After conversion, we can get:

[0107] EI(x)=σ(x)φ(z)+(μ(x)-f(x * ))Φ(z)(7)

[0108]

[0109] in, is the expected value, f(x * ) is the current maximum value, μ is the mean, σ is the standard deviation, Φ(z) and φ(z) represent the normal cumulative distribution function and probability density function of the standard normal random variable, respectively. The expected increment function shows that some points are greater than f(x * ), that is, the degree of improvement of selecting this point as the next sampling point compared to the current sampling point. This acquisition function integrates both the improvement probability and the different improvement amounts.

[0110] According to the value of the acquisition function, the candidate point that maximizes the acquisition function is selected as the next sample point. The new sample point is added to the training data and the Gaussian process model is updated.

[0111] Step 3.3: Repeat steps 3.1-3.2 until the loss function decreases by less than 0.001 in each of the five consecutive iterations, then stop training, thereby effectively finding the global optimal solution or the configuration parameters close to the optimal solution. The optimal hyperparameter combination finally obtained is: the search range for the number of neurons in the hidden layer is 10-100, and the search result is 56; the search range for the number of iterations is 50-100, and the search result is 71; the search range for the optimizer is stochastic gradient descent (SGD), Adam, AdaGrad, and the search result is Adam. This achieves the preliminary optimization of the model.

[0112] Step 4: Construct the self-organizing mechanism of the neural network

[0113] The sensitivity analysis method based on Fourier transform is used to evaluate the contribution of neurons. First, the SA algorithm is used to calculate the contribution of hidden layer neurons to the network output. The input factor of SA is the input value of the output layer, which is expressed as:

[0114]

[0115] in, is the output of the last hidden layer, is its connection weight. f represents the calculation rule of the output layer, and the sensitivity analysis model is expressed as:

[0116]

[0117] in, are m input factors, is the model output; the input factor The contribution to the output independent of other input parameters is denoted by The calculation formula is as follows:

[0118]

[0119] in, yes The variance of The input variable is hour expectations, is the variance of Y(t);

[0120] The sensitivity analysis model is transferred to the frequency domain for study and combined The independent effect on the output and the synergistic effect with other input parameters are recorded as the total sensitivity The calculation method is as follows:

[0121]

[0122] in, Indicated in The Fourier coefficients of the cosine signal at frequency, Indicated in The Fourier coefficients of a sinusoidal signal at a frequency are calculated as follows:

[0123]

[0124]

[0125] Where s represents the complex field, is the specified frequency of the jth hidden layer neuron, which is determined by the output of the jth hidden layer neuron and is calculated as follows:

[0126]

[0127] in, is the maximum output of the jth neuron that has been trained in the previous moment, is the minimum output value of the j-th neuron trained in the previous moment;

[0128] Then, the cumulative contribution of the jth neuron in the hidden layer for k consecutive moments is calculated as ST j (t):

[0129]

[0130] Finally, the obtained ST j (t) is used to guide the calculation of neuron splitting and deletion.

[0131] Step 5: Network structural parameter adjustment algorithm

[0132] The sensitivity of each neuron in the hidden layer is traversed, that is, the contribution to the output, and corresponding processing is performed according to the different situations of its contribution. At the same time, the implementation of the network algorithm is often related to the setting of the threshold. First, the mean and standard deviation of the contribution data set are calculated. Then a multiple is selected, where the highest threshold is equal to the mean plus the multiple multiplied by the standard deviation, and the lowest threshold is equal to the mean minus the multiple multiplied by the standard deviation. The multiple is usually set to 1.5-3, depending on the distribution of the data and the requirements of the application. In the present invention, the highest threshold is set to 0.95 and is represented by ε1, and the lowest threshold is set to 0.75 and is represented by ε2.

[0133] 5.1 Neuron Splitting Algorithm

[0134] When ST jWhen (t)≥ε1, it indicates that the contribution of the jth neuron in the hidden layer exceeds the set maximum threshold, and the neuron has a greater impact on the output of the neural network, and the system will perform specific operations. In this case, the neuron needs to be split to increase the impact of the neuron on the output and make it more significant. At the same time, in order to ensure the convergence speed of the neural network, the newly added neurons are required to be relatively uniform, so the initial setting of the weights of the new neurons after the split is as follows:

[0135] θ new1 =τθ j (t)(17)

[0136] θ new2 =(1-τ)θ j (t)(18)

[0137] Among them, θ new1 is the weight of the first newly added neuron, θ new2 is the weight of the second newly added neuron, θ j (t) is the weight of neuron j before the network structure adjustment at the tth step of training, and τ follows the standard normal distribution. After executing the neuron splitting algorithm, the number of neurons in the hidden layer increases by one.

[0138] 5.2 Neurons remain unchanged

[0139] When ε1 ≥ ST j (t)≥ε2, indicating that the contribution of the jth neuron in the hidden layer is less than the set maximum threshold and greater than the set minimum threshold. The neuron has a moderate impact on the output of the neural network. The system will keep this neuron unchanged and there is no need to perform neuron splitting or neuron deletion operations.

[0140] 5.3 Neuron Reduction Algorithm

[0141] When ST j (t)≤ε2, indicating that the contribution of the jth neuron in the hidden layer is less than the set minimum threshold, and the neuron has little effect on the output of the neural network, and the system will perform specific operations. In this case, it is necessary to prune the neuron to further reduce its influence on the output. Delete the neuron and delete all the values ​​connected to it. After executing the neuron pruning algorithm, the number of corresponding neurons in the hidden layer is reduced by one.

[0142] Step 6: Network parameter learning algorithm

[0143] The network selects the Adam optimizer to adjust the network parameters. The relevant algorithm is defined as follows:

[0144] First, initialize the exponentially weighted average m0 of the first-order gradient and the exponentially weighted average v0 of the second-order gradient of each parameter, and the deviation correction coefficient is For the mini-batch sample at the tth time step, calculate its gradient g t . Then the exponentially weighted average of the first-order and second-order gradients is updated as follows:

[0145] m t =β1·m t-1 +(1-β1)·g t (19)

[0146]

[0147] where β1 and β2 are average coefficients, set to 0.9 and 0.999 respectively. Then calculate and Bias-corrected gradient estimate:

[0148]

[0149]

[0150] in and are the t-th power of β1 and β2 respectively.

[0151] Finally update the weights:

[0152]

[0153] Where η is the learning rate and ∈ is a constant added to increase numerical stability and is set to 10 -8 .

[0154] Step 7: Network training samples and test samples

[0155] Input the training sample data and repeat steps 4-6 until all training samples are trained; input the test sample data and the output of the model is the predicted value of the atmospheric pollutant concentration.

Claims

1. A method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization, characterized in that: The following steps are involved: Step 1: Data preprocessing: In the process of model prediction, in order to ensure the uniformity of data indicators, the data needs to be standardized; Step 2: Design an initial atmospheric pollutant concentration prediction model; Step 3: Use the Bayesian optimization algorithm to find the optimal hyperparameter combination of the model; Step 4: Use the sensitivity analysis method based on Fourier transform to evaluate the contribution of each neuron in the hidden layer to the output; Step 5: Network structure parameter adjustment algorithm, splitting and pruning neurons according to the contribution of each neuron in the hidden layer to the output; Step 6: Network parameter learning algorithm. The network selects the Adam optimizer to adjust the network parameters. Step 7: Use the test sample data as the input of the trained neural network, and the output of the neural network is the predicted result of the atmospheric pollutant concentration.

2. The method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization according to claim 1 is characterized in that: Step 1 In the model prediction process, the normalization method is used to map the data values ​​to [0,1] and the calculation formula is: Where, X * is the normalized data; X is the sample data; X min , X max are the minimum and maximum values ​​of the samples respectively.

3. The method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization according to claim 1 is characterized in that: Step 2 uses the GRU neural network as the basic network, whose structure includes an input layer and two hidden layers, and a fully connected output layer; the input layer consists of 6 nodes, corresponding to the input vector x = (x1, x2, ..., x6) T , which represents the five meteorological factors of temperature, air pressure, humidity, wind speed and wind direction at time t and the concentration of pollutants to be predicted; the output layer has 1 node, which is used to predict the concentration of pollutants to be predicted one hour later.

4. The method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization according to claim 1 is characterized in that: In step 3, the Bayesian optimization algorithm is used to optimize and determine the number of hidden layer neurons (units), number of iterations (epochs), and optimizer hyperparameters of the prediction model. The search range of the number of hidden layer neurons is set to 10-100, the search range of the number of iterations is set to 50-100, and the optimizer search range is set to stochastic gradient descent (SGD), Adam or AdaGrad; the root mean square error is selected as the loss function, the GRU model with different parameter combinations is used as the independent variable, and the root mean square error is used as the output of the Bayesian framework. The hyperparameter combination is continuously trained and adjusted until the loss function of the model no longer decreases significantly, that is, when the loss function decreases by less than 0.001 in each of 5 consecutive iterations, the training is stopped.

5. The method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization according to claim 1 is characterized in that: Step 4 First, use the SA algorithm to calculate the contribution of hidden layer neurons to the network output. The input factor of SA is the input value of the output layer, and f represents the calculation rule of the output layer. The SA model is expressed as: in, are m input factors, is the model output; the input factor The contribution to the output independent of other input parameters is denoted by The calculation formula is as follows: in yes The variance of the input variable is hour expectations, yes Variance; transfer the sensitivity analysis model to the frequency domain for study and combine The independent effect on the output and the synergistic effect with other input parameters are recorded as the total sensitivity The calculation method is as follows: in, and Respectively expressed in The Fourier coefficients of the cosine signal and the sine signal at the frequency are calculated as follows: Where s represents the complex field, is the specified frequency of the jth hidden layer neuron, which is determined by the output of the jth hidden layer neuron and is calculated as follows: in, is the maximum output of the jth neuron that has been trained in the previous moment, is the minimum output value of the j-th neuron trained in the previous moment; Then, the cumulative contribution of the jth neuron in the hidden layer for k consecutive moments is calculated as ST j (t): Finally, the obtained ST j (t) is used to guide the calculation of neuron splitting and deletion.

6. The method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization according to claim 1 is characterized in that: Step 5 will traverse the sensitivity, i.e. contribution, of each neuron in the hidden layer and make corresponding adjustments according to the different situations of their contribution.

7. The method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization according to claim 1 is characterized in that: Step 6: The network selects the Adam optimizer to adjust the network parameters.

8. The method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization according to claim 1 is characterized in that: Step 7: Input the training sample data, and repeat steps 4-6 until all training samples are trained; input the test sample data, and the output of the model is the predicted value of the atmospheric pollutant concentration.

9. The method for predicting atmospheric pollutant concentrations based on a self-organizing recurrent neural network with Bayesian optimization according to claim 1 is characterized in that: Step 1: Data Preprocessing The normalization method is used to map the data values ​​to [0,1]. The calculation formula is: Where, X * is the normalized data; X is the sample data; X min , X max are the minimum and maximum values ​​of the samples respectively; Step 2: Design the initial network structure The GRU neural network is used as the basic network. Its structure includes an input layer, a double hidden layer, and a fully connected output layer. The input layer consists of 6 nodes, corresponding to the input vector x = (x1, x2, ..., x6) T , which represents the five meteorological factors of temperature, air pressure, humidity, wind speed, and wind direction at time t, as well as the concentration of pollutants to be predicted; the output layer has 1 node, which is used to predict the concentration of pollutants to be predicted one hour later; Step 3: Bayesian optimization network structure The Bayesian optimization algorithm is used to optimize and determine the number of hidden layer neurons, the number of iterations, and the optimizer hyperparameters of the prediction model. The search range of the number of hidden layer neurons is set to 10-100, the search range of the number of iterations is set to 50-100, and the optimizer search range is set to stochastic gradient descent, Adam or AdaGrad; the root mean square error is selected as the loss function. When the loss function decreases by less than 0.001 each time in 5 consecutive iterations, the training is stopped to determine the optimal hyperparameter combination; the details are as follows: Step 3.1: Probabilistic surrogate model modification First, we select the probability proxy model and use the Gaussian process as the probability proxy model. Initially, we use the Gaussian process model to establish a prior estimate of the objective function based on the existing training data. The Gaussian distribution is expressed as: f(x)~GP(μ(x),k(x,x′)) (2) Where f(x) is a Gaussian process; μ(x) is the mean function, which is set to 0; k(x, x′) is the covariance function of x; then, based on the existing Gaussian process model, the mean and variance of the predicted value at the current unknown point are calculated; The formula is as follows: Where: x * represents the input point corresponding to the predicted value of the objective function; μ(x * ) means that the function is in x * The mean value at c ov (x * ) means that the function is in x * The covariance at the test point; c is the covariance of the test point itself; c * is the covariance vector between the test point and the training data point; C is the covariance matrix between the training data points; is the variance of Gaussian noise; I is the unit matrix; b is the target value vector of the training data point, that is, the true output vector; Finally, based on the newly observed objective function value, the posterior distribution of the Gaussian process model is updated, and the mean and covariance are updated using Bayes’ theorem; Step 3.2: Use the acquisition function to select the next sample point First, the acquisition function should be selected in Bayesian optimization to be the expected incremental EI function; Then, for the expected increment EI function, the expected increment value of each candidate point is calculated; the expected increment represents the expected increase in the value of the objective function after sampling the point; its formula is as follows: x * =argmax x EI(x) (6) After conversion, we get: EI(x)=σ(x)φ(z)+(μ(x)-f(x * ))Φ(z) (7) in, is the expected value, f(x * ) is the current maximum value, μ is the mean, σ is the standard deviation, Φ(z) and φ(z) represent the normal cumulative distribution function and probability density function of the standard normal random variable, respectively; According to the value of the acquisition function, select the candidate point that maximizes the acquisition function as the next sample point; add the new sample point to the training data and update the Gaussian process model; Step 3.3: Repeat steps 3.1-3.2 until the loss function decreases by less than 0.001 each time in 5 consecutive iterations, then stop training; Step 4: Construct the self-organizing mechanism of the neural network The sensitivity analysis method based on Fourier transform is used to evaluate the contribution of neurons. First, the SA algorithm is used to calculate the contribution of hidden layer neurons to the network output. The input factor of SA is the input value of the output layer, which is expressed as: in, is the output of the last hidden layer, is its connection weight; f represents the output layer calculation rule, then the sensitivity analysis model is expressed as: in, are m input factors, is the model output; the input factor The contribution to the output independent of other input parameters is denoted by The calculation formula is as follows: in, yes The variance of The input variable is hour expectations, yes The variance of The sensitivity analysis model is transferred to the frequency domain for study and combined The independent effect on the output and the synergistic effect with other input parameters are recorded as the total sensitivity The calculation method is as follows: in, Indicated in The Fourier coefficients of the cosine signal at frequency, Indicated in The Fourier coefficients of a sinusoidal signal at a frequency are calculated as follows: Where s represents the complex field, is the specified frequency of the jth hidden layer neuron, which is determined by the output of the jth hidden layer neuron and is calculated as follows: in, is the maximum output of the jth neuron that has been trained in the previous moment, is the minimum output value of the j-th neuron trained in the previous moment; Then, the cumulative contribution of the jth neuron in the hidden layer for k consecutive moments is calculated as ST j (t): Finally, the obtained ST j (t) used to guide the calculation of neuron splitting and deletion; Step 5: Network structural parameter adjustment algorithm The sensitivity of each neuron in the hidden layer is traversed, that is, its contribution to the output, and corresponding processing is performed according to different situations of its contribution; the highest threshold is set to 0.95 and represented by ε1, and the lowest threshold is set to 0.75 and represented by ε2; 5.1 Neuron Splitting Algorithm When ST j When (t)≥ε1, it indicates that the contribution of the jth neuron in the hidden layer exceeds the set maximum threshold. This neuron has a greater impact on the output of the neural network, and the system will perform specific operations. In this case, it is necessary to split the neuron to increase its influence on the output and make it more significant. At the same time, in order to ensure the convergence speed of the neural network, the newly added neurons are required to be relatively uniform. Therefore, the initial setting of the weights of the new neurons after the split is as follows: i new1 =tθ j (t) (17) i new2 =(1-τ)θ j (t) (18) Among them, θ new1 is the weight of the first newly added neuron, θ new2 is the weight of the second newly added neuron, θ j (t) is the weight of neuron j before the network structure adjustment at the tth step of training, and τ follows the standard normal distribution; after executing the neuron splitting algorithm, the number of neurons corresponding to the hidden layer increases by one; 5.2 Neurons remain unchanged When ε1 ≥ ST j (t) ≥ ε2, indicating that the contribution of the jth neuron in the hidden layer is less than the set maximum threshold and greater than the set minimum threshold. The neuron has a moderate impact on the output of the neural network. The system will keep this neuron unchanged and there is no need to perform neuron splitting or neuron deletion operations. 5.3 Neuron Reduction Algorithm When ST j (t)≤ε2, indicating that the contribution of the jth neuron in the hidden layer is less than the set minimum threshold, and the neuron has little effect on the output of the neural network, and the system will perform specific operations; in this case, it is necessary to delete the neuron to further reduce its influence on the output; delete the neuron and all the values ​​connected to it; after executing the neuron deletion algorithm, the number of corresponding neurons in the hidden layer is reduced by one; Step 6: Network parameter learning algorithm The network selects the Adam optimizer to adjust the network parameters. The relevant algorithm is defined as follows: First, initialize the exponentially weighted average m0 of the first-order gradient and the exponentially weighted average v0 of the second-order gradient of each parameter, and the deviation correction coefficient is For the mini-batch sample at the tth time step, calculate its gradient g t ; Then update the exponentially weighted average of the first-order and second-order gradients, the formula is as follows: m t =β1·m t-1 +(1-β1)·g t (19) Where β1 and β2 are average coefficients, set to 0.9 and 0.999 respectively; then calculate and Bias-corrected gradient estimate: in and are the t-th power of β1 and β2 respectively; Finally update the weights: Where η is the learning rate and ∈ is a constant added to increase numerical stability and is set to 10 -8 ; Step 7: Network training samples and test samples Input the training sample data and repeat steps 4-6 until all training samples are trained; input the test sample data and the output of the model is the predicted value of the atmospheric pollutant concentration.

Images