A novel attention-based modular neural network for NO2 incineration of municipal solid waste x Prediction Methods

Abstract

The present invention relates to a method for predicting NOx in a municipal solid waste incineration process based on an attention modular neural network, which realizes accurate prediction of NOx concentration, and includes the following steps: first, collecting data and preprocessing the data; second, determining the input variables and output variables of the model; then, using the attention modular neural network to establish a prediction model; finally, using the test data as the input of the model to verify the effectiveness of the model. The present invention effectively realizes accurate prediction of NOx concentration, and has important theoretical significance and application value.

Description

Technical Field

[0001] The present invention relates to a method for predicting NOx in the process of municipal solid waste incineration; a NOx prediction method based on an attention modular neural network AMNN is established. X The prediction model realizes the X Accurate prediction of concentration. It belongs to both the field of urban solid waste management and the field of intelligent modeling. Background Art

[0002] With the growth of the world's population, economic development and the acceleration of urbanization, municipal solid waste (MSW) has become a focus of attention. The large accumulation of MSW has brought tremendous pressure to the urban soil, air and water environment, and has caused serious health effects on people. The treatment of municipal solid waste is imminent.

[0003] Sanitary landfill, composting and incineration are the three main ways of treating urban solid waste. Among them, incineration technology oxidizes and decomposes organic matter into simple inorganic matter through a series of complex physical and chemical reactions. It has the characteristics of harmlessness, reduction and resource utilization, and has become one of the most effective ways to control solid waste pollution worldwide. Although incineration technology can achieve harmlessness, reduction and resource utilization of solid waste, it will also produce secondary pollutants. NOx is one of the main pollutants produced in the process of incineration of urban solid waste, which seriously affects people's health and ecological environment quality. However, the existing control technology can only detect the NOx concentration at the current moment, and cannot provide the staff with a reference value of the NOx emission concentration at the future moment, which is easy to cause the NOx emission control measures to lag behind, and the NOx emission exceeds the standard. Therefore, the accurate prediction of NOx emission concentration is of great significance to improve the efficiency of the denitrification system and ensure the safe and stable operation of urban solid waste incineration plants. Summary of the invention

[0004] The purpose of this invention is to propose a method for NO in the process of municipal solid waste incineration based on attention modular neural network. X Prediction method, using attention modular neural network to establish NO X Prediction model to achieve NO X Accurate prediction of concentration.

[0005] The present invention adopts the following technical solutions and implementation steps:

[0006] 1. Data preprocessing and determination of model input and output variables;

[0007] The data is collected by a continuous emission monitoring system with a sampling interval of 30 seconds; data preprocessing includes outlier processing and data standardization;

[0008] The 3σ criterion is used for outlier processing:

[0009] |y'-μ|≥3σ (1)

[0010] Among them, μ and σ are the mean and variance corresponding to a set of data, respectively. The outlier y' that satisfies formula (1) is removed from the data;

[0011] In order to eliminate the dimensional differences between variables and improve the accuracy of the model, the Z-score method is used to standardize the original variables:

[0012]

[0013] Among them, x m ' represents the data after the standardization of the mth variable, x m represents the data of the mth variable before standardization, μ m and σ m are the mean and variance of the mth variable respectively;

[0014] Through mechanism analysis and maximum information coefficient algorithm, six input variables related to NOx prediction are determined. The input variables of the model training set include: NOx concentration at time t, upper temperature A of the incinerator furnace, flue gas temperature A at the outlet of the incinerator furnace, flue gas temperature B at the outlet of the incinerator furnace, primary fan inlet flow rate, and flue gas oxygen content; the output variable of the model training set is the NOx concentration at time t+1;

[0015] 2. Design a modular partitioning method based on attention mechanism;

[0016] The modularity index MQ consists of the density within the module and the sparsity between modules, where the density within the module is calculated as follows:

[0017]

[0018] Among them, J C is the density within the module, P is the number of modules in the current network, and N l is the number of samples assigned to the lth module, x i is the input vector of the i-th training sample, h l and r l They represent the position and scope of action of the core node of the lth module respectively;

[0019] The sparsity between modules is calculated based on the Euclidean distance:

[0020]

[0021] Among them, J S is the sparsity between modules, d(h l ,hs ) represents the distance between the core node of the lth module and the core node of the sth module, taking into account the density within the module J C and the sparsity between modules J S , the modularity index measurement method is proposed as follows:

[0022]

[0023] The larger the value of MQ is, the higher the modularity of the network is. The modular partitioning method based on the attention mechanism is as follows: First, the core nodes of each initial submodule are determined based on the idea of ​​point density; second, samples are assigned to each submodule based on the attention mechanism; then, new core nodes of existing modules are determined by seeking the maximum "modularity" of the network; finally, the optimal number of modules in the network is determined by merging the closest core nodes and seeking the maximum "modularity" of the network;

[0024] ①Determine the center of each initial submodule

[0025] The data set cannot clearly describe the distribution of sample points. The idea of ​​point density is introduced, that is, the density of sample point distribution. If there are more points around a sample point, the greater the sample distribution density at the sample point, the more likely the sample is to become the core node of modular partitioning. Therefore, a point density function is proposed as a method for selecting the core nodes of the initial submodule.

[0026] For each training sample input vector x, the expression of its point density function is defined as follows:

[0027]

[0028] d ij =||x i -x j || (7)

[0029] Among them, d ij Represents two training sample input vectors x i and x j If x i The more points around, the larger the Z i The larger the value of ;

[0030] Calculate the point density of each sample, and use the input vector of the n training samples with the largest point density as the core node of the initial submodule. The specific settings are as follows:

[0031]

[0032]

[0033]

[0034] Among them, h i and r i Respectively represent the core node and scope of the i-th initial submodule; x Zi is the input vector of the training sample with the largest point density; d max Input vector x for training sample i and x j The maximum distance between

[0035] ② Allocate samples based on attention mechanism

[0036] At time T, when the Tth training sample x T When entering the network, calculate the x of each submodule T Attention value:

[0037]

[0038] Among them, Attention d For the d-th submodule pair x T The attention value, Q d is the core node of the d-th submodule, Q di is the i-th component of the core node of the d-th submodule, K and V are the input samples x T , as shown in formula (12), K i 、V i For sample x T The i-th component of i is the correlation value, as obtained in formula (13);

[0039] K=V=x T =[x1,…,x i ,…,x k ] (12)

[0040] Submodule d for sample x T The specific calculation steps of the attention value are as follows:

[0041] 1) According to Q d And K calculates the correlation between the two, and the calculation formula is:

[0042] S i =|Q di -K i | (13)

[0043] 2) For S i The value is normalized by softmax to get a i value;

[0044] 3) According to a obtained in step 2)i Value calculation 1-a i value;

[0045] 4) According to the obtained 1-a i The value V is weighted and summed to get the attention value Attention d , complete the submodule d to x T Calculation of attention value;

[0046] If the dth submodule is x T The attention value of is greater than the threshold λ, that is, Attention ≥ λ, then the sample x T Assigned to this module, λ is 0.5;

[0047] ③Optimization module

[0048] In order to make the network have the optimal degree of "modularity", according to formula (5), the modularity index value MQ of the entire network is calculated when the newly added samples and the core nodes of the original modules are respectively used as the core nodes of the sub-modules: t and MQ lmin :

[0049] If MQ lmin <MQ t , the network modularization degree of selecting the current newly added sample as the core node is higher, and the sample is used to replace the original module core node to become the new module core node. The initial parameter settings are as follows:

[0050] h t =x t (14)

[0051]

[0052] Among them, h t is the new core node of the module, x t is the input vector of the newly added sample, r t is the scope of the module, N lmin is the number of samples assigned to this module;

[0053] If MQ lmin >MQ t , the core node of the current module remains unchanged, and only the scope of the module needs to be adjusted:

[0054]

[0055] Among them, r lmin is the scope of the module, N lmin is the number of samples assigned to this module;

[0056] When all training samples are compared, the samples are assigned to different submodules;

[0057] ④Merge module

[0058] If there are two or more modules at time T, calculate the Euclidean distance between the core nodes of each module. i To core node c j If the distance between them is the smallest, the two modules are combined into a new module, and the initial parameters are set as follows:

[0059]

[0060] Among them, c new is the core node of the merged module, Z ci and Z cj They are core nodes c i and c j The point density of

[0061] According to formula (5), the modularity index value MQ of the entire network before and after merging the modules is calculated respectively, and the number of modules when the MQ value is the largest is found. At this time, the modularity of the network is the largest, and modular partitioning is formed;

[0062] 3. Construction of sub-network based on self-organizing fuzzy neural network;

[0063] A self-organizing fuzzy neural network is used to construct the sub-network corresponding to each module. The design of the sub-network includes three parts: network structure growth, network structure pruning and network parameter adjustment.

[0064] ① Network structure growth

[0065] When the first training sample enters the network, the sample is used as the center, width, and connection weight of the first RBF neuron to the output layer:

[0066] c1=x1 (18)

[0067] σ1=1 (19)

[0068] w1=y d1 (20)

[0069] Where c1 is the center of the first neuron, x1 is the input vector of the first training sample, σ1 is the width of the first neuron, w1 is the connection weight from the regularization layer neuron to the output layer neuron, and y d1 is the output value of the first training sample;

[0070] At time t s At this moment, the training error vector e(t s) is obtained by the following formula:

[0071]

[0072]

[0073] Among them, y f is the expected output value of the fth sample, is the fth sample at time t s The network output value; find the sample n with the largest difference between the expected output value and the network output value ts :

[0074]

[0075] Add a new rule for samples to learn and compensate for the current maximum error; the center vector and output weight settings of the newly added rules are as follows:

[0076] c t =x l (twenty four)

[0077] w t =y dl (25)

[0078] Among them, c t is the center vector of the newly added neuron, x l is the input vector of the lth training sample, w t is the connection weight from the regularization layer neurons to the output layer neurons, y dl is the output value of the lth training sample;

[0079] After each new rule is added, the Euclidean distance between the rule and the existing rules is calculated and the minimum distance d is found. min :

[0080] d min =min{dist(c t ,c j≠t )} (26)

[0081] Based on this minimum distance, the width σ of the new neuron is added t Set to:

[0082] σ t =αd min ,α∈[0.5,0.9] (27)

[0083] Among them, α is adjusted according to different prediction tasks;

[0084] Each time a new rule is added, the network parameters are adjusted through the second-order learning algorithm; when the preset maximum structure J is reached max Or the expected training accuracy E0, the network structure growth process ends; during the experiment, J max =10, E0=0.0001, and the root mean square error SSE is used to measure the training accuracy of the network, which is calculated as follows:

[0085]

[0086] Among them, y i and are the expected output value and network output value of the i-th sample, respectively, N s is the number of samples in submodule s;

[0087] ②Network structure pruning

[0088] In order to avoid redundancy in the network structure, it is proposed to measure the contribution value of neurons based on the activation intensity of neurons in the RBF layer:

[0089]

[0090] Where Φ(j) is the output of the jth RBF neuron, J min s The neuron with the smallest activation intensity in the RBF layer;

[0091] Therefore, the rule with the smallest activation intensity is deleted, and the parameters are adjusted through the second-order learning algorithm. The root mean square error value SSE_1 after deleting the rule is compared with the root mean square error value SSE_0 when the rule is not deleted; if SSE_1≤SSE_0, the selected nodes can be pruned without sacrificing the network learning ability, and then the above process is repeated. On the contrary, the selected nodes cannot be deleted; at this time, the network structure pruning process is completed and the sub-network is constructed; each time a neuron is deleted, the parameters are adjusted using the second-order learning algorithm;

[0092] ③Network parameter adjustment

[0093] The improved second-order learning algorithm is as follows:

[0094] △ t+1 =△ t -(Q t +μ t I) -1 g t (30)

[0095] Among them, Δ is the parameter that needs to be adjusted, including center, width and connection weight; Q is the Hessian matrix; g t is the gradient vector; I is the unit matrix; μ is the learning rate parameter, which is 0.01.

[0096] In order to reduce the space complexity and time complexity of the Jacobian matrix calculation process, the Hessian matrix Q and the gradient vector g are t The calculation is transformed into the summation of sub-class Hessian matrices and the summation of sub-gradient vectors:

[0097]

[0098]

[0099] q z is a Hessian-like submatrix, η z is the gradient vector, which can be calculated by the following formula:

[0100]

[0101]

[0102] Among them, e z is the expected output y of the zth sample z With the network prediction output The difference, j z is the Jacobian vector, calculated as follows:

[0103]

[0104]

[0105] According to the chain rule, each component of the Jacobian vector in equation (38) is calculated as follows:

[0106]

[0107]

[0108]

[0109] where c j , σ j , w j are the center, width and connection weight of the jth neuron to the output layer, respectively, x z is the input vector of the zth training sample;

[0110] 4. NO X Concentration prediction;

[0111] The test sample data is used as the input of the attention modular neural network, and the output of the model is the NOx concentration prediction result at time t+1; at time T, when the Tth test sample enters the AMNN, the calculation of each submodule for the sample xT Attention value Attention, when Attention ≥ λ, activate the corresponding submodule to calculate the input sample;

[0112] After each submodule calculates the sample, the output integration layer performs weighted integration on the output of each activated submodule:

[0113]

[0114] Among them, y r is the output of the rth sub-network, ε r is the weight coefficient from the rth sub-network to the output layer, which is determined according to the degree of attention of the sub-network to the input sample. The calculation formula is as follows:

[0115]

[0116] Among them, Attention r For the rth activated sub-network, input sample x T The attention value of the network; R is the total number of activated sub-networks.

[0117] The root mean square error (RMSE) and mean percentage error (MAPE) are used to quantitatively evaluate the test accuracy. The calculation formulas for RMSE and MAPE are as follows:

[0118]

[0119]

[0120] In the formula, y i and are the expected output and network output of the i-th sample, respectively, N T is the number of test samples.

[0121] The present invention has the following obvious advantages and beneficial effects:

[0122] Based on the good nonlinear mapping ability and generalization ability of the attention modular neural network, this paper establishes a stable and effective NO X The concentration prediction model can realize NO X Accurate prediction of NO concentration in municipal solid waste incineration process X Emission control is of great importance. BRIEF DESCRIPTION OF THE DRAWINGS

[0123] Figure 1 is a flow chart of the present invention;

[0124] Figure 2 It is the basic structure diagram of modular neural network;

[0125] Figure 3 It is the structure diagram of self-organizing fuzzy neural network;

[0126] Figure 4 This is a framework diagram of the NOx prediction model based on attention modular neural network;

[0127] Figure 5 Calculate the flowchart for the attention mechanism;

[0128] Figure 6 Integrate a flowchart for attention-based sub-networks;

[0129] Figure 7 This is the training result diagram of sub-network 1;

[0130] Figure 8 This is the training result diagram of sub-network 2;

[0131] Fig. 9 This is the training result diagram of sub-network three;

[0132] Fig.10 This is the model test result diagram;

[0133] Fig.11 This is the error graph for the model test. DETAILED DESCRIPTION

[0134] The present invention uses training data to establish a X Attention modular neural network model for concentration prediction; Use test data to verify the output of the attention modular neural network model X Accuracy of concentration.

[0135] As an example, actual data from a municipal solid waste incineration plant in Beijing was used to verify the effectiveness of the method proposed in the present invention. After eliminating obviously abnormal data, a total of 2100 sets of experimental data were obtained. Based on the obtained data samples, the mechanism analysis and maximum information coefficient algorithm were used to select input variables. The specific input variables determined are shown in Table 1.

[0136] Table 1

[0137]

[0138] Of the 2100 data sets, 1400 sets were used to build the prediction model, and the remaining 700 sets were used to test the model performance;

[0139] (1) Based on 1400 sets of training data, the attention modular partitioning method is used. The 1400 sets of training data are divided into three subsets, and the number of samples in each subset is 811, 654, and 343 respectively; accordingly, the AMNN is composed of three sub-networks, which are established by the sub-network construction method in step 3, where the α values ​​of the three sub-networks are set to 0.83, 0.73, and 0.72 respectively; Figure 7 , Figure 8 , Fig. 9 They are the training result diagrams of each sub-network, X-axis: number of training samples, unit is / sample, Y-axis: NO X Concentration, in mg / m 3 ;

[0140] (2) Based on 700 test data sets, AMNN was used to perform NO X Concentration prediction, test results such as Fig.10 As shown, X-axis: number of test samples, unit: pcs / sample, Y-axis: NO X Concentration, in mg / m 3 ; Test error such as Fig.11 As shown, the X-axis: number of training samples, the unit is / sample, the Y-axis: NO X Concentration test error, unit is mg / m 3 ;

[0141] (3) The root mean square error (RMSE) and the average percentage error (MAPE) were used to quantitatively evaluate the test accuracy. The calculation results were: RMSE = 9.6438, MAPE = 5.3457%.

[0142] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. It should be noted that the above description is only a specific embodiment of the present invention and does not limit the present invention. All modulations and optimizations made within the spirit and principle of the present invention should fall within the scope of the claims of the present invention.

Claims

1. A method for predicting NOx in municipal solid waste incineration process based on attention modular neural network, characterized in that: The following steps are involved: Step 1: Data preprocessing and determining model input and output variables; The data is collected by the continuous emission monitoring system with a sampling interval of 30 seconds. Data preprocessing includes outlier processing and data standardization. Through mechanism analysis and maximum information coefficient algorithm, six model input variables related to NOx prediction are determined. The input variables of the model training set include: NOx concentration at time t, incinerator furnace upper temperature A, incinerator furnace outlet flue gas temperature A, incinerator furnace outlet flue gas temperature B, primary fan inlet flow, flue gas oxygen content; the output variable of the model training set is NOx concentration at time t+1. Step 2: Design a modular partitioning method based on the attention mechanism, as follows: The modularity index MQ consists of the density within the module and the sparsity between modules, where the density within the module is calculated as follows: Among them, J C is the density within the module, P is the number of modules in the current network, and N l is the number of samples assigned to the lth module, x i is the input vector of the i-th training sample, h l and r l They represent the core node and scope of action of the lth module respectively; The sparsity between modules is calculated based on the Euclidean distance: Among them, J S is the sparsity between modules, d(h l ,h s ) represents the distance between the core node of the lth module and the core node of the sth module. According to the density within the module, J C and the sparsity between modules J S , the modularity index measurement method is proposed as follows: The larger the value of MQ is, the higher the modularity of the network is. The modular partitioning method based on the attention mechanism is as follows: first, the core nodes of each initial submodule are determined based on the idea of ​​point density; second, samples are assigned to each submodule based on the attention mechanism; then, new core nodes of existing modules are determined by seeking the maximum "modularity" of the network; finally, the optimal number of modules in the network is determined by merging the closest core nodes and seeking the maximum "modularity" of the network; a) Determine the center of each initial submodule The data set cannot clearly describe the distribution of sample points. The idea of ​​point density is introduced, that is, the density of sample point distribution. The more points there are around a sample point, the greater the sample distribution density at the sample point, and the more likely the sample is to become the core node of modular partitioning. Therefore, a point density function is proposed as a method for selecting the core nodes of the initial submodule. For each training sample input vector x, the expression of its point density function is defined as follows: d ij =||x i -x j || (7) Among them, d ij Represents two training sample input vectors x i and x j If x i The more points around, the larger the Z i The larger the value of ; Calculate the point density of each sample, and use the input vectors of the first n training samples with the largest point density as the core nodes of the initial submodule. The specific settings are as follows: h i =x Zi (8) Among them, h i and r i Respectively represent the core node and scope of the i-th initial submodule; x Zi is the input vector of the training sample with the largest point density; ξ is the adjustment coefficient, and its value range is Between; d max Input vector x for training sample i and x j The maximum distance between b) Sample allocation based on attention mechanism At time T, when the Tth training sample x T When entering the network, calculate the x of each submodule T Attention value: Among them, Attention d For the d-th submodule pair x T The attention value, Q d is the core node of the d-th submodule, Q di is the i-th component of the core node of the d-th submodule, K and V are the input samples x T , as shown in formula (12), K i 、V i For sample x T The i-th component of i is the correlation value, as obtained in formula (13); K=V=x T =[x1,L,x i ,L,x k ] (12) Submodule d for sample x T The specific calculation steps of the attention value are as follows: 1) According to Q d And K calculates the correlation between the two, and the calculation formula is: S i =|Q di -K i | (13) 2) For S i The value is normalized by softmax to get a i value; 3) According to a obtained in step 2) i Value calculation 1-a i value; 4) According to the obtained 1-a i The value V is weighted and summed to get the attention value Attention d , complete the submodule d to x T Calculation of attention value; If the dth submodule is x T The attention value of x is greater than the threshold λ, that is, Attention ≥ λ, then T Assigned to this module, λ is 0.5; c) Optimization module In order to make the network have the optimal degree of "modularity", according to formula (5), the modularity index value MQ of the entire network is calculated when the newly added samples and the core nodes of the original modules are respectively used as the core nodes of the sub-modules: t and MQ lmin : If MQ lmin <MQ t , the network modularization degree of selecting the current newly added sample as the core node is higher, and the sample is used to replace the original module core node to become the new module core node. The initial parameter settings are as follows: h t =x t (14) Among them, h t is the new core node of the module, x t is the input vector of the newly added sample, r t is the scope of the module, N lmin is the number of samples assigned to this module; If MQ lmin >MQ t , the core node of the current module remains unchanged, and only the scope of the module needs to be adjusted: Among them, r lmin is the scope of the module, N lmin is the number of samples assigned to this module; When all training samples are compared, the samples are assigned to different submodules; d) Merge modules If there are two or more modules at time T, calculate the Euclidean distance between the core nodes of each module. i to c j If the distance between them is the smallest, the two modules are combined into a new module, and the initial parameters are set as follows: Among them, c new is the core node of the merged module, Z ci and Z cj They are core nodes c i and c j The point density of According to formula (5), the modularity index value MQ of the entire network before and after merging the modules is calculated respectively, and the number of modules when the MQ value is the largest is found. At this time, the modularity of the network is the largest, and modular partitioning is formed; Step 3: Use self-organizing fuzzy neural network to construct the sub-network corresponding to each module, as follows; ① Network structure growth When the first training sample enters the network, the sample is used as the center, width, and connection weight of the first RBF neuron to the output layer: c1=x1 (18) σ1=1 (19) w1=y d1 (20) Where c1 is the center of the first neuron, x1 is the input vector of the first training sample, σ1 is the width of the first neuron, w1 is the connection weight from the regularization layer neuron to the output layer neuron, and y d1 is the output value of the first training sample; At time t s At this moment, the training error vector e(t s ) is obtained by the following formula: Among them, y f is the expected output value of the fth training sample, is the fth sample at time t s The network output value of Find the sample with the largest difference between the expected output value and the network output value Add a new rule for samples to learn and compensate for the current maximum error; the center vector and output weight settings of the newly added rules are as follows: c t =x l (24) w t =y dl (25) Among them, c t is the center of the newly added neuron, x l is the input vector of the lth training sample, w t is the connection weight from the regularization layer neurons to the output layer neurons, y dl is the output value of the lth training sample; After each new rule is added, the Euclidean distance between the rule and the existing rules is calculated and the minimum distance d is found. min : d min =min{dist(c t ,c j≠t )} (26) Based on this minimum distance, the width σ of the new neuron is added t Set to: s t =αd min ,α∈[0.5,0.9] (27) Among them, α is adjusted according to different prediction tasks; Each time a new rule is added, the network parameters are adjusted through the second-order learning algorithm; when the preset maximum structure J is reached max Or the expected training accuracy E0, the network structure growth process ends; during the experiment, J max =10, E0=0.0001, and the square error SSE is used to measure the training accuracy of the network, which is calculated as follows: Among them, y i and are the expected output value and network output value of the i-th training sample, respectively, N s is the number of samples in submodule s; ②Network structure pruning In order to avoid redundancy in the network structure, a measure based on the activation intensity of neurons in the RBF layer is proposed: Where Φ(j) is the output of the jth RBF neuron, J min s The neuron with the smallest activation intensity in the RBF layer; Therefore, the rule with the smallest activation intensity is deleted, and the parameters are adjusted through the second-order learning algorithm. The root mean square error value SSE_1 after deleting the rule is compared with the root mean square error value SSE_0 when the rule is not deleted; if SSE_1≤SSE_0, the selected rule can be pruned without sacrificing the network learning ability, and then the above process is repeated. On the contrary, the selected rule cannot be deleted; at this time, the network structure pruning process is completed and the sub-network is constructed; each time a neuron is deleted, the parameters are adjusted using the second-order learning algorithm; ③Network parameter adjustment The improved second-order learning algorithm is as follows: △ t+1 =△ t -(Q t +m t I) -1 g t (30) Among them, Δ is the parameter that needs to be adjusted, including center, width and connection weight; Q is the Hessian matrix; g t is the gradient vector; I is the unit matrix; μ is the learning rate parameter, which is 0.01; In order to reduce the space complexity and time complexity of the Jacobian matrix calculation process, the Hessian matrix Q and the gradient vector g are t The calculation is transformed into the summation of sub-class Hessian matrices and the summation of sub-gradient vectors: q z is the subclass Hessian matrix, η z is the sub-gradient vector, which can be calculated by the following formula: Among them, e z is the expected output value y of the zth sample z And the network predicted output value The difference, j z is the Jacobian vector, calculated as follows: According to the chain rule, each component of the Jacobian vector in equation (38) is calculated as follows: where c j , σ j , w j are the center, width and connection weight of the jth neuron to the output layer, respectively, x z is the input vector of the zth training sample; Step 4: Use the test data as the input of the model, and the output of the model is NO at time t+1. X Concentration prediction: At time T, when the Tth test sample enters the model, the calculation of each submodule’s response to x T Attention value Attention, when Attention ≥ λ, activate the corresponding submodule to calculate the input sample; After each submodule calculates the sample, the output integration layer performs weighted integration on the output of each activated submodule: Among them, y r is the output value of the rth sub-network, ε r is the weight coefficient from the rth sub-network to the output layer, which is determined according to the degree of attention of the sub-network to the input sample. The calculation formula is as follows: Among them, Attention r For the rth activated sub-network, input sample x T The attention value of the network; R is the total number of activated sub-networks.

Images