Industrial soft measurement method of sparse ONLSTM based on non-negative notching

Abstract

The invention discloses an industrial soft measurement method of sparse ONLSTM based on non-negative notching, and belongs to the field of soft measurement modeling and application in the modern industrial process. According to the method, the problems of modeling difficulty increase and model performance reduction caused by nonlinearity, dynamics and variable redundancy of a complex industrial process are considered, an NNG algorithm is embedded into an input layer weight matrix and a hidden layer weight matrix of an ONLSTM network, and a DNNG-ONLSTM neural network prediction model is constructed to realize prediction of key quality variables of the complex industrial process; on the premise of ensuring the prediction capability of the model, redundant input data is eliminated. And meanwhile, an NNG algorithm is added to a hidden layer, so that the calculation amount of the model is reduced, redundant nodes of the network model are eliminated, the sparsification of the model is realized, the complexity and training difficulty of the model are reduced, and the generalization performance of the model is improved.

Description

Technical Field

[0001] The present invention relates to an industrial soft sensing method based on a sparse ONLSTM with non-negative strangulation, and belongs to the field of soft sensing modeling and application of modern industrial processes. Background Art

[0002] With the continuous advancement of industrial intelligence, the complexity and informationization of industrial production are increasing. To improve product quality, it is necessary to monitor certain quality variables that are difficult to collect directly or in real time but are closely related to product quality. However, the acquisition of quality variables is difficult due to limitations in measurement technology and equipment, the complexity of production processes and techniques, physical and chemical properties, and data processing and transmission issues. Soft sensing technology effectively predicts relevant quality variables by constructing mathematical models between easily accessible auxiliary variables and quality variables. It has been widely used in industrial process monitoring and control.

[0003] To improve the accuracy of industrial process monitoring and control, more efficient and robust soft sensor modeling algorithms are key. Artificial neural networks, with their powerful nonlinear mapping capabilities and ability to learn without relying on prior knowledge, have become a research hotspot in the field of soft sensing. In recent years, gated recurrent neural networks, represented by long short-term memory (LSTM) networks, have better captured long-term dependencies in data by controlling the flow of information through their unique gating structure.

[0004] In "Modeling Method and Application of Rough Rolling Exit Temperature Based on Feature Fusion", Zhang Kai et al. constructed a feature extraction model that combined a long short-term memory network (LSTM) and a temporal convolutional network (TCN), used LSTM to achieve feature fusion, and ultimately improved prediction accuracy. In "LogicLSTM: Logically-driven long short-term memory model for fault diagnosis in gearboxes", Hogea E. et al. obtained LogicLSTM by logically guiding the pre-trained LSTM network to perform temporal data processing with logical reasoning capabilities to improve classification prediction accuracy and interpretability. However, in the above two methods, the hidden layer neurons of the LSTM network are independent of each other, and the sequential hierarchical information between neurons is not considered, which may result in different neurons learning the same feature information.

[0005] In addition, there is significant strong correlation and coupling between actual industrial process data, which leads to a large amount of redundant information in the data. Traditional network models are unable to identify and eliminate the interference of redundant variables, resulting in increased prediction difficulty and reduced prediction performance of the model. Summary of the Invention

[0006] To address the problem that existing soft measurement technologies fail to fully consider the nonlinearity, dynamics and variable redundancy of complex industrial processes, resulting in increased modeling difficulty and reduced model performance, the present invention embeds the DNNG (Double Non-negative Garrote) algorithm into the input layer weight matrix and hidden layer weight matrix of the ONLSTM (Ordered Neurons Long Short-Term Memory) network, and proposes an industrial soft measurement method based on sparse ONLSTM with non-negative strangulation. This method constructs a DNNG-ONLSTM (Double Non-negative Garrote-Ordered Neurons Long Short-Term Memory) neural network prediction model, which can accurately predict key quality variables through process variables.

[0007] The technical solution of the present invention is:

[0008] Step 1: Collect input and output data of the industrial process as a sample data set;

[0009] The input data are process variables in industrial processes;

[0010] The output data is the quality variable corresponding to the output of the process variable in the industrial process;

[0011] The input and output data are collected by continuous and uniform sampling every 5 minutes to obtain a sample data set.

[0012] Step 2: Preprocess and divide the sample data set obtained in step 1;

[0013] Preprocessing includes missing value and outlier processing and data standardization;

[0014] Specifically, for variables that only contain some time points, if there are too many incomplete data and it is impossible to supplement them, such variables will be deleted, and variables whose data in the sample are all constant values ​​will be deleted;

[0015] For variables with some null values, the null values ​​are replaced by the average of the two preceding and following data.

[0016] Secondly, according to the process requirements, the operating range of the original data variables is summarized, and then the maximum and minimum clipping methods are used to eliminate some samples that are not within this range, and outliers are removed according to the 3σ criterion; the specific process is as follows:

[0017] First, the measured variables are measured with equal precision, and x1, x2, ..., x are obtained independently. N , find its arithmetic mean and residual error Where i = 1, 2, ..., N; and calculate the standard deviation σ, if a measurement value x i The residual error v i (1≤i≤N), if the following formula (31) is satisfied:

[0018]

[0019] The error is considered to be a gross error, and the data x containing the error i should be eliminated;

[0020] Finally, the input data and output data are standardized using the z-score method.

[0021] The sample data set is divided into: the first 80% of the preprocessed sample data set is used as the training data set, and the remaining 20% ​​is used as the test data set.

[0022] Step 3: Build ONLSTM neural network;

[0023] Construct an ONLSTM neural network with a forget gate, input gate, output gate, and candidate unit state.

[0024] Step 4: Train the ONLSTM neural network obtained in step 3;

[0025] The ONLSTM neural network is trained using an optimization algorithm based on the gradient descent method. The momentum method and the adaptive learning rate method are combined to adjust the learning rate of each parameter by calculating the first-order moment (mean) and second-order moment (variance) of the gradient.

[0026] Step 5: Optimize the hyperparameters of the ONLSTM neural network trained in step 4. Use grid search (GS) to tune the hyperparameters of the ONLSTM neural network and select the best parameter combination to train the ONLSTM neural network obtained in step 4 again.

[0027] Step 51: Determine candidate confidence values ​​of different hyperparameters based on prior knowledge and generate a grid to be searched;

[0028] Step 52: Use the GS method to tune the hyperparameters of the ONLSTM neural network through R 2 , MSE and MAE are used to judge the stability and accuracy of the parameters and select the optimal parameter combination; R 2 The closer to 1, the smaller the MSE and MAE, which means the better the parameter combination;

[0029] Step 53: Substitute the optimized parameters into the training to generate the initial ONLSTM neural network and obtain the initial input weights of the network.

[0030] Step 6: Embed the DNNG algorithm into the optimized ONLSTM neural network in step 5, optimize the input weights of the ONLSTM neural network, delete redundant variables, and establish a DNNG-ONLSTM neural network prediction model;

[0031] Step 61: Add the adaptive shrinkage coefficient of the DNNG algorithm to the input weights of the optimized ONLSTM neural network to establish the DNNG-ONLSTM neural network optimization expression, where the DNNG algorithm is to embed the same NNG algorithm into the input layer weight matrix and hidden layer weight matrix of the ONLSTM network respectively;

[0032] Step 62: Solve the DNNG-ONLSTM neural network optimization expression. For a given strangulation parameter S, use the COBYLA algorithm to solve the DNNG-ONLSTM neural network optimization expression to obtain the optimal contraction vector.

[0033] Step 63: Embed the optimal shrinkage vector into the DNNG-ONLSTM neural network to obtain a new set of prediction weight coefficients. After updating the calculation formula, the optimized DNNG-ONLSTM neural network prediction model output is obtained.

[0034] Step 7: Input the process variables in the industrial process into the DNNG-ONLSTM neural network prediction model obtained in step 6 to predict the corresponding quality variables.

[0035] The industrial processes in step 1 include: petrochemical industry, blast furnace refining, fermentation process, etc.

[0036] The ONLSTM neural network in step 3 includes the main forget gate, main input gate, forget gate, input gate, output gate and candidate unit state; the expression of the ONLSTM neural network is:

[0037] cumax(·)=cumsum(softmax(·)) (7)

[0038]

[0039] h t =o t ⊙tanh(C t ) (14)

[0040] The cumax(·) function is composed of the nested cumsum(·) function and the softmax(·) function, and is the activation function of the new main forget gate of ONLSTM. 1-cumax(·) is the reverse cumax(·) function and is the activation function of the new main input gate of ONLSTM. Represents the main forget gate, wt Represents the input at the current moment, represents the input weight of the main forget gate, represents the hidden layer weight of the main forget gate, Represents the bias of the main forget gate; represents the main output gate, represents the input weight of the main input gate, represents the hidden layer weight of the main input gate, represents the bias of the main input gate; w t express and The information overlapping part corresponds to the information overlapping path; w1 represents Medium to higher than w t The part corresponds to the main forgetting pathway; w2 represents Medium below w t The part corresponds to the main input path; C t Represents the candidate unit state, C t-1 represents the candidate unit state at the previous moment, represents the output of the candidate unit state; h t Represents the implicit state, o t Represents the output of the output gate.

[0041] The training process based on the gradient descent method in step 4 includes:

[0042] Step 41: Forward calculation: calculate the output values ​​of each gate unit and ONLSTM unit;

[0043] Step 42: Reverse calculation; calculate the loss function value of the prediction model. The loss function used in model training is the root mean square error, and its expression is:

[0044]

[0045] where y i and are the actual value and estimated value of the data sample, respectively, and n is the number of data samples. MSE reflects the training status of the network. The smaller the MSE value, the better the data set fitting effect;

[0046] Step 43: Gradient Update: Based on the corresponding error terms, the Adam optimization algorithm is used to update the network weights. The Adam optimization algorithm is a first-order optimization algorithm that can replace the traditional stochastic gradient descent algorithm. It has higher computational efficiency and better convergence performance within the same training cycle, while requiring less computational space.

[0047] In step 6, the DNNG algorithm is embedded in the ONLSTM neural network optimized in step 5, the input weights of the ONLSTM neural network are optimized, redundant variables are deleted, and a DNNG-ONLSTM neural network prediction model is established.

[0048] In step 61: After the DNNG algorithm is embedded into the ONLSTM neural network, the output of each gating unit of the DNNG-ONLSTM neural network is obtained as follows:

[0049]

[0050] Among them, f′ t represents the output of the forgotten gate after update, σ(·) represents the Sigmoid activation function, represents the input at the current moment i, α i represents the shrinkage coefficient of the embedded ONLSTM input weight matrix, ⊙ represents the vector dot multiplication operation, Represents the implicit state of the previous moment, Represents the input weight of the updated forget gate, represents the implicit state of the previous moment, β i represents the shrinkage coefficient of the weight matrix embedded in the ONLSTM hidden layer, represents the hidden layer weight of the updated forget gate, Represents the bias of the updated forget gate; i′ t represents the output of the input gate after update, represents the input weight of the updated input gate, represents the hidden layer weight of the input gate after the update, represents the bias of the updated input gate; o′ t represents the output of the output gate after update, represents the input weight of the output gate after update, represents the hidden layer weight of the updated output gate, Represents the bias of the updated output gate; represents the output of the candidate state unit after update, tanh(·) represents the hyperbolic tangent activation function, represents the input weight of the candidate state unit after update, represents the hidden layer weight of the candidate state unit after update, Represents the bias of the candidate state unit after update; represents the output of the newly added main forget gate, cumax(·) represents the new activation function of ONLSTM, Represents the input weight of the main forget gate after update, Represents the hidden layer weight of the main forget gate after update, Represents the bias of the main forget gate after update; Represents the output of the newly added main input gate, 1-cumax represents the reverse cumax function, represents the input weight of the main input gate after update, represents the hidden layer weight of the main input gate after update, represents the bias of the main input gate after update; w′ t represent and The information overlap part; C′ t is the updated output;

[0051] Then the DNNG-ONLSTM formula can be expressed as:

[0052]

[0053] Where g represents the activation function of the fully connected layer; W y and b y are the learning weight matrix and bias vector of the fully connected output layer respectively; p and q are the input data dimension and the number of nodes in the model hidden layer respectively; α and β are the shrinkage vectors of the embedded ONLSTM input weight matrix and the hidden layer input matrix respectively;

[0054] Step 62: Solve the optimization expression of the DNNG-ONLSTM neural network. For a given strangulation parameter S, use the trust region algorithm to solve equation (25) to obtain the optimal shrinkage vector θ * , and substitute it into it to obtain a new set of prediction weight coefficients:

[0055]

[0056] Among them, when α i = 0, it means eliminating the input data from the candidate input data And β j = 0 means to delete the recursive hidden neurons from the model and candidate status.

[0057] Step 63: Embed the optimal shrinkage vector into the DNNG-ONLSTM neural network to obtain a new set of prediction weight coefficients. After updating the calculation formula, the optimized DNNG-ONLSTM neural network prediction model output is obtained; its expression is:

[0058]

[0059] The beneficial effects of the present invention are:

[0060] The present invention provides an industrial soft-sensing method based on a non-negative strangled sparsification matrix ONLSTM. The method embeds a DNNG algorithm into an ONLSTM network to construct a DNNG-ONLSTM neural network prediction model and applies it to the prediction of key quality variables in complex industrial processes. On the one hand, the proposed method fully utilizes data to achieve modeling by automatically learning hierarchical structure information through ONLSTM. On the other hand, by embedding two sets of NNG algorithms into the input data and hidden layer nodes of the ONLSTM model, non-negative constraints are introduced to achieve variable selection and network structure sparse optimization. Under the premise of ensuring the model's prediction ability, redundant input data is eliminated. At the same time, considering that the complexity of the ONLSTM network structure is increased compared to LSTM, the NNG algorithm is added to its hidden layer to reduce the model calculation amount and eliminate redundant nodes of the network model to achieve model sparsification, reduce model complexity and training difficulty, and improve the generalization performance of the model. The method proposed by the present invention can reduce the influence of irrelevant variables on the prediction results, accurately capture the subtle fluctuations and changing trends of SO2 concentration, and achieve accurate and stable prediction of SO2 concentration in the desulfurization process of thermal power plants. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 Schematic diagram of LSTM unit structure in the prior art;

[0063] Figure 2 A schematic diagram of the ONLSTM unit structure in the sparse ONLSTM industrial soft sensing method based on non-negative strangulation provided by the present invention;

[0064] Figure 3 A schematic diagram of the overall structure of the ONLSTM neural network prediction model in the sparse ONLSTM industrial soft sensing method based on non-negative strangulation provided by the present invention;

[0065] Figure 4 A schematic diagram of the overall internal structure topology of the ONLSTM neural network prediction model in the sparse ONLSTM industrial soft sensing method based on non-negative strangulation provided by the present invention;

[0066] Figure 5 This is a schematic diagram of the desulfurization process flow of a coal-fired power plant involved in Example 2 provided by the present invention;

[0067] Figure 6A comparison chart of the net flue gas SO2 concentration emission prediction curves obtained by using different soft sensing methods in Example 2 provided by the present invention;

[0068] Figure 7 The error distribution histogram of the net flue gas SO2 concentration emission prediction results obtained by using different soft sensing methods in Example 2 provided by the present invention;

[0069] Figure 8 A schematic diagram of variable selection coefficients in the sparse ONLSTM industrial soft sensing method based on non-negative strangulation in Example 2 provided by the present invention;

[0070] Figure 9 This is a schematic diagram of the structural optimization coefficients in the sparse ONLSTM industrial soft sensing method based on non-negative strangulation in Example 2 provided by the present invention. DETAILED DESCRIPTION

[0071] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0072] First, let’s introduce the basic professional knowledge involved:

[0073] 1. ONLSTM Neural Network

[0074] ONLSTM adds a main forget gate and a main input gate to sort the hidden layer neurons on the basis of LSTM, thereby introducing the concept of information hierarchy. ONLSTM divides the input information into three parts through the main forget gate and the main input gate, namely the main forget path, the main input path and the information overlap path, and then calculates them separately and merges them into the output. Its structure is as follows Figure 1 shown.

[0075] Figure 1 It is a traditional LSTM network structure. Figure 2 Add a new structure to ONLSTM. The calculation process of the traditional LSTM gate unit is as follows:

[0076] f t =σ(x t W xf +h t-1 W hf +b f ) (1)

[0077] i t =σ(x t W xi +h t-1 W hi +b i ) (2)

[0078] o t=σ(x t W xo +h t-1 W ho +b o ) (3)

[0079]

[0080] Among them, f t ,i t , o t , are the outputs of the forget gate, input gate, output gate, and candidate unit state respectively; σ(·) represents the Sigmoid activation function; x t Represents the input at the current moment, W xf Represents the input weight of the forget gate, h t-1 represents the implicit state of the previous moment, W hf represents the hidden layer weight of the forget gate, b f Represents the bias of the forget gate; W xi Represents the input weight of the input gate, W hi represents the hidden layer weight of the input gate, b i Represents the bias of the input gate; W xo Represents the input weight of the output gate, W ho represents the hidden layer weight of the input gate, b o Represents the bias of the output gate; tanh(·) represents the hyperbolic tangent activation function, W xx Represents the input weight of the candidate unit state, W hc The hidden layer weight representing the candidate unit state, h c Represents the bias of the candidate cell state.

[0081] The update calculation process of the hidden state and unit state in LSTM is:

[0082]

[0083] h t =o t ⊙tanh(C t ) (6)

[0084] Among them, C t and h t is the updated unit state and hidden state, and ⊙ is the vector dot product operation.

[0085] On this basis, ONLSTM introduces a new activation function, and the calculation process of its new parameter update is as follows:

[0086] cumax(·)=cumsum(softmax(·)) (7)

[0087]

[0088]

[0089] h t =o t ⊙tanh(C t ) (14)

[0090] The cumax(·) function is composed of the nested cumsum(·) function and the softmax(·) function, and is the activation function of the new main forget gate of ONLSTM. 1-cumax(·) is the reverse cumax(·) function and is the activation function of the new main input gate of ONLSTM. It represents the main gate of oblivion; Represents the input weight of the main forget gate; Represents the hidden layer weight of the main forget gate; Represents the bias of the forget gate; Represents the main input gate; Represents the input weight of the main input gate; Represents the hidden layer weights of the main input gate; Represents the bias of the main input gate; express and The information overlapping part corresponds to the information overlapping path; w1 represents Medium to higher than w t The part corresponds to the main forgetting pathway; w2 represents Medium below w t The part corresponds to the main input path.

[0091] The order of ONLSTM hidden neurons is defined as an information hierarchy, where high-level neurons store long-term information that is retained for a long time, while low-level neurons store short-term information that can be quickly forgotten. Formula (13) represents C t The update logic intuitively displays the fusion calculation process of different pathways. ONLSTM can process information flow at different levels, thereby achieving better model prediction results.

[0092] 2. Non-negative strangulation algorithm

[0093] The NNG (Non-Negative Garrote) algorithm is a model optimization algorithm based on a constraint function. It integrates information node selection with the optimization of weight coefficients and was originally used to solve linear optimization problems. The basic principle of the NNG algorithm is to add a constraint function to the least squares algorithm, constructing a nonlinear function with constraints. The function then finds the extreme value of this function to obtain the optimal solution for the parameters under these constraints. Within the constraints, the solution process compresses some parameters to near zero while maintaining prediction accuracy.

[0094] Take the linear model as an example:

[0095] y=βx+ε (15)

[0096] Where x=[x1,x2,…x p ] and y represent input and output variables respectively, β=[β1,β2,...,β p ] represents the coefficient; p is the input dimension, and ε is the random error.

[0097] Let X∈R n*p is the input matrix, n is the total number of samples; Y∈R n is the output matrix; let Representing coefficient estimates, we obtain a nonlinear function with constraints:

[0098]

[0099] Where S is the strangulation hyperparameter; θ=[θ1,θ2,…θ p ] is the contraction vector; θ i is the shrinkage coefficient corresponding to the i-th input variable.

[0100] Solving equation (15) yields the optimized contraction vector θ * Substituting into formula (15) we can get the updated coefficients Its expression is:

[0101]

[0102] The output after optimization is for:

[0103]

[0104] The stranglement hyperparameter sets limits on the shrinkage vector and requires the function to achieve the optimal solution within these limits. During the optimization process, the smaller the stranglement hyperparameter, the greater the constraint and the stronger the restriction, resulting in more shrinkage coefficients approaching zero. If a shrinkage coefficient approaches zero, its corresponding coefficient is also estimated to approach zero, meaning the corresponding input is deleted.

[0105] 3. Desulfurization process of coal-fired power plants

[0106] Desulfurization process of coal-fired power plants Figure 5 As shown in the figure, this coal-fired power plant uses a dual-absorption tower structure and limestone-gypsum wet process for flue gas desulfurization (FGD). This technology is widely adopted for its rapid reaction, efficient desulfurization, and low cost, and is the mainstream FGD process for large-scale thermal power plants. Its basic operating principle is based on the chemical reaction between CaCO₃ in the limestone slurry and SO₂ in the flue gas to produce gypsum, thereby achieving FGD. A complete FGD process consists of multiple subsystems, with the FGD absorption system being the core of the entire process and determining the FGD efficiency.

[0107] Example 1

[0108] This embodiment provides an industrial soft sensing method based on the non-negative strangulation sparsification matrix ONLSTM, such as Figure 3 As shown, the method includes:

[0109] Step 1: Collect input and output data of the industrial process as a sample data set;

[0110] The input data are: important process variables selected from industrial process data sets that may affect the prediction of the quality variables to be measured;

[0111] The output data is the quality variable corresponding to the output of the process variable in the industrial process;

[0112] The input and output data are collected by continuous and uniform sampling at intervals of 5 minutes to obtain a sample data set.

[0113] Industrial processes include petrochemicals, blast furnace refining, fermentation, and power generation. For example, in the power generation process, the SO2 concentration in the flue gas desulfurization process of a coal-fired power plant is predicted. The input data include the SO2 concentration in the flue gas from the furnace, the SO2 concentration in the flue gas from the first-stage absorption tower outlet, the pH value of the gypsum slurry in the first-stage absorption tower, the flue gas temperature at the absorption tower inlet, the total coal volume, the net flue gas flow rate at the furnace chimney inlet, the O2 concentration in the flue gas from the furnace, the median desulfurization efficiency of the unit, the outlet temperature and pressure of the second-stage absorption tower, the current of the gypsum slurry discharge pump A, and the power of the vacuum belt dehydrator. The output data is the SO2 concentration in the desulfurization process of the thermal power plant.

[0114] Step 2: Preprocess and divide the sample data set obtained in step 1;

[0115] Preprocessing includes missing value and outlier processing and data standardization;

[0116] Specifically, for variables that only contain some time points, if there are too many incomplete data and it is impossible to supplement them, such variables will be deleted, and variables whose data in the sample are all constant values ​​will be deleted;

[0117] For variables with some null values, the null values ​​are replaced by the average of the two preceding and following data.

[0118] Secondly, according to the process requirements, the operating range of the original data variables is summarized, and then the maximum and minimum clipping methods are used to eliminate some samples that are not within this range, and outliers are removed according to the 3σ criterion; the specific process is as follows:

[0119] First, the measured variables are measured with equal precision, and x1, x2, ..., x are obtained independently. N , find its arithmetic mean and residual error Where i = 1, 2, ..., N; and calculate the standard deviation σ, if a measurement value x i The residual error v t (1≤t≤N), if the following formula (31) is satisfied:

[0120]

[0121] The error is considered to be a gross error, and the data x containing the error t should be eliminated;

[0122] Finally, the input data and output data are standardized using the z-score method.

[0123] The sample data set is divided into: the first 80% of the preprocessed sample data set is used as the training data set, and the remaining 20% ​​is used as the test data set.

[0124] Step 3: Build ONLSTM neural network;

[0125] Construct an ONLSTM neural network with a forget gate, input gate, output gate, and candidate unit state.

[0126] Step 4: Train the ONLSTM neural network obtained in step 3;

[0127] The ONLSTM neural network is trained using an optimization algorithm based on the gradient descent method. The momentum method and the adaptive learning rate method are combined to adjust the learning rate of each parameter by calculating the first-order moment (mean) and second-order moment (variance) of the gradient.

[0128] Step 5: Optimize the hyperparameters of the ONLSTM neural network trained in step 4;

[0129] A grid search (GS) method is used to optimize the hyperparameters of the ONLSTM neural network. First, candidate confidence values ​​for different hyperparameters are determined based on prior knowledge to generate a search grid. The GS method is then used to optimize the hyperparameters of the ONLSTM neural network and select the optimal hyperparameter combination. Finally, the optimized parameters are substituted into the training to generate the initial ONLSTM neural network and obtain the network's initial input weights.

[0130] Step 6: Embed the DNNG algorithm into the optimized ONLSTM neural network in step 4, optimize the input weights of the ONLSTM neural network, delete redundant variables, and establish a DNNG-ONLSTM neural network prediction model;

[0131] Step 61: Add the adaptive shrinkage coefficient of the DNNG algorithm to the input weights of the optimized ONLSTM neural network to establish the DNNG-ONLSTM neural network optimization expression, where the DNNG algorithm is to embed the same NNG algorithm into the input layer weight matrix and hidden layer weight matrix of the ONLSTM network respectively;

[0132] Step 62: Solve the DNNG-ONLSTM neural network optimization expression. For a given strangulation parameter S, use the COBYLA algorithm to solve the DNNG-ONLSTM neural network optimization expression to obtain the optimal contraction vector.

[0133] Step 63: Embed the optimal shrinkage vector into the DNNG-ONLSTM neural network to obtain a new set of prediction weight coefficients. After updating the calculation formula, the optimized DNNG-ONLSTM neural network prediction model output is obtained.

[0134] Step 7: Input the process variables in the industrial process into the ONLSTM neural network prediction model obtained in step 6 to predict the corresponding quality variables.

[0135] The industrial processes in step 1 include: petrochemical industry, blast furnace refining, fermentation process, etc.

[0136] The ONLSTM neural network in step 3 includes the main forget gate, main input gate, output gate and candidate unit state; its expression is:

[0137] cumax(·)=cumsum(softmax(·)) (7)

[0138]

[0139] h t =o t ⊙tanh(C t ) (14)

[0140] The cumax(·) function is composed of the nested cumsum(·) function and the softmax(·) function, and is the activation function of the new main forget gate of ONLSTM. 1-cumax(·) is the reverse cumax(·) function and is the activation function of the new main input gate of ONLSTM. It represents the main gate of oblivion; Represents the input weight of the main forget gate; Represents the hidden layer weight of the main forget gate; Represents the bias of the main forget gate; Represents the main input gate; Represents the input weight of the main input gate; Represents the hidden layer weights of the main input gate; represents the bias of the main input gate; w t express and The information overlapping part corresponds to the information overlapping path; w1 represents Medium to higher than w t The part corresponds to the main forgetting pathway; w2 represents Medium below w t The part corresponds to the main input path.

[0141] The training process based on the gradient descent method in step 4 includes:

[0142] Step 41: Forward calculation: calculate the output values ​​of the gated unit and the ONLSTM unit;

[0143] Step 42: Reverse calculation; calculate the loss function value of the prediction model. The loss function used in model training is the root mean square error, and its expression is:

[0144]

[0145] where y i and are the actual value and estimated value of the data sample, respectively, and n is the number of data samples. MSE reflects the training status of the network. The smaller the MSE value, the better the data set fitting effect;

[0146] Step 43: Gradient Update: Based on the corresponding error terms, the Adam optimization algorithm is used to update the network weights. The Adam optimization algorithm is a first-order optimization algorithm that can replace the traditional stochastic gradient descent algorithm. It has higher computational efficiency and better convergence performance within the same training cycle, while requiring less computational space.

[0147] In step 6, the DNNG algorithm is embedded in the optimized ONLSTM neural network in step 4, the input weight of the ONLSTM neural network is optimized, redundant variables are deleted, and the DNNG-ONLSTM neural network prediction model is established, as shown in the following example: Figure 3 As shown;

[0148] In step 61: After the DNNG algorithm is embedded into the ONLSTM neural network, the output of each gating unit of the DNNG-ONLSTM neural network is obtained as follows:

[0149]

[0150] Among them, f′ t represents the output of the forgotten gate after update, σ(·) represents the Sigmoid activation function, represents the input at the current moment i, α i represents the shrinkage coefficient of the embedded ONLSTM input weight matrix, ⊙ represents the vector dot multiplication operation, Represents the implicit state of the previous moment, Represents the input weight of the updated forget gate, represents the implicit state of the previous moment, β i represents the shrinkage coefficient of the weight matrix embedded in the ONLSTM hidden layer, represents the hidden layer weight of the updated forget gate, Represents the bias of the updated forget gate; i′ t represents the output of the input gate after update, represents the input weight of the updated input gate, represents the hidden layer weight of the input gate after update, represents the bias of the updated input gate; o′ t represents the output of the output gate after update, represents the input weight of the output gate after update, represents the hidden layer weight of the updated output gate, Represents the bias of the updated output gate; represents the output of the candidate state unit after update, tanh(·) represents the hyperbolic tangent activation function, represents the input weight of the candidate state unit after update, represents the hidden layer weight of the candidate state unit after update, Represents the bias of the candidate state unit after update; represents the output of the newly added main forget gate, cumax(·) represents the new activation function of ONLSTM, Represents the input weight of the main forget gate after update, Represents the hidden layer weight of the main forget gate after update, Represents the bias of the main forget gate after update; Represents the output of the newly added main input gate, 1-cumax represents the reverse cumax function, represents the input weight of the main input gate after update, represents the hidden layer weight of the main input gate after update, represents the bias of the main input gate after update; w′ t represent and The information overlap part; C′ t is the updated output;

[0151] Then the DNNG-ONLSTM formula can be expressed as:

[0152]

[0153]

[0154] Where g represents the activation function of the fully connected layer; W y and b y They are the learning weight matrix and bias vector of the fully connected output layer, respectively. p and q are the input data dimension and the number of nodes in the model hidden layer, respectively. α and β are the shrinkage vectors of the embedded ONLSTM input weight matrix and the hidden layer input matrix, respectively. The overall internal structure topology of the DNNG-ONLSTM neural network prediction model is as follows: Figure 4 As shown;

[0155] Step 62: Solve the optimization expression of the DNNG-ONLSTM neural network. For a given strangulation parameter S, use the trust region algorithm to solve equation (25) to obtain the optimal shrinkage vector θ * , and substitute it into it to obtain a new set of prediction weight coefficients:

[0156]

[0157] Among them, when α i = 0, it means eliminating the input data from the candidate input data And β j = 0 means to delete the recursive hidden neurons from the model and candidate status.

[0158] Step 63: Embed the optimal shrinkage vector into the DNNG-ONLSTM neural network to obtain a new set of prediction weight coefficients. After updating the calculation formula, the optimized DNNG-ONLSTM neural network prediction model output is obtained; its expression is:

[0159]

[0160] Step 7: Input the process variables in the industrial process into the ONLSTM neural network prediction model obtained in step 6 to predict the corresponding quality variables.

[0161] Example 2

[0162] This embodiment provides an application of a soft sensing method based on a non-negative strangled sparsification matrix ONLSTM in predicting SO2 concentration in a desulfurization process of a thermal power plant. The method is implemented based on the method provided in Example 1 and includes:

[0163] Step 1: Collect the process variables in the flue gas desulfurization industrial distributed control system of the thermal power plant as input data and SO2 concentration as output variable; perform continuous and uniform sampling every 5 minutes, and obtain the sample data set recorded as (X all ,Y all );

[0164] Among them, the input data is shown in Table 1:

[0165] Table 1 Input data for SO2 concentration prediction method in desulfurization process of thermal power plant

[0166]

[0167]

[0168] Step 2: Input the input data into the DNNG-ONLSTM neural network prediction model to predict the SO2 concentration in the desulfurization process of the thermal power plant.

[0169] In order to reflect the superiority of the DNNG-ONLSTM algorithm proposed in this invention, its performance is compared with the LSTM algorithm, STA-LSTM (Spatial-Temporal Attention Long Short-Term Memory) algorithm, SA-CNN-ONLSTM (Spatial Attention Convolutional Neural Network-Ordered Neurons Long Short-Term Memory) algorithm, ONLSTM algorithm and NNG-ONLSTM (Non-Negative Garrote-Ordered Neurons Long Short-Term Memory) algorithm. The hyperparameter settings are shown in Table 2, and the simulation results are shown in Table 3.

[0170] Among them, the LSTM algorithm can refer to the introduction in "Hochreiter S, Schmidhuber J. Long Short-Term Memory [J]. Neural Computation, 1997, 9 (8): 1735-1780."; the STA-LSTM algorithm can refer to the introduction in "Yuan Xiaofeng, Li Lin, Shardt Yuri AW, et al. Deep learning with spatiotemporalattention-based LSTM for industrial soft sensor model development [J]. IEEE Transactions on Industrial Electronics, 2020, 68 (5): 4404-4414."; the SA-CNN-ONLSTM algorithm can refer to "Li Xiangyu, Sui Lin, Xiong Weili. Soft measurement algorithm based on self-attention mechanism and convolutional ONLSTM network [J]. Journal of Intelligent Systems, 2023, 18 (05): 957-965."; the ONLSTM algorithm can refer to "Shen Yikang, Tan Shawn, Sordoni Alessandro, et al. Ordered neurons: Integrating tree structures into recurrent neural networks[J].arXiv preprint arXiv:1810.09536,2018.”; NNG-ONLSTM is an ablation experiment. For details, please refer to the introduction in “Goyal T, Durrett G.Neural syntactic preordering for controlled paraphrase generation[J].arxiv preprint arxiv:2005.02013,2020.”

[0171] Table 2 Basic structures and hyperparameter settings of different networks

[0172]

[0173] The following parameters are used as evaluation indicators of algorithm performance:

[0174] (1) Model input size (MIS): the number of valid input data of the model; its expression is:

[0175]

[0176] Among them, V i Represents the input data x i Is it a valid input for the model? n is the number of input data, λ i is the coefficient corresponding to the input data.

[0177] (2) Coefficient of determination R 2 : It is used to measure the degree of fit between the model output and the true value, indicating the degree of fit of the model to the data. The closer the value is to 1, the better the model fits. Its expression is:

[0178]

[0179] Among them, n is the total number of test set samples, y i is the actual value of the test set sample output, is the output prediction value of the model, It is y i The average value of .

[0180] (3) Mean Square Error (MSE): reflects the degree of difference between the estimator and the estimated value, and is the average of the sum of the squares of the differences between the predicted value and the true value. MSE can quantify the accuracy of the model prediction and thus judge the degree of fit of the model. The smaller the MSE value, the stronger the prediction ability of the model, and vice versa, it means that the model has a large prediction error; its expression is:

[0181]

[0182] (4) Mean Absolute Error (MAE): This measures the difference between the predicted value and the true value. It is the average of the sum of the absolute values ​​of the deviations between the predicted value and the actual value. The smaller the MAE value, the closer the model's prediction result is to the true value, and the better the prediction performance. Its expression is:

[0183]

[0184] Table 3 SO2 concentration prediction results obtained by each model

[0185]

[0186] As shown in Table 3, compared with the LSTM and ONLSTM algorithms, the MSE and MAE of the STA-LSTM and SA-CNN-ONLSTM algorithms that introduce the attention mechanism and the NNG-ONLSTM and DNNG-ONLSTM algorithms that introduce the variable selection mechanism are relatively small, and the determination coefficient R 2The value is closer to 1, indicating that correctly distinguishing different input data can effectively improve the model's prediction accuracy. By making appropriate trade-offs in input data, both NNG-ONLSTM and DNNG-ONLSTM effectively reduce the impact of irrelevant variables on the prediction results and significantly improve the network's prediction performance. The DNNG-ONLSTM algorithm selects the fewest key production parameters and exhibits superior performance evaluation metrics. Specifically, it achieves the lowest MSE and MAE, and a significantly higher coefficient of determination than other algorithms. This result fully demonstrates the effectiveness of the DNNG-ONLSTM algorithm.

[0187] The prediction performance of different algorithms varies. To better highlight this difference, the present invention provides the SO2 concentration prediction result curve and prediction error frequency distribution histogram of the desulfurization process flow of different algorithms, as shown in Figure 2. Figure 6 and Figure 7 As shown. Figure 6 It can be seen that compared with other algorithms, the neural network prediction model based on DNNG-ONLSTM proposed in this invention has stronger data fitting ability, shows more sensitive performance, and can more accurately capture subtle fluctuations and changing trends in SO2 concentration.

[0188] Depend on Figure 7 It can be seen that the standard deviation of the neural network prediction model based on DNNG-ONLSTM proposed in the present invention is the smallest, indicating that the prediction results are more reliable and accurate, with high prediction accuracy and strong stability, which helps to better realize real-time monitoring and control of the desulfurization process output in the actual production process.

[0189] In addition, the present invention counts the shrinkage coefficients of all candidate input data (retain 2 decimal places) and plots them, as shown in the figure. Figure 8 As shown in the figure, the selected production parameters are consistent with the expert experience in the actual production process, indicating that the neural network prediction model based on DNNG-ONLSTM proposed in the present invention is feasible.

[0190] Figure 9 The numerical statistics of the shrinkage coefficients corresponding to the neuron nodes in each hidden layer of the algorithm are given (retaining 2 decimal places). Combining the data in the figure with the algorithm prediction results in Table 3, it can be concluded that the R of the neural network prediction model based on DNNG-ONLSTM proposed in this invention is 2 It is closest to 1, and MIS, MSE and MAE are the smallest, which achieves the optimization of the network structure while ensuring the prediction accuracy of the model.

[0191] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0192] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An industrial soft sensing method based on sparse ONLSTM with non-negative strangulation, characterized in that: The method comprises: Step 1: Collect input data and output data of the industrial process as a sample data set; Step 2: Preprocess and divide the sample data set obtained in step 1; Step 3: Construct an ONLSTM neural network with a forget gate, input gate, output gate, and candidate unit state; Step 4: Train the ONLSTM neural network obtained in step 3; Step 5: Optimize the hyperparameters of the ONLSTM neural network trained in step 4; Step 6: Embed the DNNG algorithm into the optimized ONLSTM neural network in step 5, optimize the input weights of the ONLSTM neural network, delete redundant variables, and establish a DNNG-ONLSTM neural network prediction model; Step 7: Input the process variables in the industrial process into the DNNG-ONLSTM neural network prediction model obtained in step 6 to predict the corresponding quality variables.

2. The method according to claim 1, characterized in that The step 6 comprises: Step 61: After embedding the DNNG algorithm into the ONLSTM neural network, the output of each gating unit of the DNNG-ONLSTM neural network is obtained as follows: Among them, f′ t represents the output of the forgotten gate after update, σ(·) represents the Sigmoid activation function, represents the input at the current moment i, α i represents the shrinkage coefficient of the embedded ONLSTM input weight matrix, ⊙ represents the vector dot multiplication operation, Represents the implicit state of the previous moment, Represents the input weight of the updated forget gate, represents the implicit state of the previous moment, β i represents the shrinkage coefficient of the weight matrix embedded in the ONLSTM hidden layer, represents the hidden layer weight of the updated forget gate, Represents the bias of the updated forget gate; i′ t represents the output of the input gate after update, represents the input weight of the updated input gate, represents the hidden layer weight of the input gate after update, represents the bias of the updated input gate; o′ t represents the output of the output gate after update, represents the input weight of the output gate after update, represents the hidden layer weight of the updated output gate, Represents the bias of the updated output gate; represents the output of the candidate state unit after update, tanh(·) represents the hyperbolic tangent activation function, represents the input weight of the candidate state unit after update, represents the hidden layer weight of the candidate state unit after update, Represents the bias of the candidate state unit after update; represents the output of the newly added main forget gate, cimax(·) represents the new activation function of ONLSTM, Represents the input weight of the main forget gate after update, Represents the hidden layer weight of the main forget gate after update, Represents the bias of the main forget gate after update; Represents the output of the newly added main input gate, 1-cumax represents the reverse cumax function, represents the input weight of the main input gate after update, represents the hidden layer weight of the main input gate after update, represents the bias of the main input gate after update; w′ t represent and The information overlap part; C′ t is the updated output; Then the DNNG-ONLSTM formula can be expressed as: Among them, g represents the activation function of the fully connected layer; W y and b y are the learning weight matrix and bias vector of the fully connected output layer respectively; p and q are the input data dimension and the number of nodes in the model hidden layer respectively; α and β are the shrinkage vectors of the embedded ONLSTM input weight matrix and the hidden layer input matrix respectively; Step 62: Solve the optimization expression of the DNNG-ONLSTM neural network. For a given strangulation parameter S, use the trust region algorithm to solve the optimal shrinkage vector θ * , and substitute it into it to obtain a new set of prediction weight coefficients: Among them, when α i = 0, it means eliminating the input data from the candidate input data And β j = 0 means to delete the recursive hidden neurons from the model and candidate status; Step 63: Embed the optimal shrinkage vector into the DNNG-ONLSTM neural network to obtain a new set of prediction weight coefficients. After updating the calculation formula, the optimized DNNG-ONLSTM neural network prediction model output is obtained; its expression is:

3. The method according to claim 2, characterized in that The step 4 comprises: The ONLSTM neural network is trained using an algorithm based on gradient descent. The momentum method and adaptive learning rate method are combined to adjust the learning rate of each parameter by calculating the first-order moment (mean) and second-order moment (variance) of the gradient. Step 41: forward calculation; Calculate the output values ​​of the gated unit and ONLSTM unit; Step 42: reverse calculation; Calculate the loss function value of the prediction model. The loss function used in model training is the root mean square error, and its expression is: where y i and are the actual value and estimated value of the data sample, respectively, and n is the number of data samples; MSE reflects the training status of the network. The smaller the MSE value, the better the fitting effect of the data set. Step 43: Gradient update; Based on the corresponding error terms, the Adam optimization algorithm is used to update the network weights.

4. The method according to claim 3, characterized in that The step 5 includes: using a grid search method to tune the hyperparameters of the ONLSTM neural network; Step 51: Determine candidate confidence values ​​for different hyperparameters and generate a grid to be searched; Step 52: Use the GS method to tune the hyperparameters of the ONLSTM neural network and select the optimal hyperparameter combination. The selection standard is R 2 The closer to 1, the smaller the MSE and MAE. Step 53: Substitute the optimized parameters into the training to generate the initial ONLSTM neural network and obtain the initial input weights of the network.

5. The method according to claim 4, characterized in that In step 1: The input data is the process variables in the industrial process; the output data is the quality variables of the output corresponding to the process variables in the industrial process; The input data and output data are collected by continuous and uniform sampling every 5 minutes to obtain a sample data set.

6. The method according to claim 5, characterized in that In step 2: The preprocessing includes missing value and outlier processing and data standardization; According to the actual process requirements, the operating range of the original data variables is determined, and the maximum and minimum limit methods are used to eliminate samples that are not within this range, and outliers are removed according to the 3σ criterion; The specific process is: First, the measured variables are measured with equal precision, and x1, x2, ..., x are obtained independently. N , find its arithmetic mean and residual error Where i = 1, 2, ..., N; and calculate the standard deviation σ, if a measurement value x i The residual error v i (1≤i≤N), satisfying the expression: The error is considered to be a gross error, and the data x containing the error i should be eliminated; Finally, the input data and output data were normalized using the z-score method; The sample data set is divided into: the first 80% of the preprocessed sample data set is used as a training data set, and the remaining 20% ​​is used as a test data set.

7. The method according to claim 6, characterized in that The industrial processes include petrochemicals, blast furnace refining and fermentation processes.

8. A method for predicting SO2 concentration in a flue gas desulfurization process, characterized in that: The method is implemented based on the method described in any one of claims 1 to 7.

9. The method according to claim 8, characterized in that The process variables include: SO2 concentration in the flue gas from the furnace, SO2 concentration in the flue gas at the outlet of the first-stage absorption tower, pH value of the gypsum slurry in the first-stage absorption tower, flue gas temperature at the inlet of the absorption tower, total coal quantity, net flue gas flow at the inlet of the furnace chimney, O2 concentration in the flue gas from the furnace, median value of the desulfurization efficiency of the unit, outlet temperature of the second-stage absorption tower, outlet pressure of the second-stage absorption tower, current of the gypsum slurry discharge pump A and power of the vacuum belt dehydrator.

10. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store computer instructions, and the computer instructions are loaded and executed by the processor to implement the industrial soft measurement method based on non-negative strangulation and sparse ONLSTM according to any one of claims 1 to 7.

Images