A soft sensing method for fermentation process based on topology-guided temporal convolutional network

Abstract

A fermentation process soft-sensing method based on a topologically guided temporal convolutional network belongs to the technical field of fermentation process soft-sensing. The method includes the following steps: 1) Data acquisition and integration: Using a simulation platform to acquire penicillin fermentation processes under different operating conditions, the acquired data is divided, collected, and integrated; 2) Data selection: Selecting data, removing redundant and useless data, and establishing a causal graph between variables; 3) Constructing and training a TGTCN model; and 4) Model prediction. The method adopts a fermentation process soft-sensing method based on a topologically guided temporal convolutional network. It utilizes a graph attention network and a temporal convolutional network to extract data in both temporal and spatial dimensions, increasing the generalizability of the model and enabling accurate measurement of key product quality across different fermentation processes.

Description

Technical Field

[0001] The invention belongs to the technical field of fermentation process soft measurement, and in particular relates to a fermentation process soft measurement method based on a topology-guided temporal convolutional network. Background Art

[0002] Fermentation involves the participation of organisms and produces the target product under appropriate fermentation conditions, such as pH, temperature, and agitation. This process is widespread across various industrial processes and inherently exhibits strong nonlinearity and complex mechanisms. Both theoretical modeling and process control present significant challenges, particularly due to the lack of suitable online sensors for analyzing key product qualities, such as product concentration and active bacterial cell concentration during penicillin fermentation. This necessitates offline analysis, significantly limiting the control and optimization of the fermentation process.

[0003] As data-driven modeling approaches demonstrate significant advantages in industrial processes, and to overcome the difficulty of analyzing key product qualities, they are widely used to estimate key product qualities from other online measurable variables. Due to the strong nonlinearity between product quality and operational variables in fermentation processes, data-driven modeling approaches for fermentation processes are primarily categorized into statistical methods, shallow machine learning methods, and deep learning methods. Statistical methods primarily include principal component analysis (PCA) and partial least squares (PLS). Shallow machine learning methods refer to traditional machine learning models other than deep learning, including shallow artificial neural networks (ANNs), support vector machines (SVMs), and Gaussian process models (GPMs). Deep learning primarily encompasses various deep neural networks with three or more layers, such as temporal convolutional networks (TCNs). Deep neural networks have been proven to be capable of approximating any nonlinear system and are widely used in process modeling.

[0004] Recently, Graph Neural Networks (GNNs), a branch of deep learning, have demonstrated superior performance in many fields. As a variant of traditional GNNs, Graph Convolution Networks (GCNs) demonstrate powerful representation capabilities by performing convolution operations on structured data.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A fermentation process soft sensing method based on a topology-guided temporal convolutional network comprises the following steps:

[0007] 1) Data acquisition and integration

[0008] The simulation platform InPenSim was used to obtain the penicillin fermentation process under different working conditions, and the data were divided, collected and integrated in batches of one hour.

[0009] 2) Data selection

[0010] The data were selected according to the Granger Causality Test (GC Test), redundant and useless data were removed and a causal diagram between variables was established.

[0011] 3) Modeling training

[0012] A topology-guided temporal convolutional network (TGTCN) model is constructed. The input of the model is the causal connection matrix between the selected retained data and variables. The TGTCN model is then used to learn and train the integrated data.

[0013] 4) Model prediction

[0014] The trained TGTCN model is used to predict the outlet concentration of penicillin fermentation products, and the prediction results are given.

[0015] Further, the process of step 1) is:

[0016] Step 1.1: Set up the InPenSim simulation process and obtain the output data of 36 variables of the simulation platform;

[0017] Step 1.2: Divide the data into training set, validation set and test set according to the set ratio;

[0018] Step 1.3: Since the data differences between different variables are large, it is necessary to standardize the data to facilitate model processing and calculation. The specific formula is as follows:

[0019]

[0020] Among them, X' is the data after standardization, X is the original data without standardization, μ is the mean of the data, and σ is the standard deviation of the data.

[0021] Further, the process of step 2) is:

[0022] Step 2.1: Use Granger causality test to select data variables. First, two different autoregressive models can be built to evaluate Granger causality:

[0023]

[0024] Among them, a 1j,l (i=1,2) and b 1j,l (j=1,2) is the coefficient of the autoregressive model; ε 1(t) and ε 1(2)(t) are the prediction errors of the full model and the simplified model respectively; p is the order of the autoregressive model, that is, the time lag involved in the model; J is the total number of process variables, x j(t) (j=3,4,...,J) represents the observed value of the jth variable at the tth sampling time point;

[0025] Step 2.2: Make the null hypothesis H0:F X2→X1 =0 and alternative hypothesis H1:F X2→X1 >0. If the null hypothesis is rejected, the F test means that there is a Granger causal relationship between X2 and X1. The F statistic defined as follows can be used to test this hypothesis:

[0026]

[0027] Where R0 and R1 are the residual sums of squares of the two models in formula (2), N is the sample size, and k is the confidence limit. If the statistic is greater than the critical value derived from the F distribution, the null hypothesis is rejected;

[0028] Step 2.3: Sort the variables according to the k value of Granger causality test to complete the variable selection process;

[0029] Step 2.4: Establish the causal relationship between variables based on the k value of Granger causality test.

[0030] Further, the process of step 3) is:

[0031] Step 3.1: TGTCN is based on the relevant algorithm to build a fermentation process outlet concentration prediction model; TGTCN is mainly based on the graph attention network (GAT) and TCN; the input of the graph attention layer is a set of node features, Where V is the number of nodes and T is the number of features in each node. In order to obtain sufficient expressive power, at least one learnable linear transformation is required. To do this, a weight matrix Perform a linear transformation on each node. Then perform a self-attention mechanism on the node: To calculate the attention coefficient e ijSubsequently, in order to make the coefficients between different nodes easier to compare, all the choices of node j are normalized using the softmax function. Finally, multi-head attention is used on this basis. Specifically, the results of K independent attention mechanisms are averaged to produce the following output representation:

[0032]

[0033] Among them, H represents the number of attention mechanisms, e ij This formula allows each node to pay attention to all other nodes, and incorporates the graph structure into the mechanism through masking, only calculating the nodes e ij ,in is the neighborhood of node i in the graph. Using multi-head attention is beneficial to stabilizing the learning process of self-attention; where a(.) is the LeakyReLU activation function.

[0034] TCNs are based on two principles: (a) the network produces an output with the same length as the input; and (b) there is no leakage from the future into the past. To achieve the first point, TCNs are implemented as regular one-dimensional convolutions with a kernel size of (1×a), where a represents the size of the convolution kernel, and the length of each hidden layer is the same as the length of the input layer. To achieve the second point, TCNs use causal convolutions, meaning that the output at the current time is convolved only with elements at the current time and earlier times. Figure 2 A TCN framework with two hidden layers and a kernel size of (1×a) is shown. TCN emphasizes the use of a deep, multi-layer framework to capture the long-term dynamics of historical data and the network's ability to look back in time to make predictions. The combination of GAT and TCN is called a topologically guided temporal convolutional layer (TGTCL).

[0035] Step 3.2: Stack TGTCL to form TGTCN and use the training data to train TGTCN, use the validation data to select hyperparameters for TGTCN, and finally obtain the trained TGTCN model.

[0036] Further, the process of step 4) is:

[0037] Step 4.1: Input the test data into the trained TGTCN model. Use the mean square error (RMSE) as the evaluation metric, calculated as follows:

[0038]

[0039] in, represents the real data, y irepresents the output of the model, and n represents the number of samples in the test set. Generally speaking, the smaller the RMSE, the closer the model's predicted value is to the true value, that is, the better the model's prediction effect.

[0040] By adopting the above technology, compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] In the present invention, a soft measurement framework based on topology-guided temporal convolutional network is proposed; in the TGTCN framework, the adjacency matrix topological structure describing the relationship between process variables is adopted as the carrier of knowledge; CGC detection is used to construct a directed topological graph of variable relationships, and a self-attention mechanism is used to discover knowledge from data and a regularized loss function is used to constrain the knowledge discovery process, and the discovered knowledge is used as an auxiliary to complete the topological graph; and based on the multi-head graph attention mechanism, a subgraph is divided for each variable and its neighbors in the topological graph; a temporal convolution operation is used for each subgraph to model the temporal characteristics of the data; this method uses graph attention network and temporal convolutional network to extract data in time and space dimensions, which increases the generalization of the model and can accurately measure the quality of key products in different fermentation processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a diagram of the TGTCL framework structure of the present invention;

[0043] Figure 2 It is a model structure diagram of the present invention;

[0044] Figure 3 is a comparison chart of the predicted results and the actual results in an embodiment of the present invention; DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0046] Reference Figure 2 A soft sensor modeling method for a fermentation process based on a time series graph network comprises the following steps:

[0047] (1) Obtaining a penicillin fermentation process dataset

[0048] Step 1.1: Set up the InPenSim simulation platform, measure variables every 0.2 hours, and set up six different operating conditions. Ultimately, obtain 6 × 36 × 1100 data points for 36 variables at 1100 moments under the six operating conditions.

[0049] Step 1.2: Divide the data into three parts according to the division criteria: 3×6×1100, 2×36×1100, and 1×36×1100, which are the training set, validation set, and test set respectively.

[0050] Step 1.3: Process each data according to the standardization formula.

[0051] (2) Data selection for the penicillin fermentation process dataset is performed as follows:

[0052] Step 2.1: Calculate the p-value of each variable with respect to the penicillin concentration variable based on the Granger causality test formula.

[0053] Step 2.2: Filter variables based on p-values ​​and retain the top 7 variables with p < 0.05: hot and cold water flow rate R W , volume V V , volume weight V W , exhaust gas CO2 concentration E CO2 , CO2 rising rate R CO2 , exhaust gas O2 concentration E O2 and O2 rise rate R O2 .

[0054] Step 2.3: Calculate the p-values ​​between each of the seven variables. If p < 0.05, consider the two variables to have a causal relationship and create a causal diagram.

[0055] (3) Perform TGTCN model training. The process is as follows:

[0056] Step 3.1: Input the training set into the model to obtain the first prediction value of the model.

[0057] Step 3.2: Based on the difference between the predicted value and the output value, adjust the parameters of the model so that the difference between the predicted value and the true value is reduced.

[0058] Step 3.3: Input the validation set into the model and obtain the output of the model for the validation set.

[0059] Step 3.4: Reset and adjust the model's hyperparameters based on the model's output results on the validation set.

[0060] Step 3.5: Repeat steps 3.1 to 3.4 until the difference between the model's predicted value and the true value is within the allowable error.

[0061] (4) Use the test data to test the model. The process is as follows:

[0062] Step 4.1: Input the training data into the trained TCN model and TGTCN model to obtain the predicted values ​​of the TCN model and TGTCN model on the test set respectively.

[0063] Step 4.2: Calculate the RMSE between the predicted value and the true value of the test set to evaluate the performance of the TCN model and the TGTCN model. The evaluation results are shown in the table

[0064] Table 1 RMSE comparison between TGTCN and TCN

[0065]

[0066] From the above comparison results, it can be seen that the quality prediction effect of the key products of the penicillin fermentation process of the present invention is better than that of the traditional prediction model.

[0067] The present invention adopts a fermentation process soft measurement method based on a topology-guided temporal convolutional network, which improves the prediction effect of the quality of key products in the fermentation process and has universality and versatility.

[0068] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The scope of protection of the present invention should not be regarded as limited to the specific forms described in the embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A soft-sensor modeling method for fermentation process based on topologically guided temporal convolutional network, characterized in that: The method comprises the following steps: 1) Data acquisition and integration: Use the simulation platform to obtain the penicillin fermentation process under different working conditions, and divide, collect and integrate the acquired data; 2) Data selection: select data, remove redundant and useless data, and establish a causal diagram between variables; 3) Modeling and Training: A topologically guided temporal convolutional network (TGTCN) model is constructed. The model input is the causal connection matrix between the data and variables retained after data selection. The TGTCN model is then used to learn and train the integrated data. The specific process of step 3) is as follows: Step 3.1) Construct the TGTCN model based on the graph attention network GAT and TCN; the input of the graph attention layer is a set of node features, Where V is the number of nodes, T is the number of features in each node; using the weight matrix Perform a linear transformation on each node; then perform a self-attention mechanism on the node To calculate the attention coefficient e ij ; Subsequently, in order to make the coefficients between different nodes easier to compare, all the choices of node j are normalized using the softmax function; finally, on this basis, multi-head attention is adopted; the results of H attention mechanisms are averaged to produce the following output representation: Among them, H represents the number of attention mechanisms, e ij It shows the importance of the feature of node j to node i. The formula uses masking to make each node focus only on the nodes connected to itself, and calculates the node e ij ,in is the neighborhood of node i in the graph, a(.) is the LeakyReLU activation function; Step 3.2) Stack TGTCL to form TGTCN and use the training data to train TGTCN, use the validation data to select hyperparameters for TGTCN, and finally obtain the trained TGTCN model. 4) Model prediction: The trained TGTCN model is used to predict the outlet concentration of fermentation products and the prediction results are given.

2. The fermentation process soft-sensor modeling method based on topology-guided temporal convolutional network according to claim 1, characterized in that: The specific process of step 1) is as follows: Step 1.1) Set up the simulation process and obtain the output data of the simulation platform variables; Step 1.2) Divide the data into training set, validation set and test set according to the set ratio; Step 1.3) Since the data differences between different feature variables are large, the data needs to be standardized to facilitate model processing and calculation. The specific formula is as follows: Among them, X' is the data after standardization, X is the original data without standardization, μ is the mean of the data, and σ is the standard deviation of the data.

3. The fermentation process soft-sensor modeling method based on topology-guided temporal convolutional network according to claim 1, characterized in that: The specific process of step 2) is as follows: Step 2.1) Use Granger causality test to select data variables and build two different autoregressive models to evaluate Granger causality. The formula is as follows: Among them, a 1j,l (i=1,2) and b 1j,l is the coefficient of the autoregressive model; ε 1(t) and ε 1(2)(t) are the prediction errors of the full model and the simplified model respectively; p is the order of the autoregressive model, that is, the time lag involved in the model; J is the total number of process variables, x j(t) (j=3,4,...,J) represents the observed value of the jth variable at the tth sampling time point; Step 2.2) Make the null hypothesis H0:F X2→X1 =0 and alternative hypothesis H1:F X2→X1 >0; if the null hypothesis is rejected, the F test means that there is a Granger causal relationship between X2 and X1; the F statistic defined as follows is used to test this hypothesis: Where R0 and R1 are the residual sums of squares of the two models in formula (2), N is the sample size, and k is the confidence limit; if the statistic is greater than the critical value derived from the F distribution, the null hypothesis is rejected; Step 2.3) Sort the variables according to the k value of Granger causality test to complete the variable selection process; Step 2.4) Establish the causal relationship between variables based on the k value of Granger causality test.

Images