A fermentation process soft measurement method based on convolutional ONLSTM and self-attention

Abstract

The application discloses a fermentation process soft measurement method based on convolution ONLSTM and self-attention, and belongs to the technical field of soft measurement. The method considers the redundant information in the fermentation process variables, proposes a convolutional ordered neuron long short-term memory network (ONLSTM) multilayer time series prediction model with a self-attention mechanism, first extracts local features of input variables in the fermentation process by using a CNN and reduces the dimensionality; then inputs the extracted features into a multilayer ONLSTM network for time series feature extraction, judges the importance of each input variable through a level, and filters the redundant information in the characteristic variables; finally, the feature weights are dynamically adjusted in combination with the self-attention mechanism, the internal dependency relationship between the input variables is utilized, high weight is given to the high correlation variables, the full connection layer activation function is optimized, high-precision prediction of the dominant variables in the fermentation process is realized, and the method is verified to have high-precision prediction on the penicillin concentration by taking the penicillin fermentation process as an example.

Description

Technical Field

[0001] This invention relates to a soft measurement method for fermentation processes based on convolutional ONLSTM and self-attention, belonging to the field of soft measurement technology. Background Technology

[0002] In modern chemical processes, accurate monitoring of key quality parameters is crucial for ensuring safe and reliable production and meeting product quality requirements. However, many process variables in actual production are difficult to measure directly or in real time, making it difficult to obtain real-time feedback signals through closed-loop control, leading to feedback control failure. Soft sensing technology can effectively solve this problem and is therefore widely used in chemical production control processes.

[0003] Soft sensing technology establishes mathematical models by exploring the relationships between easily measurable auxiliary variables and difficult-to-measurable dominant variables to monitor key quality variables. Soft sensing modeling can generally be divided into two categories: mechanism-based modeling and data-driven modeling. Mechanism-based modeling involves analyzing the process mechanisms during production to establish mathematical models; this process is complex and challenging. Data-driven modeling, on the other hand, does not require a deep understanding of the process mechanisms; it only requires analyzing and modeling the data collected during production. Commonly used data-driven modeling methods include principal component regression, partial least squares regression, Gaussian process regression, support vector machines, and artificial neural networks, among other machine learning models.

[0004] Deep learning, as an important branch of machine learning, has superior data parsing capabilities when faced with large amounts of data and deep feature information, and it is also highly adaptable in data-driven soft sensing. Hochreiter et al. proposed a Long Short-Term Memory (LSTM) network, which, as a common deep learning model, has good modeling performance in processing time series information and extracting time series features. For example, Yuan et al. proposed a supervised LSTM and applied it to the penicillin fermentation process, which can effectively extract nonlinear dynamic features between process variables (see YUAN Xiaofeng, LI Lin, WANG Yalin. Nonlinear Dynamic Soft Sensor Modeling With Supervised Long Short-Term Memory Network[J].IEEE Transactions on Industrial Informatics,2020,16(5):3168-3176.). To improve the prediction performance of LSTM, researchers have also combined it with other algorithm models to form a hybrid prediction model. For example, Zheng et al. combined Convolutional Neural Networks (CNN) with LSTM and applied it to the soft sensor modeling of penicillin fermentation, which effectively improved the model's ability to discover the relevant features between samples (see HENG Jiaqi, MA Lianwei, WU Yi, et al. Nonlinear Dynamic Soft Sensor Development with a Supervised Hybrid CNN-LSTM Network for Industrial Processes[J].ACS OMEGA,2022,7(19):16653-16664).

[0005] Although LSTM models have shown good performance in addressing long-term dependencies in time series, they struggle to efficiently focus on the internal relationships between input variables. In soft measurement techniques, different auxiliary variables have varying impacts on prediction results, so it's necessary to consider how to differentiate the degree of influence of each auxiliary variable on the prediction results to improve prediction accuracy. Furthermore, the features extracted by LSTM models still contain redundant information, which can affect the prediction accuracy of the constructed model during the prediction process. Summary of the Invention

[0006] To address the existing problems, this invention provides a hybrid soft measurement model combining a self-attention mechanism-based convolutional ordered neuron long short-term memory network (SA-CNN-ONLSTM), and applies it to the penicillin fermentation process.

[0007] A soft measurement method for fermentation process based on convolutional ONLSTM and self-attention is proposed. The method first uses a CNN to extract local features and reduce the dimensionality of input variables in the fermentation process. Then, the extracted features are input into a multi-layer ONLSTM network for temporal feature extraction, and the importance of each input variable is judged by the layer, filtering out redundant information in the feature variables. Finally, the feature weights are dynamically adjusted using a self-attention mechanism, and high-correlation variables are assigned high weights based on the internal dependencies between input variables. Simultaneously, the activation function of the fully connected layer is optimized to predict the dominant variables in the fermentation process. The input variables are auxiliary variables in the fermentation process, and the dominant variables are key quality parameters in the fermentation process.

[0008] Optionally, the method includes:

[0009] Step 1: Obtain auxiliary variables during the fermentation process as input variables for the soft measurement prediction model. The soft measurement prediction model is a multi-layer temporal prediction model of a convolutional ordered neuron long short-term memory network with self-attention mechanism, including CNN layer, ONLSTM layer, self-attention layer and fully connected layer.

[0010] Step 2: Use CNN layers to extract local features and reduce dimensionality of the input variables;

[0011] Step 3: Input the extracted local features into the ONLSTM network for temporal feature extraction, and filter redundant information of the feature variables by judging the importance of the input variables through the hierarchy;

[0012] Step 4: Dynamically adjust feature weights using the self-attention mechanism to regulate the output of the hidden layer within the ONLSTM neuron;

[0013] Step 5: Improve the activation function of the fully connected layer to obtain the improved multilayer temporal prediction model of the convolutional ordered neuron long short-term memory network with self-attention mechanism.

[0014] Step 6: Use the improved multilayer temporal prediction model of convolutional ordered neuron long short-term memory network with self-attention mechanism to predict the dominant variables in the fermentation process.

[0015] Optionally, the method further includes,

[0016] The auxiliary variables and their corresponding dominant variables in the fermentation process were obtained and used to form the training set and the test set;

[0017] The improved multilayer temporal prediction model of the convolutional ordered neuron long short-term memory network with self-attention mechanism is trained using auxiliary variables and corresponding dominant variables in the training set to determine the parameters of the soft measurement prediction model.

[0018] Optionally, the ONLSTM network adds a main forget gate, a main input gate, and an information overlap portion pathway to each neuron based on the LSTM network. The main forget gate represents the historical information level, and the main input gate represents the input information level. The input information is updated in partitions according to the information level. Each neuron in the LSTM network contains three gate structures and one memory cell. The three gates are the forget gate, the input gate, and the output gate. When updating the state of the memory cell, a cumulative sum function is introduced to update its activation function.

[0019] Optionally, the ONLSTM network determines the importance of the information contained in each neuron based on the value of the cumulative sum function, retains neurons containing important information, and removes neurons that do not contain important information.

[0020] Optionally, the ONLSTM network is obtained by stacking multiple ONLSTM layers. By stacking multiple ONLSTM layers, the output of the previous ONLSTM layer is passed to the next ONLSTM layer, which is used as the input of the ONLSTM layer to establish a stacked ONLSTM layer. By using stacked multiple ONLSTM layers, the deep features of the input variables can be further extracted.

[0021] Optionally, the activation function of the fully connected layer is the Selu function obtained by multiplying the Elu function by a coefficient λ, where λ = 1.0507700987.

[0022] This application also provides a soft measurement method for penicillin fermentation process, wherein the method uses the above method to predict the penicillin concentration during penicillin fermentation; during the prediction process, the input variables include: aeration rate, agitator power, substrate feed rate, substrate supply temperature, substrate mass concentration, pH value, fermenter temperature, CO2 concentration, culture medium volume, reaction heat and cold water flow acceleration rate, and the dominant variable is the penicillin concentration in the fermenter.

[0023] Optionally, the parameters of the soft measurement prediction model include the number of CNN layers, CNN layer filters, number of ONLSTM layers, number of neurons in the ONLSTM hidden layer, learning rate, time step, and self-attention layer unit.

[0024] This application also provides the application of the above method in the field of fermentation.

[0025] The beneficial effects of this invention are:

[0026] To address the nonlinear dynamic modeling problem in the fermentation process and considering the redundant information in the process variables, this application proposes a multi-layer temporal prediction model using a convolutional ordered long short-term memory network (ONLSTM) with a self-attention mechanism. First, a CNN layer is introduced to extract and reduce the dimensionality of local features from the input sample data through convolutional operations. Then, multiple ONLSTM layers are stacked after the CNN layer, leveraging its ability to effectively solve the problem of long-term dependence of temporal features to further extract sample temporal features, addressing the dynamic nature of the fermentation process while discarding redundant information from the training process to reduce its impact on the prediction results. A self-attention mechanism layer is further introduced to update feature weights and adjust the output of the hidden layers within the ONLSTM neurons. Finally, the activation function of the fully connected layer is improved to reduce the influence of data nonlinearity and extract the feature output of the entire model, achieving high-precision prediction of the dominant variables in the fermentation process. The high-precision prediction of penicillin concentration is verified using the penicillin fermentation process as an example. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of the penicillin fermentation process;

[0029] Figure 2 This is a structural diagram of the SA-CNN-ONLSTM multilayer temporal prediction model with self-attention mechanism of convolutional ordered neuron long short-term memory network provided in the embodiments of the present invention;

[0030] Figure 3 This is an enlarged view of the ONLSTM structure in the SA-CNN-ONLSTM model provided in this embodiment of the invention;

[0031] Figure 4 This is a simulation diagram of the self-attention weights of each input sample in the self-attention layer of the SA-CNN-ONLSTM model provided in this embodiment of the invention;

[0032] Figure 5 This is a simulation diagram showing the proportion of self-attention weights for each sample in the self-attention layer of the SA-CNN-ONLSTM model provided in this embodiment of the invention.

[0033] Figure 6A These are the penicillin concentration prediction curves and error simulation graphs obtained using the LSTM model;

[0034] Figure 6B These are the penicillin concentration prediction curves and error simulation graphs obtained using the CNN-LSTM model;

[0035] Figure 6C These are the penicillin concentration prediction curves and error simulation graphs obtained using the CNN-ONLSTM model;

[0036] Figure 6D These are the penicillin concentration prediction curves and error simulation graphs obtained using the AT-CNN-ONLSTM model;

[0037] Figure 6E The penicillin concentration prediction curve and error simulation graph are obtained using the SA-CNN-ONLSTM model provided in this application.

[0038] Figure 7 The graph shows the box plots of the prediction errors of penicillin concentration for each network model. The horizontal axis represents LSTM (1), CNN-LSTM (2), CNN-ONLSTM (3), AT-CNN-ONLSTM (4), and SA-CNN-ONLSTM (5) as the model in this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0040] Example 1:

[0041] This embodiment provides a soft measurement method for the fermentation process based on convolutional ONLSTM and self-attention. The method constructs a multilayer temporal prediction model SA-CNN-ONLSTM with a convolutional ordered neuron long short-term memory network and a self-attention mechanism. It collects auxiliary variables and dominant variables in the fermentation process to form training and testing sets, completes the training of the SA-CNN-ONLSTM model to determine the parameters of the model, and uses the trained SA-CNN-ONLSTM model to predict the values ​​of the dominant variables in the actual fermentation process based on the easily collected auxiliary variables.

[0042] Specifically, the method includes:

[0043] Step 1: Obtain auxiliary variables during the fermentation process as input variables for the soft measurement prediction model. The soft measurement prediction model is a multi-layer temporal prediction model of a convolutional ordered neuron long short-term memory network with self-attention mechanism, including CNN layer, ONLSTM layer, self-attention layer and fully connected layer.

[0044] Step 2: Use CNN layers to extract local features and reduce dimensionality of the input variables;

[0045] Step 3: Input the extracted local features into a multi-layer ONLSTM network for temporal feature extraction, and filter redundant information of the feature variables by judging the importance of the input variables through the layers.

[0046] Step 4: Dynamically adjust feature weights using the self-attention mechanism to regulate the output of the hidden layer within the ONLSTM neuron;

[0047] Step 5: Improve the activation function of the fully connected layer to obtain the improved multilayer temporal prediction model of the convolutional ordered neuron long short-term memory network with self-attention mechanism.

[0048] Step 6: Use the improved multilayer temporal prediction model of convolutional ordered neuron long short-term memory network with self-attention mechanism to predict the dominant variables in the fermentation process.

[0049] Example 2

[0050] This embodiment provides a soft measurement method for the fermentation process based on convolutional ONLSTM and self-attention. The method first uses a CNN to extract local features and reduce the dimensionality of the input variables in the fermentation process. Then, the extracted features are input into a multi-layer ONLSTM network for temporal feature extraction, and the importance of each input variable is judged by the layer, filtering out redundant information in the feature variables. Finally, the feature weights are dynamically adjusted using a self-attention mechanism, utilizing the internal dependencies between input variables to assign high weights to highly correlated variables, while optimizing the activation function of the fully connected layer to predict the dominant variables in the fermentation process. The input variables are auxiliary variables in the fermentation process, and the dominant variables are key quality parameters in the fermentation process. Specifically, it includes:

[0051] Step 1: Obtain auxiliary variables during the fermentation process as input variables for the soft measurement prediction model. The soft measurement prediction model is a multi-layer temporal prediction model of a convolutional ordered neuron long short-term memory network with self-attention mechanism, including CNN layer, ONLSTM layer, self-attention layer and fully connected layer.

[0052] Step 2: Use CNN layers to extract local features and reduce dimensionality of the input variables;

[0053] Step 3: Input the extracted local features into the ONLSTM network for temporal feature extraction, and filter redundant information of the feature variables by judging the importance of the input variables through the hierarchy;

[0054] Step 4: Dynamically adjust feature weights using the self-attention mechanism to regulate the output of the hidden layer within the ONLSTM neuron;

[0055] Step 5: Improve the activation function of the fully connected layer to obtain the improved multilayer temporal prediction model of the convolutional ordered neuron long short-term memory network with self-attention mechanism.

[0056] Step 6: Use the improved multilayer temporal prediction model of convolutional ordered neuron long short-term memory network with self-attention mechanism to predict the dominant variables in the fermentation process.

[0057] by Figure 1 The following is a further detailed explanation of the method described in this application, using the penicillin fermentation process shown as an example:

[0058] First, the Pensim simulation platform was chosen to collect data generated during the penicillin fermentation process. This platform was developed by Professor Cinar for the actual penicillin fermentation process. The simulation platform includes essential components such as a fermenter, stirrer, and ventilation equipment, as well as feedstock components for substrate, acid, alkali, cooling water, and hot water, and is equipped with corresponding controllers. The fermentation process flow is attached. Figure 1 As shown, this scheme selects 11 variables, including aeration rate, CO2 concentration, pH value, and fermenter temperature, as auxiliary variables, and penicillin concentration as the key quality variable, i.e., the dominant variable. The sampling variables and set values ​​are shown in Table 1.

[0059] Table 1. Pensim sampling variables and settings

[0060]

[0061] Several sets of auxiliary variables and their corresponding dominant variables generated by the Pensim simulation platform were collected to form training and testing sets. The constructed SA-CNN-ONLSTM model was then trained using these sets. During training, the root mean square error (RMSE), mean absolute error (MAE), and correlation index (R²) were selected as the evaluation metrics for model prediction. 2 The calculation formulas are as follows:

[0062]

[0063]

[0064]

[0065] Among them, y i y represents the true value of penicillin concentration. i(pre) This represents the model-predicted value of penicillin concentration, where n represents the number of variables.

[0066] The SA-CNN-ONLSTM prediction model constructed in this application mainly consists of four layers: a CNN layer, an ONLSTM layer, a self-attention layer, and a fully connected layer. Considering the dynamic and nonlinear characteristics of the fermentation process, a CNN layer is first introduced to extract and reduce the dimensionality of local features of the input sample data through convolutional operations. Then, multiple ONLSTM layers are stacked after the CNN layer to leverage its advantage of effectively solving the problem of long-term dependence of temporal features, further extracting temporal features of the samples, addressing the dynamic nature of the fermentation process, and discarding redundant information during training to reduce its impact on the prediction results. A self-attention mechanism layer is further introduced to update feature weights and adjust the output of the hidden layers within the ONLSTM neurons. Finally, the activation function of the fully connected layer is improved to reduce the impact of data nonlinearity and extract the feature output of the entire model. The proposed model structure is attached. Figure 2 As shown.

[0067] In the ONLSTM layer, the sequential information of the hidden layer neurons is ordered and processed based on the traditional LSTM network. Compared to the insufficient judgment of the importance of the information transmitted by neurons in traditional LSTM, ONLSTM introduces the concept of information hierarchy, constructs an information importance evaluation index, and uses a partitioned update method to select and discard input information according to the information hierarchy, thereby improving the network model's ability to judge the importance of information. The ONLSTM model structure is shown in the attached figure. Figure 3 As shown.

[0068] The calculation formulas for the input gate, output gate, and forget gate of ONLSTM are the same as those for traditional LSTM networks. The difference lies in the calculation of the cell state C. t The update function has been improved, and a new activation function has been introduced into the neuron to extract the hierarchical structure of information more accurately:

[0069]

[0070] In the formula, cumax() represents the cumulative sum function, and based on this, a "master forgetting gate" is introduced into the hidden layer neurons. And "Main Input Gate" The calculation formula is:

[0071]

[0072]

[0073] and These represent the weights and biases corresponding to the main forget gate and the main input gate, respectively. Activation function. The cell state is converted into binary form. After adding the main forget gate and the main input gate, the improved rule for updating the cell state within the neuron is as follows:

[0074]

[0075]

[0076]

[0077]

[0078] In the formula ω t The overlapping part of the main forget gate and the main input gate.

[0079] The proposed hybrid model's ONLSTM layer improves upon the internal structure of LSTM hidden layer neurons by introducing a hierarchical concept to represent the importance of input temporal information. Due to the large volume and strong nonlinear characteristics of input samples in the penicillin fermentation process, variables exhibit high correlation. Furthermore, the large data volume introduces redundant variables that negatively impact model prediction accuracy. The hierarchical concept introduced in the ONLSTM model effectively extracts important information from the input data, thereby eliminating redundant information. The hierarchy is calculated within the ONLSTM unit. By introducing a main forget gate and a main input gate, the temporal features input to the hidden layer neurons are filtered and classified within these two gates. A partitioned update approach is used to pass information from different levels to different channels in the ONLSTM model, assigning corresponding weights to information of varying importance and retaining and propagating neurons containing highly important information.

[0080] Specifically, the importance of information is divided into regions based on the evaluation indicators of different input information levels, and data from different regions is fed into different update channels. The hierarchy of the main forget gate and the main input gate is set as d. f and d i When d f ≤d i This means that the current input x t The level should be higher than historical information h t-1 The hierarchy, meaning the information flows of the two intersect, requires the current input information to be integrated to a level higher than or equal to d. f In the hierarchy. When d f >d i , indicating historical records h t-1 and the current input x t Since they are mutually exclusive, they can only maintain their initial state; while for the remaining parts, the current input directly writes its information into [0, d]. i The interval is defined, while historical information is directly written to [d]. f,d max In this case, the current input and historical information combined cannot fill the entire storage space, leaving the remaining information to be forgotten.

[0081] The proposed model uses multiple stacked ONLSTM layers. By stacking multiple ONLSTM layers, the output of the previous ONLSTM layer is passed to the next ONLSTM layer, which is then used as the input to the next ONLSTM layer, thus establishing a stacked ONLSTM layer. By using stacked ONLSTM layers, deeper features of the input variables can be extracted. Compared with the single-layer ONLSTM model, the stacked ONLSTM model structure has a stronger fitting ability for the nonlinear features of the input time series features, and the model has stronger robustness.

[0082] The self-attention mechanism in the SA-CNN-ONLSTM prediction model has good application performance in sequence modeling. It utilizes the feature self-correlation of temporal information in soft measurement modeling to directly calculate the internal dependency of features and adaptively determine the magnitude of the input feature weights. At the same time, it is combined with LSTM to optimize the output features of the hidden layer, so that the model has better performance in long-term dependency problems.

[0083] The calculation process for determining the weight coefficients in the self-attention mechanism is as follows:

[0084] u a =tanh(Wh) t +b) (11)

[0085]

[0086]

[0087] In the formula, W represents the weight matrix, h t This represents the output of the previous LSTM model hidden layer at the current time step, where b represents the bias value, and u... a This represents the input hidden layer nonlinear transformation layer, u r Let θ represent the input sequence vector, s represent the self-attention feature weights, and s represent the input sequence vector. t This represents the sum of the products of the attention feature weights with different probabilities and the outputs of each hidden layer.

[0088] In modeling the penicillin fermentation process, standard attention mechanisms are insufficient for regression prediction problems in soft sensor modeling because they lack the additional information available in natural language processing tasks. Therefore, an improved standard attention mechanism is derived, resulting in a self-attention mechanism, which forms the self-attention layer in the proposed hybrid model.

[0089] The self-attention mechanism layer plays a crucial role in optimizing output features within the entire model. The input to the self-attention mechanism is the output of the hidden neurons in the previous ONLSTM layer. By combining the self-attention mechanism with the ONLSTM network model, the output of the ONLSTM layer is dynamically weighted based on the internal dependencies between input variables. Key feature variables are assigned high weights, indicating that these features have a significant impact on the model's prediction results. The self-attention mechanism filters feature information based on feature relevance and focuses the model's predictions on key features.

[0090] In the fully connected layers of the SA-CNN-ONLSTM prediction model:

[0091] Because the penicillin fermentation process is nonlinear, and activation functions introduce nonlinear characteristics into neural networks, the neural network model can better approximate nonlinear feature functions by imbuing neurons with nonlinear properties. Selecting an appropriate activation function for fully connected layers can effectively filter important input neuron information, thereby improving model prediction accuracy. Common activation functions in fully connected layers include: Sigmoid, ReLU, Elu, Leaky ReLU, and Selu. The formula for the Selu activation function is as follows:

[0092]

[0093] The Selu function is an Elu function multiplied by a coefficient λ. The Selu function can prevent gradient vanishing and exploding problems. By multiplying by the coefficient λ, its slope is greater than 1 in certain regions. This means that when a relatively small change comes in, the change can be amplified by a factor of 1.0507700987. Therefore, its input is amplified, which is a feature that Elu does not have.

[0094] Table 2 shows the basic parameter settings for the constructed penicillin concentration prediction model. The model parameters play a crucial role in the network performance; therefore, setting appropriate network model parameters can effectively improve the model's prediction accuracy. This approach continuously adjusts the model parameters through experimentation, thereby selecting the model parameters with the most ideal experimental results for predicting penicillin concentration.

[0095] Table 2. Parameter settings for the SA-CNN-ONLSTM model

[0096]

[0097]

[0098] Table 3 compares the performance of several common activation functions. Choosing a suitable activation function for the fully connected layer can effectively mitigate the impact of nonlinear characteristics of the input variables on prediction accuracy. Based on the evaluation metrics, this application selects Selu as the activation function for the fully connected layer, and the model achieves the best prediction performance.

[0099] Table 3 Evaluation metrics of model performance for different activation functions

[0100]

[0101] Figure 4 and Figure 5 The self-attention weights for each input sample in the self-attention layer and the weights assigned to input variables under different neurons are dynamically adjusted. By averaging the weights, the proportion of weight assigned to input variables under different neurons is obtained. The self-attention mechanism assigns weighted values ​​to input variables based on their internal correlation features; highly correlated input variables are given higher weights. The results show that input variable 5 has the lowest self-attention weight proportion, while input variables 4, 6, and 7 have relatively high proportions. This means that input variable 5 has the least impact on the prediction result of penicillin concentration, while input variables 4, 6, and 7 have a greater impact.

[0102] Table 4. Prediction results of penicillin concentration by each network model.

[0103]

[0104] Table 4 above compares the performance metrics of LSTM, CNN-LSTM, CNN-ONLSTM, AT-CNN-ONLSTM, and SA-CNN-ONLSTM models for predicting penicillin concentration. Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E The five algorithm models are shown below, representing the penicillin concentration prediction curves. Figure 7 The prediction errors of five different penicillin concentration algorithms are presented. Experimental results show that the SA-CNN-ONLSTM model proposed in this application achieves the highest RMSE (0.01551), the best fit to the actual value, the smallest error, and the highest prediction accuracy. Compared with other deep learning models, the proposed model effectively reduces the impact of redundant input information and has higher prediction accuracy, providing strong technical support for the optimized control of penicillin fermentation processes.

[0105] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fermentation process soft-sensing method based on convolutional ONLSTM and self-attention, characterized in that, The method first uses a CNN to extract local features and reduce the dimensionality of the input variables in the fermentation process; then, the extracted features are input into a multi-layer ONLSTM network for temporal feature extraction, and the importance of each input variable is judged by the layer to filter redundant information in the feature variables; finally, the feature weights are dynamically adjusted by combining a self-attention mechanism, and high-correlation variables are given high weights by utilizing the internal dependencies between input variables, while optimizing the activation function of the fully connected layer to achieve the prediction of the dominant variables in the fermentation process. The input variables are auxiliary variables in the fermentation process, and the dominant variables are key quality parameters in the fermentation process. The method includes: Step 1: Obtain auxiliary variables during the fermentation process as input variables for the soft measurement prediction model. The soft measurement prediction model is a multi-layer temporal prediction model of a convolutional ordered neuron long short-term memory network with self-attention mechanism, including CNN layer, ONLSTM layer, self-attention layer and fully connected layer. Step 2: Use CNN layers to extract local features and reduce dimensionality of the input variables; Step 3: The extracted local features are input into a multi-layer ONLSTM network for temporal feature extraction. The ONLSTM network adds a main forget gate, a main input gate, and an information overlap path to each neuron in the LSTM network. The main forget gate represents the historical information level, and the main input gate represents the input information level. The input information is partitioned and updated according to its information level to determine the importance of input variables and filter redundant information. Each neuron in the LSTM network contains three gate structures and one memory cell. The three gates are the forget gate, the input gate, and the output gate. When updating the state of the memory cell, a cumulative sum function is introduced to update its activation function. Step 4: The ONLSTM network is obtained by stacking multiple ONLSTM layers. By stacking multiple ONLSTM layers, the output of the previous ONLSTM layer is passed to the next ONLSTM layer, which is used as the input of the ONLSTM layer to establish a stacked ONLSTM layer. By stacking multiple ONLSTM layers, the deep features of the input variables can be further extracted. Combined with the self-attention mechanism, the feature weights are dynamically adjusted to regulate the output of the hidden layer in the ONLSTM neuron. Step 5: Improve the activation function of the fully connected layer, wherein the activation function of the fully connected layer is the Elu function multiplied by a coefficient. The Selu function obtained later An improved multilayer temporal prediction model of a convolutional ordered neuron long short-term memory network with self-attention mechanism was obtained. Step 6: Use the improved multilayer temporal prediction model of convolutional ordered neuron long short-term memory network with self-attention mechanism to predict the dominant variables in the fermentation process.

2. The method of claim 1, wherein, The method also includes, The auxiliary variables and their corresponding dominant variables in the fermentation process were obtained and used to form the training set and the test set; The improved multilayer temporal prediction model of the convolutional ordered neuron long short-term memory network with self-attention mechanism is trained using auxiliary variables and corresponding dominant variables in the training set to determine the parameters of the soft measurement prediction model.

3. The method of claim 1, wherein, The ONLSTM network determines the importance of the information contained in each neuron based on the value of the cumulative sum function, retains neurons containing important information, and removes neurons that do not contain important information.

4. A soft-sensing method for a penicillin fermentation process, characterized in that, The method described herein uses the method described in any one of claims 1-3 to predict the penicillin concentration during penicillin fermentation. During the prediction process, the input variables include: aeration rate, mixer power, substrate feed rate, substrate supply temperature, substrate mass concentration, pH value, fermenter temperature, CO2 concentration, culture medium volume, reaction heat and cold water flow acceleration rate, and the dominant variable is the penicillin concentration in the fermenter.

5. The method of claim 4, wherein, The parameters of the soft measurement prediction model include the number of CNN layers, CNN layer filters, number of ONLSTM layers, number of neurons in the ONLSTM hidden layer, learning rate, time step, and self-attention layer unit.

6. The application of the method according to any one of claims 1-3 in the field of fermentation.

Images