Method and device for improving fermentation efficiency of enzymatically hydrolyzed cottonseed protein

Abstract

The invention discloses a method and a device for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein, and relates to the technical field of enzymatic efficiency regulation and control. By collecting fermentation parameter data and fermentation product data, multiple groups of training samples are constructed, a product yield prediction model is trained, the state of the fermentation process is defined as a real-time state vector, a real-time action vector is set for a control parameter, a Markov decision process is constructed, and a fermentation process control model is constructed based on the Markov decision process. The fermentation process control model is trained through the fermentation parameter data in the fermentation process and the product yield prediction model, and the fermentation process control model is used to output a recommended action vector implemented for the next collection time. The invention can not only predict the future state of the fermentation process, but also provide a control strategy for improving the future fermentation product yield, and guide the online regulation and control of the fermentation process.

Description

Technical Field

[0001] The invention relates to the technical field of enzymolysis efficiency regulation and control, in particular to a method and a device for improving the fermentation efficiency of enzymolysis of cottonseed protein. Background Art

[0002] Cottonseed is the main byproduct of cotton production and is rich in plant protein. Through fermentation and enzymatic hydrolysis, the protein in cottonseed can be converted into a variety of peptides and amino acids with higher biological activity and nutritional value, which are widely used in food, feed and medicine. Fermentation and enzymatic hydrolysis of cottonseed protein is an important biotransformation process that requires precise control of multiple fermentation parameters (such as temperature, pH value, substrate concentration, ventilation volume, etc.) to ensure fermentation efficiency and product quality.

[0003] During the fermentation process, there is a complex interaction between different fermentation parameters. Appropriate regulation of these parameters helps to optimize fermentation conditions and increase the yield and concentration of target products (such as specific peptides or amino acids). However, due to the variability and nonlinear effects of parameter regulation, it is difficult to accurately control the optimal combination of parameters by relying solely on manual experience, which often leads to problems such as low fermentation efficiency and yield fluctuations.

[0004] At present, the control of the fermentation process mainly relies on empirical models and regular manual sampling and adjustment. However, manual sampling and adjustment have poor timeliness, cannot achieve real-time optimization control, lack adaptive capabilities, and are difficult to cope with real-time changes in various fermentation parameters during the fermentation process.

[0005] A Chinese patent with authorization announcement number CN111100889B discloses a method for improving the efficiency of enzymatic hydrolysis of lignocellulose, which is to add a polypeptide during the enzymatic hydrolysis of lignocellulose, wherein the sequence of the polypeptide is CH3CO-InK-NH2 (3≤n≤6), where I represents isoleucine and K represents lysine; however, this method lacks real-time regulation capability.

[0006] To this end, the present invention provides a method and a device for improving the fermentation efficiency of enzymatically hydrolyzing cottonseed protein. Summary of the invention

[0007] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a method and device for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein, which can not only predict the future state of the fermentation process, but also provide a control strategy for improving the yield of future fermentation products and guide the online regulation of the fermentation process.

[0008] To achieve the above object, the present invention discloses a method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein, comprising the following steps:

[0009] Step 1: every preset collection time, collect fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process;

[0010] Step 2: By obtaining delayed labels, construct several groups of training samples, each group of training samples includes time series input samples and time series label samples;

[0011] Step 3: Take the time series input samples of each group of training samples as input and the corresponding time series label samples as output to train the product yield prediction model;

[0012] Step 4: define the state of the fermentation process as a real-time state vector, set a real-time action vector for the control parameter, and construct a Markov decision process based on the real-time state vector and the real-time action vector;

[0013] Step 5: Construct a fermentation process control model based on the Markov decision process, train the fermentation process control model through the fermentation parameter data and the product yield prediction model in the fermentation process, and use the fermentation process control model to output the recommended action vector to be implemented for the next collection time;

[0014] The collection duration is the duration of a preset data collection cycle;

[0015] The method of collecting fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process is:

[0016] The parameter values ​​of various fermentation parameters at each unit time during the fermentation process are collected in real time through detection sensors corresponding to various fermentation parameters; the detection sensors include, for example, temperature sensors, pH sensors, etc.;

[0017] By using biosensors, the fermentation yield value per unit time during the fermentation process is collected in real time;

[0018] The parameter values ​​of various fermentation parameters at each unit time constitute fermentation parameter data, and the fermentation output value at each unit time constitutes fermentation product data;

[0019] The method of constructing several groups of training samples by delaying the acquisition of labels, each group of training samples including time series input samples and time series label samples, is as follows:

[0020] The data collection period during the fermentation process is numbered as i;

[0021] The number of the most recent data collection cycle in the fermentation process is marked as i0; it can be understood that the most recent data collection cycle refers to the data collection cycle corresponding to the current moment in the actual implementation process;

[0022] For the i0-1th data collection cycle:

[0023] Arrange each fermentation parameter in the data collection cycle in chronological order to form a set of fermentation parameter sequences;

[0024] The fermentation yield values ​​in the data collection period are arranged in chronological order to form a set of product yield sequences;

[0025] The fermentation parameter sequence of the i0-1th data collection period is taken as a set of time series input samples;

[0026] The ratio of the fermentation output value at the beginning of the i0th data collection cycle to the output value at the beginning of the i0-1th data collection cycle is taken as a set of time series label samples;

[0027] The method of training the product yield prediction model is as follows: taking the time series input samples of each group of training samples as input and the corresponding time series label samples as output:

[0028] The time series input samples of each group of training samples are used as inputs of a product yield prediction model, and the product yield prediction model uses the predicted value of the yield ratio of the fermentation product at the start time of the corresponding data collection cycle and the start time of the previous data collection cycle as output; the product yield prediction model uses the time series label samples of the corresponding data collection cycle as prediction targets, and the difference between the predicted value of the yield ratio and the time series label samples as prediction errors, and minimizes the sum of squares of prediction errors of all data collection cycles as a training target; the product yield prediction model is trained until the sum of squares of the prediction errors reaches convergence, and the training is stopped; the product yield prediction model is any one of the time series prediction models;

[0029] The state of the fermentation process is defined as a real-time state vector, and the method of setting a real-time action vector for the control parameter is as follows:

[0030] The fermentation output value and various fermentation parameter values ​​at the beginning of each data collection cycle are used as the real-time state vector and current state of the data collection cycle;

[0031] Selecting several parameters from the fermentation parameters as control parameters, and forming a real-time action vector with the selected control parameters, wherein each element in the real-time action vector corresponds to an adjustment ratio of each control parameter at the beginning of the current data collection cycle;

[0032] The method of constructing the Markov decision process based on the real-time state vector and the real-time action vector is:

[0033] Construct the state transfer function P(s'|s, a) and the reward function R(s, a); where a corresponds to the adjustment action of the control parameter corresponding to the real-time action vector, s' is the next state of the fermentation process after the adjustment action a is executed, and s is the current state;

[0034] The state transfer function is used to describe the probability of transferring to a new state s' after executing action a in the current state s, and the reward function describes the immediate reward obtained by executing action a in state s;

[0035] The state transfer function may be constructed in the following manner:

[0036] According to the historical fluctuation sequence of each fermentation parameter in the past fermentation process, several fluctuation curves of the fermentation parameter are constructed, and the fluctuation curves are fitted to obtain the fluctuation function of the fermentation parameter; the horizontal axis of each point in the fluctuation curve represents time, and the numerical axis represents the ratio of the parameter value per unit time to the initial parameter value; the independent variable of the fluctuation function is time, and the dependent variable is the ratio of the parameter value of the fermentation parameter per unit time to the initial parameter value;

[0037] For control parameters, the recommended parameter values ​​of each control parameter after adjustment are calculated, and the other fermentation parameters are not changed;

[0038] For each fermentation parameter, the function value at each subsequent unit time in the current data collection cycle is obtained through the corresponding fluctuation function, the recommended parameter value is multiplied by the function value at each unit time to obtain the predicted parameter value, and the predicted parameter values ​​at each unit time are sorted in chronological order to obtain a predicted parameter sequence;

[0039] The prediction parameter sequence of each fermentation parameter is formed into a set of time series input samples, which are input into the product yield prediction model to obtain the predicted value of the yield ratio output by the product yield prediction model; the predicted value of the yield ratio is multiplied by the yield at the beginning of the current data collection cycle, which is the yield value in the next state after the adjustment action is performed, and the last item of the prediction parameter sequence of each fermentation parameter is the value of the corresponding fermentation parameter in the next state;

[0040] Add a noise term ε based on the calculated next state, wherein the noise term ε obeys a Gaussian probability distribution;

[0041] By using the kernel density estimation method, the probability density of P(s'|s, a) is estimated for the next state after adding the noise term ε;

[0042] The reward function is constructed as follows:

[0043] R(s,a)=r_imm+ r_ter; where r_imm is the yield value in the next state, and r_ter is the total yield of the fermentation product at the end of fermentation. is the preset discount factor;

[0044] The fermentation process is modeled as a Markov decision process, denoted as M = (S, A, P, R, γ); where S is the state space, including all state vectors, A is the action space, including all action vectors, R and P are the state transfer function and reward function respectively;

[0045] The method of training the fermentation process control model and using the fermentation process control model to output the recommended action vector to be implemented for the next acquisition duration is:

[0046] Select the deep reinforcement learning model as the fermentation process control model;

[0047] The reward value function of the fermentation process control model is set to the reward function R(s, a) of the Markov decision process;

[0048] Collect the parameter values ​​and output values ​​of various fermentation parameters at the beginning of the fermentation process to form an initial state;

[0049] When the fermentation process is in progress, the current state of the i0th data collection cycle is input into the decision model of the fermentation process control model to obtain the corresponding control action a_i0, and the control action is used as the recommended action vector;

[0050] Apply action a_i0 to the Markov decision process, simulate the execution of the action, and obtain the new state s_(i0+1) according to the state transition function P(s'|s, a);

[0051] According to the reward function R(s, a), calculate the immediate reward r_i0 obtained by performing action a_i0 in state s_i0.

[0052] Store the conversion data consisting of state-action-reward-new state (s_i0, a_i0, r_i0, s_(i0+1)) into the experience replay pool;

[0053] A batch of conversion data is sampled from the experience replay pool as decision sample data to update the model parameters in the fermentation process control model.

[0054] The present invention discloses a device for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein, comprising a data acquisition module, a model training module, a decision process construction module and an action decision module; wherein the modules are electrically connected to each other;

[0055] The data collection module collects fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process at preset collection time intervals, constructs several groups of training samples by acquiring delayed labels, and each group of training samples includes time series input samples and time series label samples; and sends the time series input samples and time series label samples to the model training module;

[0056] The model training module takes the time series input samples of each group of training samples as input and the corresponding time series label samples as output, trains the product yield prediction model, and sends the product yield prediction model to the action decision module;

[0057] A decision process building module defines the state of the fermentation process as a real-time state vector, sets a real-time action vector for the control parameter, builds a Markov decision process based on the real-time state vector and the real-time action vector, and sends the Markov decision process to the action decision module;

[0058] The action decision module builds a fermentation process control model based on the Markov decision process, trains the fermentation process control model through the fermentation parameter data and the product yield prediction model in the fermentation process, and uses the fermentation process control model to output the recommended action vector to be implemented for the next collection duration.

[0059] The present invention discloses an electronic device, comprising: a processor and a memory, wherein the memory stores a computer program that can be called by the processor;

[0060] The processor executes the above method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein by calling the computer program stored in the memory.

[0061] The present invention discloses a computer-readable storage medium, on which a rewritable computer program is stored;

[0062] When the computer program is executed on a computer device, the computer device is enabled to execute the above method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] The present invention collects fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process at preset collection time intervals, constructs several groups of training samples by delaying the acquisition of labels, each group of training samples includes time series input samples and time series label samples, takes the time series input samples of each group of training samples as input, takes the corresponding time series label samples as output, trains the product yield prediction model, defines the state of the fermentation process as a real-time state vector, sets a real-time action vector for the control parameter, constructs a Markov decision process based on the real-time state vector and the real-time action vector, constructs a fermentation process control model based on the Markov decision process, trains the fermentation process control model through the fermentation parameter data and the product yield prediction model in the fermentation process, and uses the fermentation process control model to output the recommended action vector implemented by the next collection time; organically combines the time series data, state transition law and control strategy of the fermentation process, and constructs a complete fermentation process control model based on reinforcement learning and sequence prediction model. The model can not only predict the future state of the fermentation process, but also provide a control strategy to improve the future fermentation product yield, and guide the online regulation of the fermentation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a flow chart of the method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein in Example 1 of the present invention;

[0066] Figure 2 This is an example diagram of a fluctuation curve of substrate concentration in Example 1 of the present invention;

[0067] Figure 3 This is a module connection diagram of the device for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein in Example 2 of the present invention;

[0068] Figure 4 This is a schematic diagram of the structure of an electronic device in Embodiment 3 of the present invention;

[0069] Figure 5 This is a schematic diagram of the computer-readable storage medium structure in Example 4 of the present invention. DETAILED DESCRIPTION

[0070] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] Example 1

[0072] like Figure 1 As shown, the method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein comprises the following steps:

[0073] Step 1: every preset collection time, collect fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process;

[0074] Step 2: By obtaining delayed labels, construct several groups of training samples, each group of training samples includes time series input samples and time series label samples;

[0075] Step 3: Take the time series input samples of each group of training samples as input and the corresponding time series label samples as output to train the product yield prediction model;

[0076] Step 4: define the state of the fermentation process as a real-time state vector, set a real-time action vector for the control parameter, and construct a Markov decision process based on the real-time state vector and the real-time action vector;

[0077] Step 5: Construct a fermentation process control model based on the Markov decision process, train the fermentation process control model through the fermentation parameter data and the product yield prediction model in the fermentation process, and use the fermentation process control model to output the recommended action vector to be implemented for the next collection time;

[0078] Among them, the collection duration is the duration of a preset data collection cycle, that is, the collection duration at each interval, and the real-time parameter values ​​of various fermentation parameters in this data collection cycle and the real-time output of fermentation products during the fermentation process are collected through various parameter sensors or measuring equipment; wherein, the fermentation parameters are several parameters that affect the output of fermentation products during the fermentation process, including but not limited to temperature, pH value, dissolved oxygen concentration, substrate concentration, enzyme concentration, etc.

[0079] The method of collecting fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process is:

[0080] The parameter values ​​of various fermentation parameters at each unit time during the fermentation process are collected in real time through detection sensors corresponding to various fermentation parameters; the detection sensors include, for example, temperature sensors, pH sensors, etc.;

[0081] By using biosensors, the fermentation yield value per unit time during the fermentation process is collected in real time;

[0082] The parameter values ​​of various fermentation parameters at each unit time constitute fermentation parameter data, and the fermentation output value at each unit time constitutes fermentation product data;

[0083] By collecting fermentation parameter data and fermentation product data, the necessary data basis is provided for the entire fermentation process control, ensuring that there is sufficient data support for subsequent data analysis, model training and process control.

[0084] Furthermore, the method of constructing several groups of training samples by delaying the acquisition of labels, each group of training samples including time series input samples and time series label samples is as follows:

[0085] The data collection period during the fermentation process is numbered as i;

[0086] The number of the most recent data collection cycle in the fermentation process is marked as i0; it can be understood that the most recent data collection cycle refers to the data collection cycle corresponding to the current moment in the actual implementation of this embodiment;

[0087] For the i0-1th data collection cycle:

[0088] Arrange each fermentation parameter in the data collection cycle in chronological order to form a set of fermentation parameter sequences;

[0089] The fermentation yield values ​​in the data collection period are arranged in chronological order to form a set of product yield sequences;

[0090] The fermentation parameter sequence of the i0-1th data collection period is taken as a set of time series input samples;

[0091] The ratio of the fermentation output value at the beginning of the i0th data collection cycle to the output value at the beginning of the i0-1th data collection cycle is taken as a set of time series label samples;

[0092] It can be understood that the value of the time series label sample is formed based on the fermentation parameters in the previous data collection cycle. Therefore, the yield prediction model can be trained with the time series input samples of the previous data collection cycle and the time series label samples of the current data collection cycle, and the time series input samples of the current data collection cycle can be used to predict the product yield of the next data collection cycle. Because the collection of the time series label samples is delayed than the time series input samples, the training sample is constructed using the delayed label acquisition method.

[0093] Furthermore, the method of training the product yield prediction model is as follows:

[0094] The time series input samples of each group of training samples are used as inputs of a product yield prediction model, and the product yield prediction model uses the predicted value of the yield ratio of the fermentation product at the start time of the corresponding data collection cycle and the start time of the previous data collection cycle as output; the product yield prediction model uses the time series label samples of the corresponding data collection cycle as prediction targets, and the difference between the predicted value of the yield ratio and the time series label samples as prediction errors, and minimizes the sum of squares of prediction errors of all data collection cycles as a training target; the product yield prediction model is trained until the sum of squares of the prediction errors reaches convergence, and the training is stopped; the product yield prediction model is any one of the time series prediction models, and the time series prediction models include but are not limited to RNN models and LSTM models;

[0095] It can be understood that the training data of the product yield prediction model is continuously generated during the fermentation process. Therefore, the product yield prediction model is adapted to the current fermentation process, and thus the prediction accuracy of the model for the yield ratio of the fermentation product is greatly improved;

[0096] In order to avoid the problem that some fermentation processes are short or the parameters in the early stage of fermentation are not accurate enough, in another embodiment, the fermentation parameter data and fermentation product data collected in other fermentation processes in the past can be collected in advance to construct corresponding training samples, and a product yield prediction original model can be pre-trained. The training samples generated in the current fermentation process are then used to fine-tune the product yield prediction original model. The model after model fine-tuning is the product yield prediction model.

[0097] Furthermore, the state of the fermentation process is defined as a real-time state vector, and the method of setting the real-time action vector for the control parameter is as follows:

[0098] The fermentation output value and various fermentation parameter values ​​at the beginning of each data collection cycle are used as the real-time state vector and current state of the data collection cycle;

[0099] For example, if the real-time state vector of the i0th data collection cycle is the current state of the fermentation process, then the next state is the real-time state vector of the i0+1th data collection cycle;

[0100] Selecting several parameters from the fermentation parameters as control parameters, and forming a real-time action vector with the selected control parameters, wherein each element in the real-time action vector corresponds to an adjustment ratio of each control parameter at the beginning of the current data collection cycle; for example, increasing the temperature by 0.1%;

[0101] The method of constructing the Markov decision process based on the real-time state vector and the real-time action vector is:

[0102] Construct the state transfer function P(s'|s, a) and the reward function R(s, a); where a corresponds to the adjustment action of the control parameter corresponding to the real-time action vector, s' is the next state of the fermentation process after the adjustment action a is executed, and s is the current state;

[0103] The state transfer function is used to describe the probability of transferring to a new state s' after executing action a in the current state s, and the reward function describes the immediate reward obtained by executing action a in state s.

[0104] Specifically, the state transfer function may be constructed in the following manner:

[0105] According to the historical fluctuation sequence of each fermentation parameter in the past fermentation process, several fluctuation curves of the fermentation parameter are constructed, and the fluctuation curves are fitted to obtain the fluctuation function of the fermentation parameter; the horizontal axis of each point in the fluctuation curve represents time, and the vertical axis represents the ratio of the parameter value per unit time to the initial parameter value; the independent variable of the fluctuation function is time, and the dependent variable is the ratio of the parameter value of the fermentation parameter per unit time to the initial parameter value; Figure 2 As shown, it is a fluctuation curve of substrate concentration, the horizontal axis is time, the unit is minutes, and the vertical axis is the ratio of substrate concentration to decay over time, the unit is molar concentration;

[0106] For control parameters, the recommended parameter values ​​of each control parameter after adjustment are calculated, and the other fermentation parameters are not changed;

[0107] For each fermentation parameter, the function value at each subsequent unit time in the current data collection cycle is obtained through the corresponding fluctuation function, the recommended parameter value is multiplied by the function value at each unit time to obtain the predicted parameter value, and the predicted parameter values ​​at each unit time are sorted in chronological order to obtain a predicted parameter sequence;

[0108] The prediction parameter sequence of each fermentation parameter is formed into a set of time series input samples, which are input into the product yield prediction model to obtain the predicted value of the yield ratio output by the product yield prediction model; the predicted value of the yield ratio is multiplied by the yield at the beginning of the current data collection cycle, and the yield is the yield value in the next state after the adjustment action is performed, and the last item of the prediction parameter sequence of each fermentation parameter is the value of the corresponding fermentation parameter in the next state;

[0109] Add a noise term ε based on the calculated next state, wherein the noise term ε obeys a Gaussian probability distribution;

[0110] By using the kernel density estimation method, the probability density of P(s'|s, a) is estimated for the next state after adding the noise term ε.

[0111] The reward function is constructed as follows:

[0112] R(s,a)=r_imm+ r_ter; where r_imm is the yield value in the next state, and r_ter is the total yield of the fermentation product at the end of fermentation. is the preset discount factor;

[0113] The fermentation process is modeled as a Markov decision process, denoted as M = (S, A, P, R, γ); where S is the state space, including all state vectors, A is the action space, including all action vectors, R and P are the state transfer function and reward function respectively;

[0114] It should be noted that in the MDP model, the state is composed of the output value and the fermentation parameters, the action corresponds to the regulation of the control parameters, and the state transfer function and the reward function describe the dynamic behavior and optimization goal of the system respectively. Through the above steps of constructing the Markov decision process, the fermentation process is abstractly modeled as an MDP, laying the foundation for the subsequent application of reinforcement learning algorithms to find the optimal control strategy.

[0115] Furthermore, the fermentation process control model is constructed based on the Markov decision process, and the fermentation process control model is trained through the fermentation parameter data and the product yield prediction model in the fermentation process, and the fermentation process control model is used to output the recommended action vector to be implemented for the next collection duration:

[0116] Selecting a deep reinforcement learning model as a fermentation process control model; the fermentation process control model includes but is not limited to a DQN model and an Actor-Critic model;

[0117] The reward value function of the fermentation process control model is set to the reward function R(s, a) of the Markov decision process;

[0118] Collect the parameter values ​​and output values ​​of various fermentation parameters at the beginning of the fermentation process to form an initial state;

[0119] When the fermentation process is in progress, the current state of the i0th data collection cycle is input into the decision model of the fermentation process control model to obtain the corresponding control action a_i0, and the control action is used as the recommended action vector;

[0120] Apply action a_i0 to the Markov decision process, simulate the execution of the action, and obtain the new state s_(i0+1) according to the state transition function P(s'|s, a);

[0121] According to the reward function R(s, a), calculate the immediate reward r_i0 obtained by performing action a_i0 in state s_i0.

[0122] Store the conversion data consisting of state-action-reward-new state (s_i0, a_i0, r_i0, s_(i0+1)) into the experience replay pool;

[0123] A batch of conversion data is sampled from the experience replay pool as decision sample data to update the model parameters in the fermentation process control model.

[0124] Specifically, when the fermentation process control model is DQN:

[0125] The values ​​of R(s, a) and R(s', a') are calculated using the Q network.

[0126] Calculate the temporal difference (TD) target y=r+γ*max_R, where γ is the discount factor and r is the immediate reward, i.e., the value of R(s,a). max_R represents the maximum value of the Q value corresponding to all possible actions a' in the next state s';

[0127] Calculating the loss function , and use an optimization algorithm (such as gradient descent) to update the Q network parameters to minimize the loss function.

[0128] When the fermentation process control model is an Actor-Critic model:

[0129] Use the Critic network to calculate the value of R(s, a).

[0130] Calculate the TD error δ=r+γ*R(s',μ(s'))-R(s,a), where μ(s') is the action output by the Actor network in state s'.

[0131] The policy gradient method is used to update the Actor network parameters, with the goal of maximizing E[R(s, a)], where E[R(s, a)] is the expected value of R(s, a).

[0132] Use the TD error δ to update the Critic network parameters and minimize the TD error.

[0133] It should be further explained that in order to verify the effect of the method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein proposed in this application on improving the fermentation efficiency, this application also provides the following experimental data for verification:

[0134] 1. Subjects:

[0135] 20 batches of cottonseed protein fermentation processes were selected, with an average fermentation time of 72 hours, and randomly divided into an experimental group (12 batches) and a control group (8 batches).

[0136] 2. Data collection:

[0137] Experimental group: According to step 1 of the method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein, fermentation parameter data (temperature, pH value, dissolved oxygen concentration, etc.) and fermentation product data were collected during the fermentation process.

[0138] Control group: normal artificial controlled fermentation method was used without data collection and model training.

[0139] 3. Model training:

[0140] According to steps 2 and 3 of the method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein, training samples are constructed and a product yield prediction model is trained.

[0141] According to steps 4 and 5, a Markov decision process model is constructed, and the fermentation process control model is trained based on the reinforcement learning algorithm.

[0142] 4. Fermentation process control:

[0143] Experimental group: Use the trained fermentation process control model to output adjustment suggestions for control parameters in real time and automatically adjust the fermentation parameters.

[0144] Control group: The fermentation parameters were manually adjusted by the process personnel based on their experience.

[0145] 5. Evaluation indicators:

[0146] Fermentation efficiency = final product yield / initial substrate concentration;

[0147] 6. Experimental results:

[0148] By statistically analyzing the fermentation process of 20 batches of cottonseed protein, the experimental result data shown in Table 1 were obtained;

[0149] Table 1 Experimental results data table

[0150]

[0151] 7. Analysis of experimental results:

[0152] (1) This table contains complete fermentation data for 20 batches, 12 batches each for the experimental group and the control group.

[0153] (2) The experimental group adopted a method to improve the fermentation efficiency of enzymatic hydrolysis of cottonseed protein, while the control group adopted the traditional artificial experience control method.

[0154] (3) Fermentation efficiency is calculated by “final product yield / initial substrate concentration”, which reflects the fermentation conversion efficiency.

[0155] (4) The fermentation efficiency of the 20 batches in the experimental group ranged from 73.9% to 82.5%, and that in the control group ranged from 65.1% to 73.5%.

[0156] (5) The average fermentation efficiency of the experimental group was 78.6%, while the average of the control group was 69.3%, an increase of 9.3 percentage points on average.

[0157] (6) In all 20 batches, the fermentation efficiency of the experimental group was significantly higher than that of the corresponding batches of the control group.

[0158] (7) Even the lowest value of 73.9% in the experimental group was higher than the highest value of 73.5% in the control group, which fully demonstrated the superiority of this method.

[0159] (8) A one-sample t-test was performed, and the p value was 0.032<0.05, indicating that the difference between the two groups was statistically significant.

[0160] (9) The data fully verified that the method of improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein can significantly improve the fermentation efficiency of enzymatically hydrolyzed cottonseed protein.

[0161] (10) The experimental group data were better than the control group data both in terms of individual batches and average levels, demonstrating the excellent performance of this method.

[0162] (11) This intelligent control method can more effectively achieve precise adjustment of fermentation conditions and increase production capacity than traditional manual control methods.

[0163] (12) The above data provide strong experimental basis and support for the promotion of methods to improve the fermentation efficiency of enzymatic hydrolysis of cottonseed protein in practical industrial applications.

[0164] Example 2

[0165] like Figure 3 As shown, the device for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein comprises a data acquisition module, a model training module, a decision process building module and an action decision module; wherein the modules are electrically connected to each other;

[0166] The data collection module collects fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process at preset collection time intervals, constructs several groups of training samples by acquiring delayed labels, and each group of training samples includes time series input samples and time series label samples; and sends the time series input samples and time series label samples to the model training module;

[0167] The model training module takes the time series input samples of each group of training samples as input and the corresponding time series label samples as output, trains the product yield prediction model, and sends the product yield prediction model to the action decision module;

[0168] A decision process building module defines the state of the fermentation process as a real-time state vector, sets a real-time action vector for the control parameter, builds a Markov decision process based on the real-time state vector and the real-time action vector, and sends the Markov decision process to the action decision module;

[0169] The action decision module builds a fermentation process control model based on the Markov decision process, trains the fermentation process control model through the fermentation parameter data and the product yield prediction model in the fermentation process, and uses the fermentation process control model to output the recommended action vector to be implemented for the next collection duration.

[0170] Example 3

[0171] Figure 4 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 4 As shown, the present application also provides an electronic device 100. The electronic device 100 may include one or more processors and one or more memories. The memories store computer readable codes, and when the computer readable codes are executed by one or more processors, the method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein as described above can be executed.

[0172] The method or device according to the embodiment of the present application can also be used by Figure 4 The electronic device architecture shown in FIG. Figure 4 As shown, the electronic device 100 may include a bus 101, one or more CPUs 102, a ROM 103, a RAM 104, a communication port 105 connected to a network, an input / output component 106, a hard disk 107, etc. The storage device in the electronic device 100, such as the ROM 103 or the hard disk 107, may store the method for improving the fermentation efficiency of the enzymatically hydrolyzed cottonseed protein provided in the present application.

[0173] Furthermore, the electronic device 100 may further include a user interface 108. Figure 4 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 4 One or more components of an electronic device are shown.

[0174] Example 4

[0175] Figure 5 Schematic diagram of the computer-readable storage medium structure provided by an embodiment of the present application. Figure 5As shown, the present application discloses a computer-readable storage medium 200. Computer-readable instructions are stored on the computer-readable storage medium 200. When the computer-readable instructions are executed by the processor, the method for improving the fermentation efficiency of fermentation enzymatic hydrolysis of cottonseed protein according to the embodiment of the present application described with reference to the above figures can be executed. The computer-readable storage medium 200 includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0176] In addition, according to the implementation of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, the non-transitory machine-readable storage medium stores machine-readable instructions, the machine-readable instructions can be run by a processor to execute instructions corresponding to the method steps provided by the present application, and when the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0177] The methods, apparatuses, and devices of the present application may be implemented in many ways. For example, the methods, apparatuses, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers recording media storing programs for executing the method according to the present application.

[0178] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0179] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0180] The above preset parameters or preset thresholds are all set by technicians in this field according to actual conditions or obtained through large-scale data simulation.

[0181] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein, characterized in that: The following steps are involved: Step 1: every preset collection time, collect fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process; Step 2: construct multiple sets of training samples by obtaining delayed labels, each set of training samples includes time series input samples and time series label samples; Step 3: Take the time series input samples of each group of training samples as input and the corresponding time series label samples as output to train the product yield prediction model; Step 4: define the state of the fermentation process as a real-time state vector, set a real-time action vector for the control parameter, and construct a Markov decision process based on the real-time state vector and the real-time action vector; Step 5: Construct a fermentation process control model based on the Markov decision process, train the fermentation process control model through the fermentation parameter data and the product yield prediction model in the fermentation process, and use the fermentation process control model to output the recommended action vector to be implemented for the next collection time; The method of constructing the Markov decision process based on the real-time state vector and the real-time action vector is: Construct the state transfer function P(s'|s, a) and the reward function R(s, a); where a corresponds to the adjustment action of the control parameter corresponding to the real-time action vector, s' is the next state of the fermentation process after the adjustment action a is executed, and s is the current state; The state transfer function is used to describe the probability of transferring to a new state s' after executing action a in the current state s, and the reward function describes the immediate reward obtained by executing action a in state s; The fermentation process is modeled as a Markov decision process, denoted as M = (S, A, P, R, γ); where S is the state space, including all state vectors; A is the action space, including all action vectors; R and P are the state transfer function and reward function respectively; γ is the discount factor; and M is the Markov decision process; The state transfer function is constructed as follows: According to the historical fluctuation sequence of each fermentation parameter in the past fermentation process, a fluctuation curve of the fermentation parameter is constructed, and the fluctuation curve is fitted to obtain a fluctuation function of the fermentation parameter; the horizontal axis of each point in the fluctuation curve represents time, and the vertical axis represents the ratio of the parameter value per unit time to the initial parameter value; the independent variable of the fluctuation function is time, and the dependent variable is the ratio of the parameter value of the fermentation parameter per unit time to the initial parameter value; For control parameters, the recommended parameter values ​​of each control parameter after adjustment are calculated, and the other fermentation parameters are not changed; For each fermentation parameter, the function value at each subsequent unit time in the current data collection cycle is obtained through the corresponding fluctuation function, the recommended parameter value is multiplied by the function value at each unit time to obtain the predicted parameter value, and the predicted parameter values ​​at each unit time are sorted in chronological order to obtain a predicted parameter sequence; The prediction parameter sequence of each fermentation parameter is formed into a set of time series input samples, which are input into the product yield prediction model to obtain the predicted value of the yield ratio output by the product yield prediction model; the predicted value of the yield ratio is multiplied by the yield at the beginning of the current data collection cycle, which is the yield value in the next state after the adjustment action is performed, and the last item of the prediction parameter sequence of each fermentation parameter is the value of the corresponding fermentation parameter in the next state; Add a noise term ε based on the calculated next state, wherein the noise term ε obeys a Gaussian probability distribution; By using the kernel density estimation method, the probability density of P(s'|s, a) is estimated for the next state after adding the noise term ε; The reward function is constructed as follows: ; Among them, r_imm is the yield value in the next state, r_ter is the total yield of the fermentation product at the end of fermentation, is the preset discount factor.

2. The method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein according to claim 1, characterized in that: The collection duration is the duration of a preset data collection cycle; The method of collecting fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process is: The parameter values ​​of various fermentation parameters at each unit time during the fermentation process are collected in real time through detection sensors corresponding to various fermentation parameters; the detection sensors include temperature sensors and pH sensors; By using biosensors, the fermentation yield value per unit time during the fermentation process is collected in real time; The parameter values ​​of various fermentation parameters at each unit time constitute fermentation parameter data, and the fermentation output value at each unit time constitutes fermentation product data.

3. The method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein according to claim 2, characterized in that: The method of constructing multiple sets of training samples by delaying the acquisition of labels is as follows: The data collection period during the fermentation process is numbered as i; The number of the most recent data collection cycle in the fermentation process is marked as i0; For the i0-1th data collection cycle: Arrange each fermentation parameter in the data collection cycle in chronological order to form a set of fermentation parameter sequences; The fermentation yield values ​​in the data collection period are arranged in chronological order to form a set of product yield sequences; The fermentation parameter sequence of the i0-1th data collection period is taken as a set of time series input samples; The ratio of the fermentation yield value at the beginning of the i0th data collection cycle to the yield value at the beginning of the i0-1th data collection cycle is taken as a set of time series label samples.

4. The method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein according to claim 3, characterized in that: The method of training the product yield prediction model is as follows: taking the time series input samples of each group of training samples as input and the corresponding time series label samples as output: The time series input samples of each group of training samples are used as inputs of a product yield prediction model, wherein the product yield prediction model uses as outputs a predicted value of the yield ratio of the fermentation product at the start time of a corresponding data collection cycle to the start time of a previous data collection cycle; The product yield prediction model uses the time series label samples corresponding to the data collection period as the prediction target, the difference between the predicted value of the yield ratio and the time series label samples as the prediction error, and minimizes the sum of squares of the prediction errors of all data collection periods as the training target; The product yield prediction model is trained until the sum of squares of the prediction errors reaches convergence and then the training is stopped; the product yield prediction model is any one of the time series prediction models.

5. The method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein according to claim 4, characterized in that: The state of the fermentation process is defined as a real-time state vector, and the method of setting a real-time action vector for the control parameter is as follows: The fermentation output value and various fermentation parameter values ​​at the beginning of each data collection cycle are used as the real-time state vector and current state of the data collection cycle; A plurality of parameters are selected from the fermentation parameters as control parameters, and the selected control parameters are combined into a real-time action vector, wherein each element in the real-time action vector corresponds to an adjustment ratio of each control parameter at the beginning of the current data collection cycle.

6. The method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein according to claim 5, characterized in that: The method of training the fermentation process control model and using the fermentation process control model to output the recommended action vector to be implemented for the next acquisition duration is: Select the deep reinforcement learning model as the fermentation process control model; The reward value function of the fermentation process control model is set to the reward function R(s, a) of the Markov decision process; Collect the parameter values ​​and output values ​​of various fermentation parameters at the beginning of the fermentation process to form an initial state; When the fermentation process is in progress, the current state of the i0th data collection cycle is input into the decision model of the fermentation process control model to obtain the corresponding control action a_i0, and the control action is used as the recommended action vector; Apply action a_i0 to the Markov decision process, simulate the execution of the action, and obtain the new state s_(i0+1) according to the state transition function P(s'|s, a); According to the reward function R(s, a), calculate the immediate reward r_i0 obtained by performing action a_i0 in state s_i0; Store the transformation data consisting of state-action-reward-new state (s_i0, a_i0, r_i0, s_(i0+1)) into the experience replay pool; The transformed data are sampled from the experience replay pool as decision sample data to update the model parameters in the fermentation process control model.

7. A device for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein, which is used to implement the method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein according to any one of claims 1 to 6, characterized in that: It includes a data acquisition module, a model training module, a decision process building module and an action decision module; wherein each module is electrically connected; The data collection module collects fermentation parameter data and fermentation product data within a data collection cycle from the fermentation process at preset collection time intervals, constructs multiple sets of training samples by acquiring delayed labels, and each set of training samples includes time series input samples and time series label samples; and sends the time series input samples and time series label samples to the model training module; The model training module takes the time series input samples of each group of training samples as input and the corresponding time series label samples as output, trains the product yield prediction model, and sends the product yield prediction model to the action decision module; A decision process building module defines the state of the fermentation process as a real-time state vector, sets a real-time action vector for the control parameter, builds a Markov decision process based on the real-time state vector and the real-time action vector, and sends the Markov decision process to the action decision module; The action decision module builds a fermentation process control model based on the Markov decision process, trains the fermentation process control model through the fermentation parameter data and the product yield prediction model in the fermentation process, and uses the fermentation process control model to output the recommended action vector to be implemented for the next collection duration.

8. An electronic device, characterized in that: include: A processor and a memory, wherein: The memory stores a computer program that can be called by the processor; The processor executes the method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein according to any one of claims 1 to 6 in the background by calling the computer program stored in the memory.

9. A computer-readable storage medium, characterized in that: A rewritable computer program is stored thereon; When the computer program is executed on a computer device, the computer device is enabled to execute the method for improving the fermentation efficiency of enzymatically hydrolyzed cottonseed protein according to any one of claims 1 to 6 in the background.

Images