Fermentation control method, system and equipment based on neural network and medium

Abstract

The invention relates to the technical field of fermentation control, in particular to a fermentation control method, system and equipment based on a neural network and a medium, and the control method comprises the following steps: obtaining environmental parameters in a fermentation module, and preprocessing the environmental parameters; obtaining a risk signal of the feature data through prediction distribution quantification according to the time weighted loss function; setting a risk threshold, and obtaining an incentive reward according to an internal incentive function; establishing an auxiliary control model, and obtaining a safety control action according to the auxiliary control model; according to the hidden state of the feature data and the risk signal, a final control action is obtained through gradient strategy updating of the strategy network; acquiring a control signal according to the incentive reward, wherein the control signal is a final control action or a safety control action; the control precision and stability of the fermentation process are remarkably improved, and the negative effects of nonlinear mutation on microbial growth and product synthesis are effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fermentation control, and in particular to a fermentation control method, system, device and medium based on a neural network. BACKGROUND

[0002] At present, biological fermentation technology plays an important role in promoting green production and biological medicine due to its environmental friendliness and resource efficient utilization.

[0003] Taking the traditional ginkgo leaf fermentation industry as an example, it has been facing a series of challenges for a long time, such as the detection lag of key quality parameters, the uncertainty of production cycle, etc., which seriously restricts the optimization control level and the improvement of economic benefits of the process. Specifically, whether it is traditional PID control or model-based control method, there are significant limitations in dealing with complex batch processes; the parameters of the traditional PID controller are fixed, and it lacks adaptability, which is difficult to cope with the inherent nonlinear, time-varying and complex coupling characteristics of the biological fermentation process, especially when dealing with external environmental disturbances, it is easy to fail, resulting in inaccurate or unstable parameter control.

[0004] And the performance of the model-based control strategy is highly dependent on the accuracy of the process model, once the actual situation does not match the model, the accuracy of the prediction will be affected, thus limiting the control performance. This excessive dependence on model accuracy or artificial experience makes it difficult to further improve the control accuracy and efficiency of the fermentation process, especially under complex working conditions with long single batch duration and multi-parameter dynamic changes, it is difficult to achieve fine coordination and optimal control.

[0005] Some methods for controlling and predicting the fermentation process are proposed in the prior art. For example, CN114841416A discloses an amino acid state nitrogen prediction method for soy sauce fermentation process based on a long short-term memory network, which includes the following steps: S1, monitoring and collecting natural oil fermentation indexes on the A, B and C days after feeding of N fermentation tanks, and temperature indexes on the D to E days after feeding; S2, obtaining the time series of natural oil fermentation indexes and the time series of temperature indexes; S3, inputting the time series of natural oil fermentation indexes into a long short-term memory neural network to output a first two-dimensional feature vector; S4, inputting the time series of temperature indexes through an initialized linear layer and then through an activation function into another long short-term memory neural network to output a second two-dimensional feature vector; S5, splicing the first and second two-dimensional feature vectors to obtain a fusion feature vector, splicing the fusion feature vector with total nitrogen and moisture indexes to obtain a third two-dimensional feature vector, and outputting the predicted amino acid state nitrogen content after dimension reduction through a fully connected layer. CN120183557A discloses a fermentation process optimization and control method based on reinforcement learning and contrast learning. The method includes: obtaining historical batch data and to-be-controlled batch data of a penicillin fermentation process, using a deep reinforcement learning double actor network to construct an optimization and control model; optimizing the action value function by combining the loss function of contrast learning to enhance the recognition ability of the similarity and difference between states, inputting the historical batch data into the model for training, calculating the difference between the penicillin yield and the expected yield, updating the parameters using the back propagation algorithm, and automatically adjusting the hyperparameters through Bayesian optimization; finally, inputting the data of the to-be-controlled batch into the trained neural network control model, and outputting the penicillin yield control through iterative actions.

[0006] However, none of the existing solutions takes into account the cumulative error of the hidden state in the propagation process and the divergence trend of the prediction residual with the increase of the time step in the fermentation process, resulting in the inability of the existing methods to timely capture the risk signal of the sudden increase of the standard deviation, and thus the fermentation process still adjusts according to the historical optimal strategy, which may cause control failure, fermentation failure and other problems. SUMMARY

[0007] The present application aims to solve at least one of the above problems by providing a neural network-based fermentation control method, system, device and medium to solve the problem of low control precision and poor stability of the fermentation process in the prior art, which leads to the difficulty in effectively controlling nonlinear mutations. The present application reduces the influence of by-products in the fermentation process on the nonlinear mutation of microbial growth and product synthesis.

[0008] The object of the present application is achieved by the following technical solutions: The first aspect of the present application discloses a neural network-based fermentation control method for controlling fermentation parameters in a fermentation module, the control method comprising the following steps: obtaining environmental parameters in the fermentation module, preprocessing the environmental parameters to obtain feature data; obtaining the difference of hidden states in adjacent time steps of the feature data, and quantifying the risk signal of the feature data according to a time-weighted loss function through a prediction distribution; setting a risk threshold and obtaining an incentive reward according to an intrinsic incentive function; establishing an auxiliary control model and obtaining a safety control action according to the auxiliary control model; obtaining a final control action according to the hidden state of the feature data and the risk signal, and through gradient policy update of a policy network; obtaining a control signal according to the incentive reward, the control signal being the final control action or the safety control action; wherein the risk signal includes standard deviation, prediction uncertainty and prediction residual.

[0009] Preferably, the preprocessing of the environmental parameters comprises the following steps: cumulatively processing the obtained environmental parameters to convert them into cumulative variables; obtaining the nonlinear correlation between the cumulative variables according to a mutual information method to obtain feature variables with correlation; normalizing the feature variables and then expanding the normalized data to obtain the feature data; wherein the environmental parameters include dissolved oxygen concentration, pH value and feed flow rate.

[0010] Preferably, the expansion processing uses a sliding window method with fixed length time steps to expand the data.

[0011] Preferably, in the process of obtaining the difference of hidden states in adjacent time steps of the feature data, the difference of hidden states in adjacent time steps is taken as one of the inputs of the next layer of gate units.

[0012] Preferably, after obtaining the incentive reward according to the intrinsic incentive function, the following steps are further included: setting an external reward, superimposing the incentive reward and the external reward to obtain a composite reward.

[0013] Preferably, obtaining the safety control action according to the auxiliary control model comprises the following steps: taking the partial derivative of the control signal matched with the environmental parameters through Newton iteration method to solve the safety control action satisfying the minimum work requirement through reverse iteration of the auxiliary control model.

[0014] Preferably, the control signal according to the incentive reward comprises the following steps: determining whether the incentive reward is greater than zero; if the incentive reward is greater than zero, selecting the safety control action as the control signal; if the incentive reward is equal to zero, selecting the final control action as the control signal; outputting the control signal to an execution module in the fermentation module.

[0015] The second aspect of the present application discloses a control system for implementing any of the above methods, the control system comprising: a collection module, the collection module being configured to acquire environmental parameters in a fermentation module, and to preprocess the environmental parameters to obtain feature data; a prediction module, the prediction module being configured to obtain a difference value of a hidden state at adjacent time steps of the feature data, and to quantify a risk signal of the feature data according to a time-weighted loss function through a prediction distribution; an incentive module, the incentive module being configured to set a risk threshold, and to obtain an incentive reward according to an intrinsic incentive function; a control module, the control module being configured to establish an auxiliary control model, and to obtain a safety control action according to the auxiliary control model; the control module is further configured to obtain a control signal according to the incentive reward, the control signal being the final control action or the safety control action.

[0016] The third aspect of the present application discloses a computer device, comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement any of the above methods.

[0017] The fourth aspect of the present application discloses a computer readable storage medium, the computer readable storage medium storing a computer program, and a processor executing the computer program to implement any of the above methods.

[0018] Compared with the prior art, the present application has the following beneficial effects: The present application effectively deals with the nonlinearity and uncertainty in the fermentation process by capturing the change of hidden state in real time, quantifying the risk signal, dynamically adjusting the reward mechanism, and introducing an auxiliary control model, thereby improving the control precision and stability. In addition, the present application can capture the dynamic change of the hidden state in the ginkgo leaf fermentation process in real time, effectively identify the abnormal fluctuations of the risk signal, and realize timely response of the control action while ensuring the safety of the system by introducing a double control strategy. The present application significantly improves the control precision and stability of the fermentation process, and effectively reduces the negative impact of nonlinear mutation on microbial growth and product synthesis.

[0019] Compared with the traditional control method, the scheme can better cope with the complex coupling relationship and time-varying characteristics in the fermentation process, and improve the consistency of product quality and production efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flowchart of the fermentation control method. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0022] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative position relationship, movement condition, etc. between components in a certain posture, and if the certain posture changes, the directional indications also change accordingly.

[0023] In the present application, unless otherwise explicitly specified and limited, the terms "connection", "fixation" and the like should be understood in a broad sense, for example, "connection" can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be direct connection, or indirect connection through an intermediate medium; can be internal connection of two elements or interaction relationship between two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0024] In addition, if the present application has descriptions involving "first", "second" and the like, the "first", "second" and the like are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first" and "second" can explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing throughout the text includes three parallel solutions. Taking "A and / or B" as an example, it includes A solution, or B solution, or A and B solution. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, and when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor is it within the scope of protection claimed by the present application.

[0025] Example 1 In the traditional existing ginkgo leaf fermentation control process, the non-linear dynamic characteristics and complex coupling relationship lead to insufficient control accuracy of key parameters. The biological fermentation system has strong time-varying and uncertainty, and the microbial metabolic pathway is easily affected by environmental parameter disturbance. When the key variables such as dissolved oxygen concentration, pH value and feed flow rate suddenly change, the traditional control architecture is difficult to establish the dynamic mapping relationship between the input and output in real time. The control strategy with fixed parameters cannot capture the time sequence correlation characteristics of the hidden state in the fermentation process, and the method based on the mechanism model is limited by the model mismatch problem. When the prediction distribution deviates, the risk signal cannot be effectively quantified, resulting in that the control action lags behind the actual working condition change.

[0026] For example, during the operation of the batch fermentation tank, the dissolved oxygen concentration sensor collects data at a frequency of once per minute, the response delay of the pH electrode reaches 30 seconds, and the action period of the feed execution mechanism is 15 seconds. When the microorganism enters the logarithmic growth phase, the metabolic activity intensifies, leading to a sudden increase in dissolved oxygen demand. At this time, the non-linear correlation between the characteristic variables exceeds the preset threshold. Although the pre-processing method based on the sliding window can extract the time sequence characteristics, the propagation of the hidden state difference in the gating unit has cumulative error, and the prediction residual presents a divergent trend with the increase of time step. At this time, the control signal generation module fails to timely capture the risk signal of the sudden increase of standard deviation, and the execution mechanism still adjusts the feed rate according to the historical optimal strategy, resulting in that the dissolved oxygen level of the fermentation broth is continuously lower than the critical value.

[0027] In the face of the above problems, the embodiment first considers how to capture the dynamic changes of the hidden state in the fermentation process in real time. The traditional method relies on the analysis of historical data with fixed time steps, but the non-linear correlation caused by the metabolic mutation of microorganisms cannot be captured by the static model. To this end, by introducing the difference calculation of the hidden state in adjacent time steps and combining the time weighted loss function to quantify the prediction distribution, the abnormal fluctuations of risk signals such as standard deviation and prediction residual can be effectively identified.

[0028] Further, in view of the problem of lagging control action, the embodiment explores the feasibility of a dual control strategy. If the final control action generated by the strategy network is directly used, it may exacerbate system instability when the risk signal exceeds the threshold. To this end, an auxiliary control model is established to solve the action that meets the minimum safety requirement, and an internal incentive function is used to dynamically adjust the reward mechanism, so that the control signal realizes the balance between exploration and conservation within the controllable risk range.

[0029] For this purpose, with reference to Figure 1 The embodiment provides a fermentation control method based on a neural network, which is used for controlling fermentation parameters in a fermentation module (such as a fermentation tank and the like). The control method comprises the following steps: An environment parameter in the fermentation module is acquired, and the environment parameter is pre-processed to obtain feature data. obtain a difference value of the hidden state within adjacent time steps of the feature data, and quantify a risk signal of the feature data according to a time-weighted loss function through a prediction distribution; set a risk threshold, and obtain an incentive reward according to an intrinsic incentive function; establish an auxiliary control model, and obtain a safety control action according to the auxiliary control model; obtain a final control action according to the hidden state of the feature data and the risk signal, and through gradient policy updating of a policy network; obtain a control signal according to the incentive reward, the control signal being the final control action or the safety control action; The risk signal includes a standard deviation, a prediction uncertainty, and a prediction residual.

[0030] It can be understood that by quantifying the prediction deviation through a dynamic risk signal, combining an intrinsic incentive reward mechanism and an auxiliary control model, and synchronously implementing risk perception and safety constraints in neural network policy updating, both control accuracy and system stability can be considered in a complex nonlinear fermentation process.

[0031] The control method in this embodiment first obtains environmental parameters in the fermentation module, pre-processes the environmental parameters to obtain feature data, and the pre-processing process includes accumulation of the environmental parameters, extraction of relevant feature variables, normalization, and expansion processing.

[0032] Then, a difference value of the hidden state within adjacent time steps of the feature data is obtained, and a risk signal of the feature data is quantified according to a time-weighted loss function through a prediction distribution. The risk signal includes a standard deviation, a prediction uncertainty, and a prediction residual. This step can effectively capture dynamic changes in the fermentation process by calculating the difference value of the hidden state and combining the time-weighted loss function.

[0033] Next, a risk threshold is set, and an incentive reward is obtained according to an intrinsic incentive function. This step introduces a dynamic reward mechanism, which helps to balance exploration and conservation within a controllable risk range.

[0034] An auxiliary control model is established, and a safety control action is obtained according to the auxiliary control model. The introduction of the auxiliary control model can provide a control action that meets the minimum safety requirements when the risk signal exceeds the threshold.

[0035] A final control action is obtained according to the hidden state of the feature data and the risk signal, and through gradient policy updating of a policy network. This step uses the learning ability of a neural network to generate an optimal control strategy according to the current state and risk information.

[0036] Finally, the control signal is obtained according to the incentive reward, and the control signal is a final control action or a safety control action; by comparing the incentive reward with the reward threshold, the final control action or the safety control action is selected as the output, and a double control strategy is realized.

[0037] The above control method effectively deals with the nonlinearity and uncertainty in the fermentation process by capturing hidden state changes in real time, quantifying risk signals, dynamically adjusting the reward mechanism, and introducing an auxiliary control model, thereby improving control accuracy and stability. In addition, the above control method can capture the dynamic changes of hidden states in the ginkgo leaf fermentation process in real time, effectively identify abnormal fluctuations in risk signals, and realize timely response of control actions while ensuring system safety through the introduction of a double control strategy. The control accuracy and stability of the fermentation process are significantly improved, the negative impact of nonlinear mutations on microbial growth and product synthesis is effectively reduced, and compared with traditional control methods, the scheme of the embodiment can better cope with the complex coupling relationship and time-varying characteristics in the fermentation process, thereby improving the consistency of product quality and production efficiency.

[0038] Optionally, the preprocessing of the environmental parameters comprises the following steps: The acquired environmental parameters are subjected to time series accumulation processing to convert into accumulated variables; According to the mutual information method, the nonlinear correlation between the accumulated variables is obtained to obtain feature variables with correlation; The feature variables are subjected to normalization processing, and the normalized data are subjected to expansion processing to obtain feature data; The environmental parameters include dissolved oxygen concentration, pH value, and feed flow rate.

[0039] It can be understood that the existing preprocessing process of environmental parameters may not effectively extract feature variables with correlation, resulting in insufficient accuracy of subsequent risk signal prediction and control action. Through accumulation processing, the real-time data of dissolved oxygen concentration, pH value, and feed flow rate are accumulated in the time dimension to generate accumulated variables reflecting the parameter change trend. The mutual information method calculates the information entropy difference between the accumulated variables to quantify the nonlinear dependence relationship between the variables, and selects the feature variable combination with a mutual information value higher than a preset threshold. The normalization processing preferably adopts the minimum-maximum normalization method to map feature variables of different dimensions to the [0, 1] interval. The expansion processing adopts a sliding window method with a fixed length time step to cut the normalized data into continuous data segments according to the time sequence.

[0040] Specifically, the original data of dissolved oxygen concentration is accumulated to generate cumulative dissolved oxygen amount, the original data of pH value generates cumulative pH change amount, and the feed flow rate generates cumulative feed amount. Mutual information method is used to analyze the correlation between the three cumulative variables, for example, when the mutual information value of cumulative dissolved oxygen amount and cumulative feed amount exceeds 0.5, it is determined that the two have strong correlation. Normalization processing converts the numerical range of cumulative dissolved oxygen amount from 200 mg / L-500 mg / L to 0.0-1.0, eliminating the influence of dimensional difference on neural network training. The sliding window method with a step size of 10 seconds expands the data of 30 minutes into a data matrix of 180 time steps, retaining the dynamic characteristics of the time series. Through the combination of the above steps, the feature data can reflect the time accumulation effect, nonlinear correlation and normalized numerical distribution of the parameters at the same time, providing high-quality input for subsequent hidden state difference calculation.

[0041] In this embodiment, the expansion processing is performed by a sliding window method with a fixed length time step.

[0042] It should be noted that in the sliding window method, a fixed number of time step data is first intercepted according to the preset window length, for example, each window contains 8 continuous time step feature variables; the window slides along the time axis with a fixed step size, for example, it slides 4 time steps each time to form a new data segment; the data in the window is arranged in time sequence to avoid loss of time sequence correlation caused by random segmentation. By fixed step sliding, part of the overlapping data between adjacent windows is retained to ensure the continuity of the time series, so that the hidden state difference calculation can capture the subtle changes between adjacent time steps. The parameter combination of window length and step size is verified by experiments, for example, in the rapid fluctuation stage of dissolved oxygen concentration, a shorter window length is used with a larger step size to balance the calculation efficiency and dynamic feature capture ability. This method improves the regularity of data expansion through structured segmentation, provides input data with time sequence consistency for the subsequent gating unit, and enhances the stability of hidden state difference calculation.

[0043] In this embodiment, in the process of obtaining the difference value of the hidden state in the adjacent time steps of the feature data, the difference value of the hidden state in the adjacent time steps is taken as one of the inputs of the next layer of gating unit.

[0044] It can be understood that the gating unit adopted in the embodiment preferably adopts a recurrent neural network structure with a memory function, and the input dimension thereof includes the difference between the input data at the current time step and the previous hidden state. The difference data is processed through a linear transformation layer and then spliced with the original input vector to participate in the weight calculation of the forget gate and the input gate in the gating unit. This difference input method cooperates with the data preprocessing of the sliding window expansion, and the time span of the difference data is associated with the length of the sliding window. For example, when the length of the sliding window is 5, the difference calculation covers the difference between the hidden states of adjacent two time steps.

[0045] Specifically, during the forward propagation of the recurrent neural network, the difference between the hidden states at each time step is calculated in real time and transmitted to the next layer of gating units. The difference data participates in the calculation of the gating mechanism through an independent weight matrix, so that the gating unit can dynamically adjust the information retention ratio according to the amplitude of the change in the hidden state. When the difference is large, the weight of the forget gate is reduced to retain more historical information, and the weight of the input gate is increased to enhance the fusion of the current change feature. This mechanism is linked with the time-weighted loss function, and the trend of the difference data directly affects the uncertainty quantification of the prediction distribution, thereby improving the sensitivity of the risk signal to the dynamic process. By directly injecting the change amount of the hidden state into the calculation process of the gating unit, the model's ability to capture the mutation characteristics of the time series is enhanced, and the calculation of the risk signal can more timely reflect the abnormal fluctuations of the fermentation parameters.

[0046] In the embodiment, the safety control action obtained according to the auxiliary control model includes the following steps: The partial derivative of the control signal matched with the environmental parameter is calculated by the Newton iteration method, so as to solve the safety control action meeting the minimum work requirement through the auxiliary control model by reverse iteration.

[0047] It can be understood that the Newton iteration method determines the gradient direction of the objective function by calculating the partial derivative of the control signal with respect to the current environmental parameter, and then iteratively updates the control signal until the preset convergence condition is met. During the reverse iteration process, the increment of the control signal is adjusted based on the actual measurement value of the current environmental parameter at each iteration, ensuring that the solution path always meets the dynamic constraints. The minimum work requirement is realized by setting a lower threshold for the objective function, for example, limiting the fluctuation range of the dissolved oxygen concentration to a preset interval, or limiting the adjustment amplitude of the feed flow rate to be less than the maximum execution capability of the equipment.

[0048] Specifically, when the environmental parameter deviates from the safety range, the auxiliary control model constructs a target function according to the measured data of the current dissolved oxygen concentration, pH value and feed flow rate, calculates the partial derivative of the control signal to the target function, and determines the gradient descent direction. The control signal is updated through multiple iterations until the target function value reaches the threshold value corresponding to the minimum working requirement. For example, when the dissolved oxygen concentration is lower than the safety threshold, the model quickly solves the adjustment amount of the feed flow rate through the Newton iteration method, ensuring that the dissolved oxygen concentration is restored to the safety interval within three iterations. This method improves the solving efficiency through mathematical optimization, avoids the time loss of the traditional trial-and-error method, and ensures the accuracy of the control action through gradient calculation, thereby realizing the real-time generation of safe control signals in the dynamically changing fermentation process.

[0049] In this embodiment, the control signal obtained according to the incentive reward comprises the following steps: determining whether the incentive reward is greater than zero; if the incentive reward is greater than zero, selecting the safety control action as the control signal; if the incentive reward is equal to zero, selecting the final control action as the control signal; outputting the control signal to an execution module (a device for regulating environmental parameters) in the fermentation module.

[0050] It should be noted that when the incentive reward is greater than zero, it indicates that the current environmental parameter has a potential abnormality that exceeds the risk threshold, at which time the safety control action is preferred to ensure that the fermentation process is maintained within the minimum working requirement range. For example, when the dissolved oxygen concentration fluctuation causes the prediction residual to exceed the threshold, the safety control action inversely solves the adjustment amount of the feed flow rate through the Newton iteration method to suppress parameter mutation. When the incentive reward is equal to zero, it indicates that the environmental parameter is in a stable state, at which time the final control action generated by the strategy network is adopted, such as the pH value adjustment instruction optimized based on the hidden state and the risk signal, to improve the product synthesis efficiency. The control signal is adjusted by the valve or motor of the execution module, and the execution priority of the safety control action is higher than that of the final control action, thereby balancing the control accuracy and stability under the premise of controllable risk.

[0051] Embodiment 2 In this embodiment, after obtaining the incentive reward according to the intrinsic incentive function, the following step is further included: setting an external reward, and superimposing the incentive reward and the external reward to obtain a composite reward.

[0052] It can be understood that the external reward is derived from the externally set target parameter or operation experience, such as the product concentration threshold, the fermentation efficiency index or the energy consumption constraint; the superimposition method adopts weighted summation or linear combination, and the weight coefficient is dynamically adjusted according to the real-time working condition; the composite reward serves as the feedback signal for updating the strategy network, guiding the control action to approach the external optimization target while meeting the safety constraint.

[0053] Specifically, the operator sets the product synthesis rate as the external reward reference value according to historical data, and the intrinsic incentive function generates a negative incentive reward when the prediction residual exceeds the risk threshold; the external reward and the incentive reward are superimposed according to a preset ratio and input into the policy network, and the generation direction of the control action is adjusted through gradient policy update. For example, when the dissolved oxygen concentration fluctuates, the external reward item in the composite reward guides the feed flow rate to adjust in the direction of maintaining product synthesis, and the incentive reward item suppresses aggressive control actions that exceed the safety boundary, thereby achieving a dynamic balance between safety and economy.

[0054] Embodiment 3 To achieve the above object, the embodiment further provides a control system, which comprises: The acquisition module is configured to acquire environmental parameters in the fermentation module, and preprocess the environmental parameters to obtain feature data; The prediction module is configured to obtain a difference value of hidden states in adjacent time steps of the feature data, and obtain a risk signal of the feature data by quantifying a prediction distribution according to a time weighted loss function; The incentive module is configured to set a risk threshold, and obtain an incentive reward according to an intrinsic incentive function; The control module is configured to establish an auxiliary control model, and obtain a safety control action according to the auxiliary control model; The control module is further configured to obtain a control signal according to the incentive reward, wherein the control signal is a final control action or the safety control action.

[0055] It should be noted that the acquisition module processes the environmental parameters through a sliding window to generate time series feature data, ensuring data dimension consistency. The prediction module inputs the difference value of the hidden states into a gate unit to quantify the risk signal and evaluate the potential deviation of the current control strategy. The incentive module generates a composite reward by dynamically adjusting the risk threshold and combining the intrinsic incentive function, balancing the exploration and safety requirements. The control module establishes an auxiliary control model, and when the risk signal exceeds the threshold, the safety control action based on the Newton iteration method is preferentially selected; when the risk is controllable, the final control action generated by the gradient update of the policy network is adopted. For example, the sliding window length can be set to 5-10 time steps to ensure that the feature data contains sufficient time series information. By matching the risk signal and the safety control action in real time, the system can switch to the safety control mode within 20 milliseconds when the dissolved oxygen concentration mutates or the pH value fluctuates, avoiding microbial growth inhibition.

[0056] As a preferred embodiment, the preferred implementation of the scheme in this embodiment is as follows: the control system is configured to include four functional modules, the acquisition module is connected with the upper computer through the RS485 communication interface by the dissolved oxygen sensor, the pH meter and the flow meter, and the dissolved oxygen concentration, the pH value and the feeding flow rate data in the fermenter are obtained in real time. The data is normalized and pretreated (for example, Z-score standardization or minimum-maximum normalization method), and then the pretreated data is converted into a fixed length time sequence by using the sliding window method, and output to the prediction module after Z-score standardization. The prediction module uses a bidirectional LSTM network to process feature data, calculates the difference value of adjacent time step hidden states, and outputs a risk signal combined with a Gaussian process regression model. The signal includes the standard deviation of the predicted residual and the confidence interval width. The incentive module has a risk threshold comparator, which triggers a negative incentive generated by the built-in reward calculation unit based on the hyperbolic tangent function when the risk signal exceeds the preset threshold. The control module integrates a reinforcement learning strategy network and an auxiliary controller. The former generates a candidate control action according to the hidden state, and the latter solves the control amount that meets the safety range of the dissolved oxygen concentration through the Jacobian matrix. Finally, the arbitration unit selects the safety control action or the final control action according to the positive and negative of the incentive reward.

[0057] Through the above technical scheme, the embodiment realizes real-time dynamic regulation and control of the fermentation process environmental parameters, effectively solves the poor adaptability problem of the traditional control method in the nonlinear time-varying system. Through the risk quantification mechanism of the prediction module, the parameter mutation risk is identified in advance and the safety control strategy is triggered, avoiding the control failure caused by model mismatch. The collaborative work of the incentive module and the control module can ensure that the key parameters of the fermentation process are always within the safety threshold range while maintaining the optimal control performance, significantly improving the robustness of the system under complex working conditions.

[0058] It should be noted that the modules in the control system in this embodiment correspond one by one to the steps in the control method in the foregoing embodiment, therefore, the specific embodiments of this embodiment can refer to the embodiments of the foregoing control method, which will not be repeated here.

[0059] Embodiment 4 In order to make the technical scheme in the present application more clear, the preprocessing process is described in detail here, which includes four core steps: accumulation processing, mutual information screening, normalization processing and sliding window expansion.

[0060] For the operating variables (such as feeding flow rate, pH value, adjusting flow rate) collected in the fermentation module, since the instantaneous value is discrete and the fermentation process has a continuous accumulation effect, it needs to be accumulated in the time dimension to convert it into a cumulative variable reflecting the total amount, the expression satisfies: ; where, is the cumulative variable value at time ; is the instantaneous flow rate at time ; is the time interval between two samplings.

[0061] In the above expressions, the discrete operation variables are converted into continuous cumulative variables by multiplying the discrete instantaneous flow rate by the time interval and summing over the time axis, which enables the operation variables to reflect the parameter variation trend and time accumulation effect, and solves the time dimension mismatching problem between the discrete control instructions and the continuous process response.

[0062] Subsequently, the cumulative variables and the original measurement variables processed through the above steps are used to measure their nonlinear correlation with the difficult-to-measure key target variables (such as the inhibitor concentration or the product concentration) using the mutual information method, and only the variables with mutual information values higher than the preset threshold are retained as the feature variables with correlation, and the expression satisfies: ; where, is the mutual information value between the variable (auxiliary variable) and (target variable); is the joint density function; , is the marginal density function.

[0063] In the above expressions, based on the information entropy theory, the nonlinear dependence between the two variables is quantified, the higher the value, the stronger the correlation between the variables, indicating that the variable is more valuable for subsequent prediction.

[0064] Subsequently, the normalization processing preferably uses the min-max normalization method to map the feature variables of different dimensions to the same numerical range through linear transformation, eliminating the dimensional differences and preventing large numerical range variables from dominating in neural network training.

[0065] For the sliding window method, according to the preset window length (e.g. 8 time steps), a fixed number of continuous data is intercepted as a single input sample, the window slides along the time axis with a fixed step (e.g. 4 time steps each time), forming a new data segment, and part of the overlapping data between adjacent windows is retained to ensure the continuity of the time series.

[0066] Finally, the final feature data is output, which has the structure wherein is the number of samples, is the window length, is the number of feature variables.

[0067] Example 5 In order to make the technical solutions in the present application more clear, the prediction module is described in detail here. It can be understood that the prediction module is embedded with a time-difference recurrent neural network model, which is used to convert the preprocessed feature data into a prediction distribution and a risk signal. The processing process includes: The preprocessed feature data is input into the time-difference recurrent neural network model. It can also be understood that the core of the model is to calculate the difference value of the hidden state of adjacent time steps to capture short-term non-stationary features. The expression satisfies: ; In the formula, is the difference value of the hidden state; is the hidden state of the layer at the moment; is the hidden state of the layer at the moment.

[0068] It can be understood that the difference value is linearly transformed and spliced together with the original input vector to participate in the weight calculation of the forget gate and the input gate in the gate unit. In addition, by calculating the gradient of the hidden state, the short-term dynamic changes and mutation information in the fermentation process are explicitly integrated into the memory unit of the neural network, solving the defect that the traditional LSTM model is easy to filter out short-term fluctuations.

[0069] Subsequently, the model optimizes its parameters through a time-weighted loss function during training (offline phase), so that the error of the prediction distribution output by the model is more sensitive to the later stage of fermentation (key stage). The prediction distribution parameters (mean and standard deviation ) output by the model are the source of the risk signal.

[0070] The expression satisfies: ; In the formula, is the time-weighted composite loss function during model training; is the predicted mean of the prediction output; is the true value, i.e., the historical data of the acquisition module; is the normalized sampling point time of the i-th sample; is the time weighting factor.

[0071] In the above expression, by imposing an exponential penalty on the error of the late fermentation data (which is larger), the model parameters learn a highly sensitive mapping relationship, ensuring that in the stage where the risk is most likely to occur (late fermentation), the predicted value of the model and the risk signal have the highest accuracy and sensitivity.

[0072] Embodiment 6 In order to make the technical solutions in the present application clearer, the control module is described in detail here. It can be understood that the core of the control module is to use the risk signal to drive the generation of the intrinsic incentive function , which converts the uncertainty of the model into the immediate reward of the control system, i.e., the incentive module compares the prediction uncertainty output by the model with the preset critical risk threshold , and once exceeds , the system is determined to be in a high-risk state, triggering an exponential growth of intrinsic incentives , which satisfies the expression: ; wherein, is the incentive reward obtained by the reinforcement learning agent in the current state ; is the preset maximum reward amplitude; is the prediction uncertainty standard deviation, i.e., the standard deviation output by the prediction module; is the critical risk threshold; is the exponential smoothing factor.

[0073] The above expression converts the low confidence of the model into a high-intensity immediate positive reward through an exponential function, which overcomes the delay and sparsity of the external reward of the policy network, and encourages the policy network to actively explore and avoid the high-risk state caused by the increase, i.e., the critical accumulation of the inhibitor.

[0074] In some embodiments, the incentive reward is combined with the external reward ​​The composite reward is obtained by superimposing the economic benefits (based on yield and energy consumption) , the expression satisfies: ; In the formula, is an internal incentive weight coefficient.

[0075] For the auxiliary control model, a multivariate nonlinear polynomial regression model is preferably used, and the control signal required to control the key parameters to the minimum working demand of the safety target value is solved in reverse by Newton iteration , the expression satisfies: ; In the formula, is the safety control action adjusted after the th iteration, for example, the safety ventilation rate; is a multivariate nonlinear polynomial regression model; is a preset safety target state, for example, a safety threshold of dissolved oxygen; is the first-order partial derivative of the polynomial model to the control signal .

[0076] In the above expression, using mathematical optimization means, the control adjustment amount under the minimum working demand (i.e. the lowest energy consumption) is quickly and accurately calculated under the condition of given current environmental parameters and safety target, to prepare for strategy switching under high-risk state.

[0077] For strategy network gradient policy update: The strategy network performs gradient policy update using the strategy target function to maximize the composite reward The expression satisfies: ; Among them, is the final strategy target function; is the strategy clipping term; is the value function loss term; is the strategy entropy term, used to encourage the strategy to maintain a certain exploratory nature.

[0078] Embodiment 7 In some embodiments, the present application also provides a computer storage medium, which has a computer program stored thereon, and the computer program realizes the steps of the method in the foregoing embodiments when executed by a processor.

[0079] In some embodiments, the computer readable storage media can be a memory such as a FRAM, ROM, PROM, EPROM, EEPROM, flash memory, a magnetic surface memory, an optical disk, or a CD-ROM, etc.; and can also be various devices including one or any combination of the above memories. The computer can be various computing devices including a smart terminal and a server.

[0080] In some embodiments, the executable instructions can be in the form of a program, software, software module, script, or code, written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0081] By way of example, the executable instructions can or can not correspond to a file in a file system, can be stored in a part of a file that holds other programs or data, for example, in one or more scripts stored in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files, for example, files that store one or more modules, sub programs, or portions of code.

[0082] By way of example, the executable instructions can be deployed to execute on one computer, or on multiple computers located at one site, or distributed across multiple sites and interconnected by a communication network.

[0083] It should be noted that, in this document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, so that a process, method, article, or system that comprises a list of elements does not include only those elements, but can also include other elements not expressly listed or inherent to such process, method, article, or system. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or system that includes the element.

[0084] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0085] Those skilled in the art can clearly understand the above-mentioned example method can be realized by means of software and the necessary general hardware platform, of course, also can be through hardware, but in many cases the former is a better implementation. Based on such understanding, the technical solutions of the present application essentially or say the part of the prior art to make contributions can be in the form of a software product, the computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disc), including a number of instructions to make a multimedia terminal device (may be a mobile phone, computer, television receiver, or network equipment, etc.) executes the method described in various embodiments of the present application.

[0086] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. Those skilled in the art can easily make various modifications to these embodiments, and apply the general principles described herein to other embodiments without having to go through creative labor. Therefore, the present application is not limited to the above embodiments, and the improvements and modifications made by those skilled in the art without departing from the scope of the present application should be within the scope of protection of the present application.

Claims

1. A fermentation control method based on neural networks, characterized in that, The control method is used to control fermentation parameters within the fermentation module, and the control method includes the following steps: The environmental parameters within the fermentation module are acquired and preprocessed to obtain characteristic data. The difference between the hidden states of the feature data within adjacent time steps is obtained, and the risk signal of the feature data is obtained by quantizing the prediction distribution based on the time-weighted loss function. Set a risk threshold and obtain incentive rewards based on the intrinsic incentive function; Establish an auxiliary control model and obtain safety control actions based on the auxiliary control model; Based on the hidden state and risk signals of the feature data, the final control action is obtained through gradient policy update of the policy network; The control signal is obtained based on the incentive reward; the control signal is the final control action or the safety control action. The risk signals include standard deviation, prediction uncertainty, and prediction residual.

2. The fermentation control method based on a neural network according to claim 1, characterized in that, The preprocessing of environmental parameters includes the following steps: The acquired environmental parameters are accumulated to convert them into cumulative variables; The nonlinear correlation between cumulative variables is obtained using the mutual information method to identify the characteristic variables with correlation. The feature variables are normalized, and then the normalized data is expanded to obtain the feature data. The environmental parameters include dissolved oxygen concentration, pH value, and feed flow rate.

3. The fermentation control method based on a neural network according to claim 1, characterized in that, The data expansion process employs a sliding window method with a fixed time step.

4. The fermentation control method based on a neural network according to claim 1, characterized in that, In the process of obtaining the difference between the hidden states of adjacent time steps of the feature data, the difference between the hidden states of adjacent time steps is used as one of the inputs of the next layer gating unit.

5. The fermentation control method based on a neural network according to claim 1, characterized in that, After obtaining the incentive reward based on the intrinsic incentive function, the following steps are also included: Set up external rewards, and combine incentive rewards with external rewards to obtain compound rewards.

6. The fermentation control method based on a neural network according to claim 1, characterized in that, The process of obtaining safety control actions based on the auxiliary control model includes the following steps: By using Newton's iteration method to calculate the partial derivative of the control signal that matches the environmental parameters, the safety control action that meets the minimum working requirements can be solved by back-iteration through the auxiliary control model.

7. The fermentation control method based on a neural network according to claim 6, characterized in that, The process of obtaining control signals based on incentive rewards includes the following steps: Determine whether the incentive reward is greater than zero; if the incentive reward is greater than zero, select the safety control action as the control signal; if the incentive reward is equal to zero, select the final control action as the control signal. The control signal is output to the execution module within the fermentation module.

8. A control system for implementing the method as described in any one of claims 1-7, characterized in that, The control system includes: The acquisition module is used to acquire environmental parameters within the fermentation module and preprocess the environmental parameters to obtain feature data. The prediction module is used to obtain the difference of the hidden state of the feature data in adjacent time steps, and to obtain the risk signal of the feature data by prediction distribution quantization according to the time-weighted loss function. The incentive module is used to set a risk threshold and obtain incentive rewards based on the inherent incentive function. The control module is used to establish an auxiliary control model and obtain safety control actions based on the auxiliary control model. The control module is also used to obtain control signals based on incentive rewards, wherein the control signals are final control actions or safety control actions.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

Images