Dynamic control method of low-temperature fermentation of condiments based on reinforcement learning

Abstract

The present invention discloses a method for dynamic control of low-temperature fermentation of condiments based on reinforcement learning, comprising the following steps: S1, receiving a flavor target input and converting it into a metabolic intent vector; S2, constructing a target metabolic pathway map based on the metabolic intent vector; S3, collecting current fermentation state data; S4, constructing the current metabolic pathway map and mapping it to a non-Euclidean embedding space to obtain an embedded representation of the current metabolic pathway; S5, mapping the target metabolic pathway map to the same non-Euclidean embedding space to obtain an embedded representation of the target metabolic pathway, and comparing it with the current metabolic pathway embedded representation to calculate the structural deviation; S6, generating a control strategy output based on the structural deviation using a Riemannian manifold strategy optimization method; S7, converting the control strategy output into a fermentation process control instruction; and S8, executing the control instruction. The present invention integrates flavor semantic parsing with Riemannian manifold reinforcement learning to achieve precise fermentation control driven by metabolic pathway structure.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of fermentation processes, and in particular to a dynamic control method for low-temperature fermentation of condiments based on reinforcement learning. Background Art

[0002] Condiments are an indispensable component of the food industry, and their flavor quality is influenced by multiple factors, including raw materials, processing technology, and fermentation process control. This is particularly true for low-temperature fermented condiments based on natural microbial fermentation, such as soy sauce, dressings, and fermented vinegar. Microbial metabolic activity is complex, takes a long time, and is significantly affected by environmental disturbances. Therefore, the ability to dynamically adjust fermentation environmental parameters becomes a crucial factor influencing the flavor of the end product.

[0003] Traditional methods for controlling the fermentation process of condiments are mostly based on empirical rules or static control logic, relying mainly on the process personnel's experience in setting process parameters such as temperature, humidity, ventilation, and stirring, supplemented by regular sampling and analysis methods to monitor the fermentation process. When faced with different batches of raw materials, environmental conditions, or variations in microbial communities, such methods lack the ability to respond to changes in the intrinsic structure of the metabolic process, making it difficult to achieve precise dynamic control. In addition, the more advanced methods in the existing technology also mainly focus on establishing physical and chemical sensor networks or applying data-driven shallow optimization models, such as fuzzy control and genetic algorithm parameter adjustment, and have not yet broken through the complete intelligent control goal of structure perception-strategy generation-environmental intervention.

[0004] In recent years, with the widespread application of artificial intelligence technology, some studies have attempted to introduce machine learning methods into fermentation process modeling and prediction, such as using neural networks to fit the mapping relationship between environmental parameters and metabolite concentrations to assist process regulation. However, such methods generally have problems such as strong dependence on specific samples, insufficient generalization ability, and inability to explain the structural evolution of metabolic processes. In addition, most models only focus on result prediction and lack the ability to generate control strategies based on metabolic pathway mechanisms. In addition, traditional deep reinforcement learning algorithms have defects such as low convergence efficiency and large search direction deviation in dealing with complex graph structures, path topologies, and dynamic strategy optimization in non-Euclidean spaces. They are difficult to directly apply to industrial biological processes such as condiment fermentation, which are nonlinear, strongly coupled, and highly dynamic in structure.

[0005] To address these issues, the existing technology has yet to provide a comprehensive intelligent fermentation control method that simultaneously integrates semantic input of flavor targets, representation of metabolic pathway structures, quantification of differences in embedding space, and output of structural perturbation control. In particular, the concept of path-guided control, which establishes an embedding representation in the metabolic structure space and then optimizes strategies and conducts intervention outputs within this space, has yet to be disclosed in the public literature or known patents. The existing technology generally overlooks the fact that microbial metabolic activity during fermentation is essentially a process of structural dynamic evolution. The core issue lies not simply in parameter control itself, but in whether strategic intervention can be implemented based on the structural representation of the metabolic mechanism.

[0006] Therefore, how to provide a dynamic control method for low-temperature fermentation of condiments based on reinforcement learning is an urgent problem that those skilled in the art need to solve. Summary of the Invention

[0007] One objective of the present invention is to propose a reinforcement learning-based method for dynamic control of low-temperature condiment fermentation. This method integrates flavor semantic parsing, metabolic pathway mapping, and Riemannian manifold reinforcement learning to implement a dynamic control mechanism driven by metabolic pathway structure during low-temperature condiment fermentation. By sensing the difference between the current and target metabolic structures in a non-Euclidean embedding space, outputting metabolic perturbation directions and mapping them into environmental control instructions, this system constructs an intelligent perception-strategy optimization-environmental intervention control system, offering the advantages of high intelligence, precise control, responsiveness, and high flavor consistency.

[0008] The method for dynamic control of low-temperature fermentation of condiments based on reinforcement learning according to an embodiment of the present invention includes the following steps:

[0009] S1. Receive a flavor target input, wherein the flavor target is a text description of a condiment, and convert the flavor target into a metabolic intent vector;

[0010] S2. Constructing a target metabolic pathway map based on the metabolic intention vector, wherein the target metabolic pathway map is composed of metabolic pathway nodes and connection relationships, and is used to represent the metabolic structure pathway corresponding to flavor formation;

[0011] S3. Periodically collecting current fermentation status data at set time intervals during the fermentation process;

[0012] S4. Constructing a current metabolic pathway map based on the current fermentation state data, and mapping the current metabolic pathway map to a non-Euclidean embedding space to obtain an embedded representation of the current metabolic pathway;

[0013] S5. Mapping the target metabolic pathway map to the same non-Euclidean embedding space as the current metabolic pathway map to obtain an embedded representation of the target metabolic pathway, and comparing the embedded representation of the target metabolic pathway with the embedded representation of the current metabolic pathway to calculate a structural deviation;

[0014] S6. Performing strategy training and updating using a Riemannian manifold strategy optimization method based on the structural deviation to generate a control strategy output in a non-Euclidean embedding space, wherein the control strategy output is represented by a metabolic pathway perturbation direction;

[0015] S7, converting the control strategy output into fermentation process control instructions, wherein the fermentation process control instructions include a temperature setting value, a humidity setting value, a ventilation rate, and a stirring frequency;

[0016] S8. Execute the control instructions to adjust the fermentation environment parameters and complete the regulation of the evolution direction of the metabolic pathway during the fermentation process.

[0017] Optionally, the S1 specifically includes:

[0018] S11. Receive text description information of a condiment input by a user, wherein the text description information of the condiment includes natural language expression of flavor, aroma, taste, process type, and regional style;

[0019] S12, performing word segmentation processing on the condiment text description information to extract keywords with sensory characteristic meanings;

[0020] S13, matching the keyword with a preset flavor-metabolite mapping dictionary, where the flavor-metabolite mapping dictionary records the association between flavor terms and specific metabolites, to obtain one or more metabolite names associated with the keyword;

[0021] S14. Retrieve the metabolic pathway number for each metabolite name by querying a locally established metabolic pathway information table, and generate a metabolic pathway index list. The metabolic pathway information table is a pre-organized structured data file containing metabolites and corresponding metabolic pathway numbers, starting substrates, intermediate metabolites, end products, and reaction sequences;

[0022] S15. Based on the metabolic pathway index list, obtain the association frequency between the metabolic pathway and the flavor target, use the association frequency ratio as the importance weight value of each metabolic pathway, and construct a metabolic intention vector. The metabolic intention vector is a multidimensional numerical sequence used to represent the importance of different metabolic pathways, where each dimension value is obtained by normalizing the association frequency of the corresponding metabolic pathway.

[0023] Optionally, the S2 specifically includes:

[0024] S21, sorting the metabolic pathways according to the importance weight values ​​in the metabolic intention vector, and selecting metabolic pathways with importance weight values ​​higher than a set weight threshold as the target metabolic pathway set;

[0025] S22, searching for structural information corresponding to the target metabolic pathway set, where the structural information includes the starting substrate, intermediate metabolites, end products, and reaction sequence involved in the target metabolic pathway;

[0026] S23, extracting metabolite entries from the structural information and establishing a target metabolic pathway node set, where each node corresponds to a metabolite and is accompanied by a position identifier in the metabolic pathway;

[0027] S24. According to the sequential reaction relationship of each metabolite in the metabolic pathway, a directed connection relationship between the target metabolic pathway nodes is constructed to form a structured metabolic pathway dependency chain, and the target metabolic pathway nodes and connection relationships constitute a target metabolic pathway map.

[0028] Optionally, the current fermentation state data specifically include temperature, humidity, pH value, oxygen concentration, carbon dioxide concentration and metabolite concentration.

[0029] Optionally, the S4 specifically includes:

[0030] S41, performing a difference operation on the metabolite concentrations at adjacent sampling time points, calculating the rate of change of the concentration of each metabolite, and marking the metabolite with a change rate higher than a preset rate threshold as an active metabolite;

[0031] S42. Constructing a current metabolic pathway node set based on the corresponding relationship of active metabolites in the locally established metabolic pathway information table;

[0032] S43. Establishing directed connections between nodes in the current metabolic pathway based on the reaction sequence and transformation direction of the active metabolites in the metabolic pathway, and constructing a current metabolic pathway map;

[0033] S44, calculating the path distance between the nodes of the current metabolic pathway based on the current metabolic pathway map, and generating a distance matrix, wherein an element of the distance matrix is ​​the shortest reaction step length between two nodes in the metabolic pathway;

[0034] S45. Process the distance matrix using Laplace eigenmapping, and map the current metabolic pathway map to a non-Euclidean embedding space by retaining the relative positional relationship and topological structure characteristics between the nodes of the current metabolic pathway. The non-Euclidean embedding space is a structured representation space that does not satisfy Euclidean symmetry and the Pythagorean theorem.

[0035] Optionally, the calculation process of the structural deviation specifically includes:

[0036] Pair the target metabolic pathway embedding representation with the nodes with the same metabolite identifier in the current metabolic pathway embedding representation in a one-to-one correspondence to form a node pair set;

[0037] For each pair of paired nodes, extract the embedding vector coordinates in the non-Euclidean embedding space and calculate the embedding distance offset value of the paired nodes. The embedding distance offset value is calculated using the Euclidean distance method and is defined as the square root of the sum of the squares of the coordinate differences in each dimension between the embedding vectors of the two nodes.

[0038] Calculate the node weighting factors of the two nodes in each pair of paired nodes respectively, and the node weighting factors are obtained by linear weighting according to the importance weight value of the node in the metabolic pathway and the local topological connectivity in the corresponding metabolic pathway map;

[0039] Taking the arithmetic average of the node weighting factors of the two nodes in the paired node pair as the final weighting factor of the paired node, and multiplying the distance offset value of the paired node by the corresponding final weighting factor to obtain a weighted offset value;

[0040] The weighted offset values ​​of each pair of paired nodes are summed and divided by the sum of the weighted factors of all node pairs to obtain a structural deviation value, which is used to quantify the overall structural difference between the current metabolic pathway map and the target metabolic pathway map in the non-Euclidean embedding space.

[0041] Optionally, the S6 specifically includes:

[0042] S61, using the structural deviation value as the state input basis, and combining it with the current metabolic pathway embedding representation, constructing a state vector in the non-Euclidean embedding space ;

[0043] S62, based on the state vector , build a policy function ,in Indicates that in a given state vector Down output action The conditional probability distribution of Represents the policy function parameters, Indicates that at time step Output metabolic pathway perturbation direction vector;

[0044] S63. Define the immediate reward function to represent the effect of structural optimization:

[0045] ;

[0046] in, Represents the time step The reward value when Represents the time step The structural deviation value when Represents the time step The structural deviation value when ;

[0047] S64. Construct Fisher information matrix based on current policy function :

[0048] ;

[0049] in, represents the expectation operator, represents the gradient of the policy logarithmic function with respect to the parameters, Represents a transpose operation;

[0050] S65. Update the strategy parameters using the natural gradient optimization method on the Riemannian manifold:

[0051] ;

[0052] in, Represents the time step The strategy parameters when Represents the time step The strategy parameters when represents the learning rate, Represents the time step The inverse of the Fisher information matrix is represents the discount factor, Represents the time step The reward value when Represents the gradient of the strategy's cumulative expected reward with respect to the strategy parameters;

[0053] S66. Apply the updated strategy parameters to the current state vector to generate a metabolic pathway perturbation direction vector, wherein the metabolic pathway perturbation direction vector is defined in a non-Euclidean embedding space and is used to represent the adjustment direction of the metabolic pathway map in the structural space as a control strategy output.

[0054] Optionally, the S7 specifically includes:

[0055] S71. Extracting the disturbance direction components on each principal component axis in the non-Euclidean embedding space according to the metabolic pathway disturbance direction vector output by the control strategy, and mapping the disturbance direction components to the fermentation process parameter adjustment direction and adjustment amplitude according to the sign and amplitude of the disturbance direction components;

[0056] S72. Setting a mapping rule between the disturbance direction component and each control parameter, wherein: the temperature setting value is positively correlated with the first principal component axis component in the disturbance direction component, the humidity setting value is positively correlated with the second principal component axis component in the disturbance direction component, the ventilation rate is positively correlated with the third principal component axis component in the disturbance direction component, and the stirring frequency is positively correlated with the fourth principal component axis component in the disturbance direction component;

[0057] S73. Based on the value range of the disturbance direction component of the metabolic pathway disturbance direction vector in the non-Euclidean embedding space, a hierarchical adjustment rule for the fermentation process parameters is constructed, wherein: when the absolute value of the disturbance direction component is less than 0.1, it is weak adjustment, and the adjustment range is set to ±2% of the original set value; when the absolute value of the disturbance direction component is between 0.1 and 0.3, it is medium adjustment, and the adjustment range is ±5%; when the absolute value of the disturbance direction component is greater than 0.3, it is strong adjustment, and the adjustment range is ±10%;

[0058] S74. When the disturbance direction component is positive, the corresponding fermentation process parameter is increased by a set amplitude; when the disturbance component is negative, the corresponding fermentation process parameter is decreased by a set amplitude, and finally a fermentation process control instruction is generated, wherein the fermentation process control instruction includes a temperature set value, a humidity set value, a ventilation rate, and a stirring frequency;

[0059] S75 , encapsulating the fermentation process control instruction into a fermentation control device-level parameter configuration instruction, and sending the instruction to the fermentation control device control execution unit to complete the dynamic adjustment of the fermentation environment.

[0060] The beneficial effects of the present invention are:

[0061] First, starting from the textual information of flavor targets, this invention establishes semantic associations between flavor terms and metabolites, and then builds a metabolic intent vector based on a local metabolic pathway knowledge table. This achieves a target mapping from sensory needs to the metabolic pathway space. This transformation from natural language description to metabolic structure abstraction overcomes the limitation of traditional control systems that can only be set based on physical or chemical indicators, providing a foundation for personalized flavor control.

[0062] Secondly, by capturing changes in metabolite concentrations in real time during fermentation, constructing a pathway map reflecting the current microbial metabolic state, and then embedding this map into a non-Euclidean space to represent the structural state, the present invention achieves dynamic structural perception of metabolic evolution. This representation preserves the topological relationships and reaction sequence information of the metabolic pathway map, significantly improving the ability to capture microscopic changes in complex metabolic processes and providing a unified and stable representation foundation for subsequent quantification of structural deviations.

[0063] Furthermore, the present invention introduces a Riemannian manifold natural gradient optimization method to train and update a structural bias-driven control policy in a non-Euclidean embedding space. Compared to traditional gradient policy optimization, this method enables more natural updates based on the geometric structure of the parameter space, improving the efficiency and convergence stability of policy learning. The resulting policy output is a metabolic pathway perturbation direction vector, which directly corresponds to the evolutionary direction in the structural space, offering the advantages of strong interpretability and controllable behavior.

[0064] Finally, the present invention establishes a mapping mechanism between disturbance vectors and fermentation process parameters, achieving a shift from structural to physical control and establishing a closed-loop regulation of the fermentation environment driven by metabolic mechanisms. By converting the components of the disturbance direction along different principal component axes into adjustment instructions for parameters such as temperature, humidity, ventilation rate, and stirring frequency, this effectively connects the guidance of metabolic pathway structural evolution with the execution level of the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0066] Figure 1 This is the overall flow chart of the dynamic control method for low-temperature fermentation of condiments based on reinforcement learning proposed by the present invention;

[0067] Figure 2 This is a flow chart of the structural deviation calculation of the dynamic control method for low-temperature fermentation of condiments based on reinforcement learning proposed by the present invention;

[0068] Figure 3 This is a flow chart of the control strategy training process for Riemannian manifold strategy optimization based on structural deviation in the reinforcement learning-based dynamic control method for low-temperature fermentation of condiments proposed in the present invention. DETAILED DESCRIPTION

[0069] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0070] refer to Figure 1-3 The dynamic control method of low-temperature fermentation of condiments based on reinforcement learning includes the following steps:

[0071] S1. Receive a flavor target input, wherein the flavor target is a text description of a condiment, and convert the flavor target into a metabolic intent vector;

[0072] S2. Constructing a target metabolic pathway map based on the metabolic intention vector, wherein the target metabolic pathway map is composed of metabolic pathway nodes and connection relationships, and is used to represent the metabolic structure pathway corresponding to flavor formation;

[0073] S3. Periodically collecting current fermentation status data at set time intervals during the fermentation process;

[0074] S4. Constructing a current metabolic pathway map based on the current fermentation state data, and mapping the current metabolic pathway map to a non-Euclidean embedding space to obtain an embedded representation of the current metabolic pathway;

[0075] S5. Mapping the target metabolic pathway map to the same non-Euclidean embedding space as the current metabolic pathway map to obtain an embedded representation of the target metabolic pathway, and comparing the embedded representation of the target metabolic pathway with the embedded representation of the current metabolic pathway to calculate a structural deviation;

[0076] S6. Performing strategy training and updating using a Riemannian manifold strategy optimization method based on the structural deviation to generate a control strategy output in a non-Euclidean embedding space, wherein the control strategy output is represented by a metabolic pathway perturbation direction;

[0077] S7, converting the control strategy output into fermentation process control instructions, wherein the fermentation process control instructions include a temperature setting value, a humidity setting value, a ventilation rate, and a stirring frequency;

[0078] S8. Execute the control instructions to adjust the fermentation environment parameters and complete the regulation of the evolution direction of the metabolic pathway during the fermentation process.

[0079] The present invention proposes a dynamic control mechanism for low-temperature fermentation of condiments based on reinforcement learning, and establishes a complete technical process around multiple key steps including flavor target driving, metabolic pathway structure modeling, non-Euclidean space representation, structural deviation calculation and strategy optimization. By introducing the linkage between flavor semantic analysis and metabolic maps, the fuzzy sensory targets are converted into quantifiable metabolic intention expressions, and a structured metabolic pathway target map is constructed. The evolution of metabolic structure is dynamically represented using non-Euclidean embedding space, and optimal control of the disturbance direction is achieved within this space, thereby opening up the path between flavor targets, microstructures and fermentation processes. Finally, the control strategy output is converted into executable process parameter adjustment instructions, which significantly improves the intelligence level and control accuracy of the fermentation process.

[0080] In this embodiment, S1 specifically includes:

[0081] S11. Receive text description information of a condiment input by a user, wherein the text description information of the condiment includes natural language expression of flavor, aroma, taste, process type, and regional style;

[0082] S12, performing word segmentation processing on the condiment text description information to extract keywords with sensory characteristic meanings;

[0083] S13, matching the keyword with a preset flavor-metabolite mapping dictionary, where the flavor-metabolite mapping dictionary records the association between flavor terms and specific metabolites, to obtain one or more metabolite names associated with the keyword;

[0084] S14. Retrieve the metabolic pathway number for each metabolite name by querying a locally established metabolic pathway information table, and generate a metabolic pathway index list. The metabolic pathway information table is a pre-organized structured data file containing metabolites and corresponding metabolic pathway numbers, starting substrates, intermediate metabolites, end products, and reaction sequences;

[0085] S15. Based on the metabolic pathway index list, obtain the association frequency between the metabolic pathway and the flavor target, use the association frequency ratio as the importance weight value of each metabolic pathway, and construct a metabolic intention vector. The metabolic intention vector is a multidimensional numerical sequence used to represent the importance of different metabolic pathways, where each dimension value is obtained by normalizing the association frequency of the corresponding metabolic pathway.

[0086] By parsing flavor target information of condiments from text descriptions, combined with keyword segmentation, flavor-metabolite mapping dictionary matching, and metabolic pathway information query, the conversion from sensory language to metabolic structure level is effectively achieved. Vocabulary such as aroma and taste in natural language are mapped to specific metabolites and further associated with known metabolic pathway numbers to form a metabolic pathway index list, providing structured input for subsequent map construction and control strategy output. Metabolic intent vectors are constructed by normalizing the frequency of occurrence between flavor targets and pathways. This not only enhances the interpretability of target guidance but also provides a quantifiable semantic basis for subsequent structural deviation measurement in non-Euclidean space, improving the system's ability to respond to diverse flavor targets.

[0087] In this embodiment, S2 specifically includes:

[0088] S21, sorting the metabolic pathways according to the importance weight values ​​in the metabolic intention vector, and selecting metabolic pathways with importance weight values ​​higher than a set weight threshold as the target metabolic pathway set;

[0089] S22, searching for structural information corresponding to the target metabolic pathway set, where the structural information includes the starting substrate, intermediate metabolites, end products, and reaction sequence involved in the target metabolic pathway;

[0090] S23, extracting metabolite entries from the structural information and establishing a target metabolic pathway node set, where each node corresponds to a metabolite and is accompanied by a position identifier in the metabolic pathway;

[0091] S24. According to the sequential reaction relationship of each metabolite in the metabolic pathway, a directed connection relationship between the target metabolic pathway nodes is constructed to form a structured metabolic pathway dependency chain, and the target metabolic pathway nodes and connection relationships constitute a target metabolic pathway map.

[0092] This section implements the specific construction from metabolic intent vectors to structural maps. It screens highly relevant metabolic pathways based on path weights, extracts their reaction sequence and material composition information, and constructs a metabolic pathway map structure oriented towards the target flavor. By establishing node sets and directed connection relationships, it not only restores the microscopic reaction process in the pathway but also lays the foundation for subsequent graph embedding representation. This graph expression method has good topological structural integrity and biological rationality, can truly reflect the metabolic evolution path during the flavor formation process, and provides a unified format for comparative analysis with real-time state maps, making target pathway control more targeted and scientifically based.

[0093] In this embodiment, the current fermentation state data specifically includes temperature, humidity, pH value, oxygen concentration, carbon dioxide concentration and metabolite concentration.

[0094] In this embodiment, the S4 specifically includes:

[0095] S41, performing a difference operation on the metabolite concentrations at adjacent sampling time points, calculating the rate of change of the concentration of each metabolite, and marking the metabolite with a change rate higher than a preset rate threshold as an active metabolite;

[0096] S42. Constructing a current metabolic pathway node set based on the corresponding relationship of active metabolites in the locally established metabolic pathway information table;

[0097] S43. Establishing directed connections between nodes in the current metabolic pathway based on the reaction sequence and transformation direction of the active metabolites in the metabolic pathway, and constructing a current metabolic pathway map;

[0098] S44, calculating the path distance between the nodes of the current metabolic pathway based on the current metabolic pathway map, and generating a distance matrix, wherein an element of the distance matrix is ​​the shortest reaction step length between two nodes in the metabolic pathway;

[0099] S45. Process the distance matrix using Laplace eigenmapping, and map the current metabolic pathway map to a non-Euclidean embedding space by retaining the relative positional relationship and topological structure characteristics between the nodes of the current metabolic pathway. The non-Euclidean embedding space is a structured representation space that does not satisfy Euclidean symmetry and the Pythagorean theorem.

[0100] By combining the rate of metabolite change, active metabolites are identified and a current metabolic pathway map is constructed. This map is then mapped to a non-Euclidean embedding space, completing the dynamic representation of the structural state. This process preserves the graph's topological features and the logic of inter-node reactions, improving the accuracy of structural comparison and model generalization. The use of Laplacian eigenmaps further enhances the consistency and robustness of graph embeddings, providing a spatial foundation for subsequent structural deviation quantification and strategy training, achieving an effective transition from data perception to structural expression.

[0101] In this embodiment, the calculation process of the structural deviation specifically includes:

[0102] Pair the target metabolic pathway embedding representation with the nodes with the same metabolite identifier in the current metabolic pathway embedding representation in a one-to-one correspondence to form a node pair set;

[0103] For each pair of paired nodes, extract the embedding vector coordinates in the non-Euclidean embedding space and calculate the embedding distance offset value of the paired nodes. The embedding distance offset value is calculated using the Euclidean distance method and is defined as the square root of the sum of the squares of the coordinate differences in each dimension between the embedding vectors of the two nodes.

[0104] Calculate the node weighting factors of the two nodes in each pair of paired nodes respectively, and the node weighting factors are obtained by linear weighting according to the importance weight value of the node in the metabolic pathway and the local topological connectivity in the corresponding metabolic pathway map;

[0105] Taking the arithmetic average of the node weighting factors of the two nodes in the paired node pair as the final weighting factor of the paired node, and multiplying the distance offset value of the paired node by the corresponding final weighting factor to obtain a weighted offset value;

[0106] The weighted offset values ​​of each pair of paired nodes are summed and divided by the sum of the weighted factors of all node pairs to obtain a structural deviation value, which is used to quantify the overall structural difference between the current metabolic pathway map and the target metabolic pathway map in the non-Euclidean embedding space.

[0107] A structural deviation quantification mechanism is proposed to accurately measure the structural differences between the target metabolic pathway embedding and the current metabolic pathway embedding. By constructing a set of node pairs, calculating the embedding distance, and combining a weighting mechanism based on pathway weights and connectivity, a normalized structural difference value is ultimately generated. This approach not only improves sensitivity to metabolic deviation trends but also provides directional and explanatory state inputs for policy training, enabling continuous iterative optimization of the control strategy and convergence to the target structure.

[0108] In this embodiment, S6 specifically includes:

[0109] S61, using the structural deviation value as the state input basis, and combining it with the current metabolic pathway embedding representation, constructing a state vector in the non-Euclidean embedding space ;

[0110] S62, based on the state vector , build a policy function ,in Indicates that in a given state vector Down output action The conditional probability distribution of Represents the policy function parameters, Indicates that at time step Output metabolic pathway perturbation direction vector;

[0111] S63. Define the immediate reward function to represent the effect of structural optimization:

[0112] ;

[0113] in, Represents the time step The reward value when Represents the time step The structural deviation value when Represents the time step The structural deviation value when ;

[0114] S64. Construct Fisher information matrix based on current policy function :

[0115] ;

[0116] in, represents the expectation operator, represents the gradient of the policy logarithmic function with respect to the parameters, Represents a transpose operation;

[0117] S65. Update the strategy parameters using the natural gradient optimization method on the Riemannian manifold:

[0118] ;

[0119] in, Represents the time step The strategy parameters when Represents the time step The strategy parameters when represents the learning rate, Represents the time step The inverse of the Fisher information matrix is represents the discount factor, Represents the time step The reward value when represents the gradient of the strategy's cumulative expected reward with respect to the strategy parameters;

[0120] S66. Apply the updated strategy parameters to the current state vector to generate a metabolic pathway perturbation direction vector, wherein the metabolic pathway perturbation direction vector is defined in a non-Euclidean embedding space and is used to represent the adjustment direction of the metabolic pathway map in the structural space as a control strategy output.

[0121] A Riemannian manifold policy optimization method was introduced to train and update control policies in a non-Euclidean structural space. By constructing a state vector, defining a reward function, constructing a Fisher information matrix, and using natural gradients for parameter iteration, efficient optimization of the policy function in the structural space was achieved. The control policy output, in the form of a metabolic pathway perturbation direction vector, can precisely guide the evolution of the metabolic pathway map along the structurally optimal direction, thereby enabling behavioral intervention at the map level, significantly improving the control's interpretability, responsiveness, and adaptability to biological mechanisms.

[0122] The Fisher information matrix is ​​a symmetric positive definite matrix used to describe the sensitivity of policy outputs to parameter changes. In the present invention, this matrix is ​​used to capture the changing trends of the policy function in non-Euclidean space and is an important basis for measuring the intensity of policy changes in different directions. This matrix allows the selection of the most natural and effective update direction in parameter space, making the control strategy more consistent with the actual structural characteristics of metabolic pathway evolution, thereby improving training convergence speed and control accuracy.

[0123] In this embodiment, the S7 specifically includes:

[0124] S71. Extracting the disturbance direction components on each principal component axis in the non-Euclidean embedding space according to the metabolic pathway disturbance direction vector output by the control strategy, and mapping the disturbance direction components to the fermentation process parameter adjustment direction and adjustment amplitude according to the sign and amplitude of the disturbance direction components;

[0125] S72. Setting a mapping rule between the disturbance direction component and each control parameter, wherein: the temperature setting value is positively correlated with the first principal component axis component in the disturbance direction component, the humidity setting value is positively correlated with the second principal component axis component in the disturbance direction component, the ventilation rate is positively correlated with the third principal component axis component in the disturbance direction component, and the stirring frequency is positively correlated with the fourth principal component axis component in the disturbance direction component;

[0126] S73. Based on the value range of the disturbance direction component of the metabolic pathway disturbance direction vector in the non-Euclidean embedding space, a hierarchical adjustment rule for the fermentation process parameters is constructed, wherein: when the absolute value of the disturbance direction component is less than 0.1, it is weak adjustment, and the adjustment range is set to ±2% of the original set value; when the absolute value of the disturbance direction component is between 0.1 and 0.3, it is medium adjustment, and the adjustment range is ±5%; when the absolute value of the disturbance direction component is greater than 0.3, it is strong adjustment, and the adjustment range is ±10%;

[0127] S74. When the disturbance direction component is positive, the corresponding fermentation process parameter is increased by a set amplitude; when the disturbance component is negative, the corresponding fermentation process parameter is decreased by a set amplitude, and finally a fermentation process control instruction is generated, wherein the fermentation process control instruction includes a temperature set value, a humidity set value, a ventilation rate, and a stirring frequency;

[0128] S75 , encapsulating the fermentation process control instruction into a fermentation control device-level parameter configuration instruction, and sending the instruction to the fermentation control device control execution unit to complete the dynamic adjustment of the fermentation environment.

[0129] By mapping metabolic pathway perturbation direction vectors in a non-Euclidean embedding space into executable fermentation control instructions, a closed-loop conversion loop from structural strategies to environmental interventions is achieved. By establishing association rules between perturbation components and temperature, humidity, ventilation, and agitation, and setting multi-level adjustment intensity standards, the strategy outputs are accurately converted into control signals recognizable by the equipment, enabling automatic and progressive adjustment of the fermentation environment, improving process stability and flavor consistency.

[0130] Example 1

[0131] To verify the feasibility of this invention, it was applied to the low-temperature fermentation process of a traditional condiment manufacturer with an annual production capacity exceeding 20,000 tons. This company, which has long produced sauce-flavored and complex-flavor condiments, faced a significant quality control challenge. Due to significant differences in microbial metabolic states between batches, fermentation flavor exhibited significant fluctuations. Even with consistent raw materials and stable control parameters, the flavor sensory experience could still differ significantly from expectations, posing a challenge to product consistency.

[0132] Especially under low-temperature fermentation conditions, the company operates within a temperature control range of 15°C to 30°C, aiming to prolong the metabolic cycle and enhance aroma refinement. However, in low-temperature environments, microbial metabolic rates generally decrease, inhibiting enzymatic activity in some pathways, which can easily lead to insufficient production of key flavor compounds. Furthermore, traditional constant control strategies centered around temperature and humidity are sluggish in low-temperature environments, making it difficult to compensate for metabolic shifts in a timely manner, further exacerbating flavor instability.

[0133] To address these challenges, the reinforcement learning-based dynamic control method for low-temperature condiment fermentation proposed in this paper was applied to the company's low-temperature fermentation experiment with a sauce-flavored, five-flavor fusion condiment. First, five experienced flavorists provided a textual description of the condiment, describing it as having a rich sauce aroma, a distinct sweet aftertaste, a slightly sour and fruity aroma, and low spiciness. The system then used a flavor analysis module to extract keywords, construct a metabolic intent vector, and generate 18 target metabolic pathways, focusing on the synthesis pathways of phenylacetate, butyl acetate, glutamate, and medium-chain fatty acids.

[0134] In this embodiment, the experimental group uses two 1500L low-temperature fermentation tanks connected in parallel, which are started synchronously at a set initial temperature of 20°C. One group is a traditional static control control group, and the other group is connected to the intelligent control system of the present invention. The control group uses fixed parameters: a constant temperature of 28°C, a humidity of 65%, a ventilation rate of 3 times per hour, and a stirring cycle of once every 6 hours. The experimental group system updates the state map once an hour based on the real-time collected state information such as temperature and humidity, pH value, oxygen / carbon dioxide concentration and metabolite concentration, and compares it with the target path map through non-Euclidean embedding representation, outputs the structural deviation value, and drives the Riemann manifold strategy optimization process.

[0135] During the entire 72-hour low-temperature fermentation cycle, the system completed 114 control strategy updates, automatically adjusting temperature, ventilation, or stirring frequency approximately 1.6 times per hour on average, achieving fully autonomous dynamic control. Flavor sample analysis revealed that the concentration of ethyl phenylacetate in the test group reached 24.6 mg / L (compared to 20.1 mg / L in the control group), butyl acetate reached 16.3 mg / L (compared to 13.7 mg / L in the control group), and glutamate increased to 89.2 mg / L (compared to 72.5 mg / L in the control group). Sensory evaluation showed that the intensity of the sauce aroma increased from 7.6 to 9.1, the persistence of the aftertaste increased by 1.5 points, and the standard deviation of the overall flavor consistency score decreased from 0.89 to 0.35.

[0136] Under low-temperature conditions, the system specifically addresses the issue of insufficient fatty acid pathway activation caused by temperature inhibition by automatically increasing stirring frequency and fine-tuning ventilation to enhance metabolic substrate diffusion and enzyme activity, significantly increasing medium-chain fatty acid content. Traditionally, operators would need to adjust control parameters approximately three times daily, while the intelligent system only intervenes 0.5 times per shift, significantly reducing reliance on human labor.

[0137] It can be seen that the present invention has extremely strong adaptability and dynamic adjustment capabilities under low-temperature fermentation environment. It not only solves the problem of slow metabolic pathway reaction caused by low temperature, but also realizes fine intervention in the direction of flavor evolution through the structured difference perception mechanism, truly controlling flavor with structure, and providing an efficient and scalable solution path for low-temperature intelligent manufacturing of condiments.

[0138] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A dynamic control method for low-temperature fermentation of condiments based on reinforcement learning, characterized in that: The steps include: S1. Receive a flavor target input, wherein the flavor target is a text description of a condiment, and convert the flavor target into a metabolic intent vector; S2. Constructing a target metabolic pathway map based on the metabolic intention vector, wherein the target metabolic pathway map is composed of metabolic pathway nodes and connection relationships, and is used to represent the metabolic structure pathway corresponding to flavor formation; S3. Periodically collecting current fermentation status data at set time intervals during the fermentation process; S4. Constructing a current metabolic pathway map based on the current fermentation state data, and mapping the current metabolic pathway map to a non-Euclidean embedding space to obtain an embedded representation of the current metabolic pathway; S5. Mapping the target metabolic pathway map to the same non-Euclidean embedding space as the current metabolic pathway map to obtain an embedded representation of the target metabolic pathway, and comparing the embedded representation of the target metabolic pathway with the embedded representation of the current metabolic pathway to calculate a structural deviation; S6. Performing strategy training and updating using a Riemannian manifold strategy optimization method based on the structural deviation to generate a control strategy output in a non-Euclidean embedding space, wherein the control strategy output is represented by a metabolic pathway perturbation direction; S7, converting the control strategy output into fermentation process control instructions, wherein the fermentation process control instructions include a temperature setting value, a humidity setting value, a ventilation rate, and a stirring frequency; S8. Execute the control instructions to adjust the fermentation environment parameters and complete the regulation of the evolution direction of the metabolic pathway during the fermentation process.

2. The dynamic control method for low-temperature fermentation of condiments based on reinforcement learning according to claim 1, characterized in that: Said S1 specifically includes: S11. Receive text description information of a condiment input by a user, wherein the text description information of the condiment includes natural language expression of flavor, aroma, taste, process type, and regional style; S12, performing word segmentation processing on the condiment text description information to extract keywords with sensory characteristic meanings; S13, matching the keyword with a preset flavor-metabolite mapping dictionary, where the flavor-metabolite mapping dictionary records the association between flavor terms and specific metabolites, to obtain one or more metabolite names associated with the keyword; S14. Retrieve the metabolic pathway number for each metabolite name by querying a locally established metabolic pathway information table, and generate a metabolic pathway index list. The metabolic pathway information table is a pre-organized structured data file containing metabolites and corresponding metabolic pathway numbers, starting substrates, intermediate metabolites, end products, and reaction sequences; S15. Based on the metabolic pathway index list, obtain the association frequency between the metabolic pathway and the flavor target, use the association frequency ratio as the importance weight value of each metabolic pathway, and construct a metabolic intention vector. The metabolic intention vector is a multidimensional numerical sequence used to represent the importance of different metabolic pathways, where each dimension value is obtained by normalizing the association frequency of the corresponding metabolic pathway.

3. The dynamic control method for low-temperature fermentation of condiments based on reinforcement learning according to claim 1, characterized in that: The S2 specifically includes: S21, sorting the metabolic pathways according to the importance weight values ​​in the metabolic intention vector, and selecting metabolic pathways with importance weight values ​​higher than a set weight threshold as the target metabolic pathway set; S22, searching for structural information corresponding to the target metabolic pathway set, where the structural information includes the starting substrate, intermediate metabolites, end products, and reaction sequence involved in the target metabolic pathway; S23, extracting metabolite entries from the structural information and establishing a target metabolic pathway node set, where each node corresponds to a metabolite and is accompanied by a position identifier in the metabolic pathway; S24. According to the sequential reaction relationship of each metabolite in the metabolic pathway, a directed connection relationship between the target metabolic pathway nodes is constructed to form a structured metabolic pathway dependency chain, and the target metabolic pathway nodes and connection relationships constitute a target metabolic pathway map.

4. The dynamic control method for low-temperature fermentation of condiments based on reinforcement learning according to claim 1, characterized in that: The current fermentation state data specifically include temperature, humidity, pH value, oxygen concentration, carbon dioxide concentration and metabolite concentration.

5. The dynamic control method for low-temperature fermentation of condiments based on reinforcement learning according to claim 1, characterized in that: The S4 specifically includes: S41, performing a difference operation on the metabolite concentrations at adjacent sampling time points, calculating the rate of change of the concentration of each metabolite, and marking the metabolite with a change rate higher than a preset rate threshold as an active metabolite; S42. Constructing a current metabolic pathway node set based on the corresponding relationship of active metabolites in the locally established metabolic pathway information table; S43. Establishing directed connections between nodes in the current metabolic pathway based on the reaction sequence and transformation direction of the active metabolites in the metabolic pathway, and constructing a current metabolic pathway map; S44, calculating the path distance between the nodes of the current metabolic pathway based on the current metabolic pathway map, and generating a distance matrix, wherein an element of the distance matrix is ​​the shortest reaction step length between two nodes in the metabolic pathway; S45. Process the distance matrix using Laplace eigenmapping, and map the current metabolic pathway map to a non-Euclidean embedding space by retaining the relative positional relationship and topological structure characteristics between the nodes of the current metabolic pathway. The non-Euclidean embedding space is a structured representation space that does not satisfy Euclidean symmetry and the Pythagorean theorem.

6. The dynamic control method for low-temperature fermentation of condiments based on reinforcement learning according to claim 1, characterized in that: The calculation process of the structural deviation specifically includes: Pair the target metabolic pathway embedding representation with the nodes with the same metabolite identifier in the current metabolic pathway embedding representation in a one-to-one correspondence to form a node pair set; For each pair of paired nodes, extract the embedding vector coordinates in the non-Euclidean embedding space and calculate the embedding distance offset value of the paired nodes. The embedding distance offset value is calculated using the Euclidean distance method and is defined as the square root of the sum of the squares of the coordinate differences in each dimension between the embedding vectors of the two nodes. Calculate the node weighting factors of the two nodes in each pair of paired nodes respectively, and the node weighting factors are obtained by linear weighting according to the importance weight value of the node in the metabolic pathway and the local topological connectivity in the corresponding metabolic pathway map; Taking the arithmetic average of the node weighting factors of the two nodes in the paired node pair as the final weighting factor of the paired node, and multiplying the distance offset value of the paired node by the corresponding final weighting factor to obtain a weighted offset value; The weighted offset values ​​of each pair of paired nodes are summed and divided by the sum of the weighted factors of all node pairs to obtain a structural deviation value, which is used to quantify the overall structural difference between the current metabolic pathway map and the target metabolic pathway map in the non-Euclidean embedding space.

7. The method for dynamic control of low-temperature fermentation of condiments based on reinforcement learning according to claim 6, characterized in that: The S6 specifically includes: S61, using the structural deviation value as the state input basis, and combining it with the current metabolic pathway embedding representation, constructing a state vector in the non-Euclidean embedding space ; S62, based on the state vector , build a policy function ,in Indicates that in a given state vector Down output action The conditional probability distribution of Represents the policy function parameters, Indicates that at time step Output metabolic pathway perturbation direction vector; S63. Define the immediate reward function to represent the effect of structural optimization: ； in, Represents the time step The reward value when Represents the time step The structural deviation value when Represents the time step The structural deviation value when ; S64. Construct Fisher information matrix based on current policy function : ； in, represents the expectation operator, represents the gradient of the policy logarithmic function with respect to the parameters, Represents a transpose operation; S65. Update the strategy parameters using the natural gradient optimization method on the Riemannian manifold: ； in, Represents the time step The strategy parameters when Represents the time step The strategy parameters when represents the learning rate, Represents the time step The inverse of the Fisher information matrix is represents the discount factor, Represents the time step The reward value when Represents the gradient of the strategy's cumulative expected reward with respect to the strategy parameters; S66. Apply the updated strategy parameters to the current state vector to generate a metabolic pathway perturbation direction vector, wherein the metabolic pathway perturbation direction vector is defined in a non-Euclidean embedding space and is used to represent the adjustment direction of the metabolic pathway map in the structural space as a control strategy output.

8. The dynamic control method for low-temperature fermentation of condiments based on reinforcement learning according to claim 1, characterized in that: The S7 specifically includes: S71. Extracting the disturbance direction components on each principal component axis in the non-Euclidean embedding space according to the metabolic pathway disturbance direction vector output by the control strategy, and mapping the disturbance direction components to the fermentation process parameter adjustment direction and adjustment amplitude according to the sign and amplitude of the disturbance direction components; S72. Setting a mapping rule between the disturbance direction component and each control parameter, wherein: the temperature setting value is positively correlated with the first principal component axis component in the disturbance direction component, the humidity setting value is positively correlated with the second principal component axis component in the disturbance direction component, the ventilation rate is positively correlated with the third principal component axis component in the disturbance direction component, and the stirring frequency is positively correlated with the fourth principal component axis component in the disturbance direction component; S73. Based on the value range of the disturbance direction component of the metabolic pathway disturbance direction vector in the non-Euclidean embedding space, a hierarchical adjustment rule for the fermentation process parameters is constructed, wherein: when the absolute value of the disturbance direction component is less than 0.1, it is weak adjustment, and the adjustment range is set to ±2% of the original set value; when the absolute value of the disturbance direction component is between 0.1 and 0.3, it is medium adjustment, and the adjustment range is ±5%; when the absolute value of the disturbance direction component is greater than 0.3, it is strong adjustment, and the adjustment range is ±10%; S74. When the disturbance direction component is positive, the corresponding fermentation process parameter is increased by a set amplitude; when the disturbance component is negative, the corresponding fermentation process parameter is decreased by a set amplitude, and finally a fermentation process control instruction is generated, wherein the fermentation process control instruction includes a temperature set value, a humidity set value, a ventilation rate, and a stirring frequency; S75 , encapsulating the fermentation process control instruction into a fermentation control device-level parameter configuration instruction, and sending the instruction to the fermentation control device control execution unit to complete the dynamic adjustment of the fermentation environment.

Images