Industrial process control system

Abstract

The application relates to the technical field of production process control, and provides an industrial production process control system. A quality index generated by a material sensing module and a process fitness are used to provide input that matches actual characteristics of the material for a path generation module. The path generation module combines equipment operating states and energy consumption and efficiency targets to dynamically calculate cumulative reward expectations through a strategy network, so that the selected process path meets order quality requirements and adapts to actual equipment operating capabilities. A process simulation module performs execution simulation for a biological conversion process, combines Monte Carlo simulation to quantitatively simulate confidence, avoids potential risks in advance, generates feedforward control instructions when the simulation confidence meets a standard, and enables a closed-loop execution module to adjust reaction conditions in advance, thereby guaranteeing stability of the biological conversion process and consistency of product quality.

Description

Technical Field

[0001] This application relates to the field of production process control technology, and in particular to industrial production process control systems. Background Technology

[0002] With the improvement of the level of intelligence in industrial production, industrial production is gradually developing towards precise control and low consumption and high efficiency. Currently, industrial production process control systems have widely integrated automated equipment linkage, process path planning, and simulation technologies. Some systems have introduced machine learning to assist in path selection, optimizing energy consumption and efficiency indicators by collecting equipment operating status data, thus initially realizing the automation and partial intelligence of the production process. Especially in biotransformation production such as compound enzymes, existing technologies can complete the continuous execution of multiple processes such as raw material pretreatment, fermentation, extraction, and filtration, using simulation to predict some production risks in advance and reduce raw material waste and time loss during the physical execution stage.

[0003] Although existing industrial process control systems have basic automation and planning capabilities, they still have significant limitations when dealing with scenarios with large fluctuations in material properties and sensitivity to bioconversion processes. Existing systems often treat material properties, process paths, and equipment status separately, generating paths based solely on fixed process requirements or data from a single device, ignoring the impact of fluctuations in material quality and process adaptation on path planning. This results in poor compatibility between the process path and the actual characteristics of the material, and easily leads to unstable product quality.

[0004] Existing simulation parameters are mostly static values ​​and have not been correlated with actual production results. When the quality of finished products deviates from the requirements, the simulation parameters cannot be adjusted and optimized through feedback, resulting in insufficient consistency between simulation results and physical execution. At the same time, the reward function of path planning mostly adopts fixed weighted logic, focusing only on energy consumption and efficiency indicators, without combining dynamic optimization with actual production, which leads to substandard quality or excessive energy consumption during actual execution.

[0005] Based on the shortcomings of the existing technology, the technical problem to be solved by this application is how to construct a production process control system that coordinates and optimizes material properties, process paths and equipment status through closed-loop feedback. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this application provides an industrial production process control system, which includes: a material sensing module, a path generation module, a process simulation module, and a closed-loop execution module;

[0007] The material sensing module acquires the physical form, chemical composition and surface defects of the material, and generates a material quality profile including quality index and process adaptability after extracting features through a deep convolutional neural network.

[0008] The path generation module receives material quality profiles and order process requirements, reads equipment operating status from the processing resource pool, maps process fitness to weighted coefficients of reward functions with energy consumption and efficiency as objectives, calculates the cumulative reward expectation of candidate process paths based on the policy network, and generates a path instruction sequence after screening.

[0009] Before physical execution, the process simulation module simulates the biotransformation process in the path instruction sequence. When the simulation confidence is less than a preset threshold, it sends abnormal simulation data and triggers path replanning. Otherwise, it generates feedforward control instructions including parameter correction and metabolic inflection point markers and starts the biotransformation process.

[0010] The closed-loop execution module sets the reaction conditions of the bioconversion process according to the feedforward control instructions, and monitors the substrate concentration change rate to generate the adjustment amount of the reaction conditions. After bioconversion, it obtains the quality of the finished product and compares it with the quality index. When it deviates from the order process requirements, it feeds back to adjust the simulation parameters and the reward function of the strategy network.

[0011] As an optional implementation, the generation of the material quality profile includes:

[0012] The physical morphology, chemical composition, and surface defects of the material are obtained, and spatiotemporal alignment and noise reduction are performed.

[0013] Based on deep convolutional neural networks, spatial distribution features of physical morphology, synergistic features of chemical composition and topological correlation features of surface defects are extracted hierarchically. The hierarchical features are then fused through a cross-modal attention mechanism to generate fused features that include basic features and process-sensitive features.

[0014] The process scenario weights are adjusted according to the sensitivity of different order processes to materials. The basic quality index is obtained by weighting the basic features and the benchmark weights, and the process sensitivity index is obtained by weighting the process sensitive features and the process scenario weights. The basic quality index and the process sensitivity index are added together to form the quality index.

[0015] Meta-learning is used to calculate the feature distance between the fused features and the order process, and the process fitness is generated by weighting. The quality index and process fitness are calibrated, and a material quality profile is generated when the calibration deviation is less than the deviation threshold.

[0016] As an optional implementation, calculating the cumulative reward expectation of the candidate process path includes:

[0017] Based on the process adaptability of the order process requirements, the equipment operating status of the processing resource pool, and the material quality profile, candidate process paths are generated, including process sequence, equipment matching, and process parameters.

[0018] With energy consumption and efficiency as the objectives, the process adaptability is mapped to the weighted coefficients of the reward function, and the constraint penalty terms of the order process requirements are added;

[0019] Based on the policy network, the correlation characteristics of energy consumption and efficiency between processes in the candidate process path are captured by the attention mechanism. Combined with the time-series characteristics of equipment operation status, the instantaneous reward value of the candidate process path is calculated online, and the cumulative reward expectation is obtained by accumulating based on the time-series discount factor.

[0020] As an optional implementation, the generated path instruction sequence includes:

[0021] Candidate paths are selected by screening based on the cumulative reward expectation of the candidate process paths and by combining the matching verification of process adaptability and equipment operating status.

[0022] Based on the fluctuating nature of the quality index, the adjustment range of process parameters for each process in the selected path is dynamically configured, and the process connection sequence of the selected path is optimized based on the correlation characteristics of energy consumption and efficiency between processes output by the strategy network.

[0023] The optimized candidate paths are validated, and the validated candidate paths are converted into equipment execution instructions that include process sequence, process parameters and adjustment range, generating a path instruction sequence.

[0024] As an optional implementation, the triggered path replanning includes:

[0025] Before physical execution, simulation is performed by combining the process parameters of the bioconversion process in the path instruction sequence, the fluctuation characteristics of the quality index in the material quality profile, and the equipment operating status.

[0026] Multiple simulation scenarios are generated through Monte Carlo simulation. For each simulation scenario, the deviation distribution between the reaction conditions in the biotransformation process and the measured values ​​of similar historical processes is calculated to form the simulation confidence level.

[0027] The preset threshold is dynamically adjusted according to the stringency of the order process requirements. When the simulation confidence level is less than the preset threshold, the abnormal source is located by causal inference and abnormal simulation data is sent to trigger path replanning.

[0028] As an optional implementation, the generation of feedforward control instructions includes:

[0029] When the simulation confidence level is not less than the preset threshold, the parameter correction amount of the process parameters is calculated based on the fluctuation nature of the quality index and the stability characteristics of the equipment operating status, combined with the simulation data.

[0030] The abrupt change characteristics of reaction conditions are extracted from simulation data. Combined with the shift patterns of metabolic inflection points in similar historical processes and the synergistic effects of chemical components, the time nodes and control parameter thresholds of metabolic inflection points are determined to form metabolic inflection point identifiers.

[0031] Integrate parameter correction values ​​and metabolic inflection point markers to generate feedforward control commands and initiate the biotransformation process.

[0032] As an optional implementation, the adjustment amount of the generation reaction conditions includes:

[0033] The reaction conditions of the biotransformation process are set according to the feedforward control instructions, the substrate concentration change rate is monitored to generate adjustment signals, and the metabolic inflection point markers of the feedforward control instructions are combined to determine the current reaction stage.

[0034] Based on the fluctuating nature of the quality index and the stable characteristics of equipment operation, a basic adjustment amount is generated for the adjustment range of the appropriate process parameters.

[0035] If the adjustment direction of the feedforward control command conflicts with that of the adjustment signal, the weights are dynamically allocated according to the simulation confidence level to form a coordinated adjustment amount. If there is no conflict, the adjustment value of the adjustment signal is superimposed on the basic adjustment amount to form a coordinated adjustment amount.

[0036] The synergistic adjustment amount is graded and corrected according to different reaction stages to form the adjustment amount of reaction conditions.

[0037] As an optional implementation, the feedback adjustment of simulation parameters includes:

[0038] After biotransformation, the quality of the finished product is obtained and compared with the quality index. When it deviates from the order process requirements, the deviation magnitude and type are analyzed, and the source of deviation is located through feature matching.

[0039] The simulation parameters to be adjusted are determined based on the deviation type of the deviation source, and the initial adjustment amount is generated by combining the parameter adjustment records and deviation magnitudes under the same type of deviation in history.

[0040] The initial adjustment amount is optimized based on the adjustment range constraint of the process parameters. The optimized simulation parameters are then verified through simulation until the quality of the finished product meets the order process requirements.

[0041] As an optional implementation, the reward function of the policy network is adjusted by feedback, including:

[0042] When the quality of the finished product deviates from the order process requirements, analyze the magnitude of the quality deviation, calculate the deviation between actual energy consumption and energy consumption target and between actual efficiency and efficiency target, and correlate the difference between process adaptability and actual production adaptability to help determine the root cause of the deviation.

[0043] If the deviation is in quality, the weight of the constraint penalty item corresponding to the order process requirements is optimized; if the deviation is in energy consumption and efficiency, the correlation between the weighted coefficients of the process fitness mapping and the reward function is corrected.

[0044] By applying weight constraints to the adjustment range of process parameters and dynamically calibrating the adjustment range based on the deviation improvement rate, the adjusted reward function is input into the strategy network to verify the adaptability of the cumulative reward expectation of the newly generated candidate process path, so as to provide feedback to adjust the reward function of the strategy network.

[0045] Compared with the prior art, the beneficial effects of this application are: the quality index and process adaptability generated by the material sensing module provide the path generation module with input that fits the actual characteristics of the material. The path generation module combines the equipment operating status with the goals of energy consumption and efficiency, and dynamically calculates the cumulative reward expectation through the strategy network, so that the selected process path not only meets the order quality requirements, but also adapts to the actual operating capacity of the equipment, avoiding the problem of theoretical optimization but practical infeasibility, and significantly improving the reliability of path execution.

[0046] The process simulation module performs execution simulations for biotransformation processes. It combines Monte Carlo simulation to quantify simulation confidence and mitigate potential risks in advance. When the simulation confidence reaches the target, the generated feedforward control instructions include parameter corrections to adapt to material fluctuations and equipment status, as well as metabolic inflection point indicators. This allows the closed-loop execution module to adjust reaction conditions in advance, avoiding problems such as degradation of active ingredients and increase of byproducts caused by parameter adjustment lag, thus ensuring the stability of the biotransformation process and the consistency of product quality.

[0047] The closed-loop execution module establishes a closed-loop feedback for finished product quality deviations, root cause identification, and parameter optimization. After accurately identifying the root causes of deviations in quality, energy consumption, and efficiency, it optimizes the simulation parameters and the reward function of the strategy network respectively. The feedback adjustment of simulation parameters makes subsequent simulations more closely resemble actual production, and the dynamic optimization of the reward function makes the evaluation logic of path planning more suitable for production goals. The synergistic optimization effect formed by the two allows the system to continuously adapt to dynamic scenarios such as material batch fluctuations and equipment status changes, and can achieve self-iterative upgrades without manual intervention.

[0048] Through multi-module collaboration and closed-loop control, the system ensures finished product quality through precise process control and metabolic inflection point capture, reduces production energy consumption by optimizing process paths and equipment linkage, and improves equipment utilization and production efficiency, providing stable and reliable technical support for large-scale industrial production. Attached Figure Description

[0049] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0050] Figure 1 This is a system flowchart of an industrial production process control system provided in an embodiment of this application;

[0051] Figure 2 The following is a logic flowchart for calculating the cumulative reward expectation of candidate process paths in an industrial production process control system provided in this application embodiment;

[0052] Figure 3 This is a logic flowchart of the trigger path replanning of the industrial production process control system provided in the embodiments of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application more apparent and understandable, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0054] like Figure 1 The diagram shown is a system flowchart of an industrial production process control system provided in this application embodiment. The system includes a material sensing module, a path generation module, a process simulation module, and a closed-loop execution module.

[0055] The application scenario of this embodiment is the industrial production of a mixture of snow lotus fruit and dogwood pulp with compound enzymes.

[0056] The material sensing module acquires the physical form, chemical composition and surface defects of the material, and generates a material quality profile including quality index and process adaptability after extracting features through a deep convolutional neural network.

[0057] Specifically, generating a material quality profile includes:

[0058] The physical morphology, chemical composition, and surface defects of the material are obtained, and spatiotemporal alignment and noise reduction are performed.

[0059] Based on deep convolutional neural networks, spatial distribution features of physical morphology, synergistic features of chemical composition and topological correlation features of surface defects are extracted hierarchically. The hierarchical features are then fused through a cross-modal attention mechanism to generate fused features that include basic features and process-sensitive features.

[0060] The process scenario weights are adjusted according to the sensitivity of different order processes to materials. The basic quality index is obtained by weighting the basic features and the benchmark weights, and the process sensitivity index is obtained by weighting the process sensitive features and the process scenario weights. The basic quality index and the process sensitivity index are added together to form the quality index.

[0061] Meta-learning is used to calculate the feature distance between the fused features and the order process, and the process fitness is generated by weighting. The quality index and process fitness are calibrated, and a material quality profile is generated when the calibration deviation is less than the deviation threshold.

[0062] The physical morphology of the mixed yacon and cornus officinalis pulp directly affects the contact efficiency between the substrate and the inoculum during fermentation. Chemical composition determines the efficacy and taste of the compound enzyme. Surface defects affect fermentation stability and product uniformity. During the testing of the mixed materials, differences in the timing of different equipment can easily lead to data asynchrony in time and space. Ambient temperature and detection errors can introduce noise, and directly using raw data can lead to distortion in subsequent feature extraction. Physical morphology data such as particle size distribution and aggregation state of the mixed pulp are obtained using a laser particle size analyzer. The types and contents of chemical components such as yacon polysaccharides, cornus officinalis flavonoids, and organic acids are detected using high-performance liquid chromatography. A three-dimensional optical profilometer is used to capture the morphology and aggregates of residual fibers in the pulp. The data includes surface defects such as uneven surface structure; based on the homogenization time of the mixture, the three types of test data are precisely matched according to the timestamp to ensure that the physical morphology, chemical composition and surface defect data of the same batch of materials correspond; random electronic noise in the acquired data is filtered by wavelet threshold denoising method, and abnormal data caused by equipment failure or local unevenness of raw materials is removed by median filtering, retaining effective information that can truly reflect the properties of the mixture; thereby eliminating the interference caused by data spatiotemporal asynchrony and noise, ensuring that the acquired multi-dimensional data of the mixture are true and consistent, avoiding the misjudgment of defect data such as residual fibers and local agglomeration as effective features, and laying the foundation for subsequent accurate extraction of material characteristics.

[0063] The physical morphology of the mixture of yacon and cornus officinalis affects the mass transfer efficiency during fermentation, the synergistic effect of chemical components directly determines the efficacy and fermentation rate of the enzyme, and surface defects affect the taste and stability of the finished product. These three modal features are interrelated and irreplaceable. A single modal feature cannot fully reflect the fermentation adaptability potential of the mixture, and the compound enzyme process has different sensitivities to different modal features. A three-branch deep convolutional neural network was constructed. The first branch uses a 3D convolutional layer to capture the spatial distribution pattern of particles and the three-dimensional structure of aggregates in the mixed pulp, focusing on identifying the spatial distribution features of particle size that can affect mass transfer efficiency. The second branch uses a fully connected layer combined with an attention mechanism to extract the synergistic effect features of chemical components, such as the ratio of yacon polysaccharides to cornus officinalis flavonoids and the adaptation relationship between organic acids and fermentation strains, highlighting the component correlation patterns that play a key role in enzyme efficacy and fermentation rate. The third branch uses a graph convolutional layer to mine surface defect features such as the connectivity of residual fibers and the topological correlation of the distribution of surface defects in aggregates, identifying structural hazards that may affect the uniformity of the finished product.

[0064] This approach integrates three types of hierarchical features using a cross-modal attention mechanism. By calculating the correlation weights between different modal features and the compound enzyme fermentation process (including strain activity and fermentation cycle), the core role of the synergistic effect of chemical components is highlighted. Simultaneously, auxiliary information such as the spatial distribution of physical morphology and the topological correlation of surface defects is retained. The result is a fused feature encompassing both fundamental and process-sensitive features. Fundamental features include the total solids content and the total proportion of core components in the mixed pulp, while process-sensitive features include adaptability to specific fermentation temperatures and strain types. This overcomes the limitations of single-modal features, precisely capturing the core patterns of various properties of the mixed materials through hierarchical extraction. Cross-modal fusion achieves complementarity in component synergy, physical morphology, and structural defect information. The generated fused feature covers both the inherent fundamental properties of the materials and highlights the adaptability to the compound enzyme process, providing comprehensive support for the accurate calculation of subsequent quality indices and process adaptability, ensuring a true reflection of material quality and production requirements.

[0065] The process requirements for compound enzyme orders vary. For example, functional enzyme orders focus on the retention rate of active ingredients such as flavonoids and polysaccharides, taste-oriented enzyme orders focus on the fineness and acidity balance of the finished product, and ordinary enzyme orders focus on production efficiency and cost control. Different orders have different sensitivities to the quality of mixed materials. If a fixed weight is used to calculate the quality index, it cannot adapt to the personalized needs of the orders, resulting in the quality index not being able to accurately guide the selection of process paths. First, analyze the process requirements of the orders and clarify the core sensitive dimensions. For example, functional enzyme orders should increase the weight of the characteristics related to the synergistic retention of active ingredients in the process sensitive characteristics, taste-oriented enzyme orders should increase the weight of the characteristics related to particle fineness and acidity balance, and ordinary enzyme orders should increase the weight of the characteristics related to fermentation efficiency.

[0066] A baseline weight reflecting the inherent quality of the mixed materials is selected. The basic features in the fusion features are weighted and summed with the baseline weight to obtain a basic quality index reflecting the general quality level of the materials. The process-sensitive features in the fusion features are weighted and summed with the adjusted process scenario weight to obtain a process-sensitive index reflecting the material's adaptation to the specific order's process quality level. The basic quality index and the process-sensitive index are accumulated to form a quality index that comprehensively reflects both the general quality of the mixed materials and the order-adaptive quality. This enables personalized order adaptation of the quality index, avoiding the distortion of quality assessment caused by a one-size-fits-all weight setting. The quality index can objectively reflect the inherent quality of the mixed materials and accurately match the production needs of different types of compound enzymes, providing a targeted quality basis for subsequent process route planning.

[0067] The diversity of compound enzyme processes leads to varying material characteristic requirements across different orders. Traditional feature distance calculation methods struggle to adapt to the differences in multiple process scenarios. Meta-learning leverages historical material data from multiple order processes to optimize feature distance calculations, improving the universality and accuracy of process adaptability assessment. While the quality index emphasizes material quality levels and process adaptability focuses on the matching degree between materials and processes, there can be assessment biases. For example, a high quality index might indicate low compatibility of active ingredients. Mutual calibration is needed to ensure the consistency and reliability of both indicators, avoiding misleading subsequent process decisions by a single indicator. Meta-learning is constructed by inputting the mixture fusion characteristics and order process requirements of different types of historical compound enzyme orders. It learns matching rules between material characteristics and process requirements under different process scenarios. Based on these matching rules, the feature distance between the current mixture fusion characteristics and the target order process requirements is calculated. A smaller distance indicates higher adaptability. The feature distance is then weighted by the importance weight of the order process to generate process adaptability.

[0068] Using the historical quality index and process adaptability of qualified mixed materials as reference benchmarks, the two currently calculated indicators are compared with the reference benchmark values ​​respectively. Through mutual verification, deviations are corrected. For example, if the quality index is high but the process adaptability is low, the weights of components related to fermentation adaptability in the quality index are adjusted based on process-sensitive characteristics. If the process adaptability is high but the quality index is low, the evaluation logic of process adaptability is adjusted based on basic characteristics. A deviation threshold is set. When the calibration deviation between the quality index and process adaptability is less than the threshold, the accuracy of the two indicators is confirmed, and they are integrated to form a complete material quality profile including the quality index and process adaptability. The introduction of meta-learning enables the process adaptability calculation to accurately adapt to the process differences of different types of compound enzymes. The indicator calibration avoids the limitations of single indicator evaluation. The final generated material quality profile can comprehensively and accurately reflect the quality level of the mixed materials and the adaptability potential of the compound enzyme process, providing reliable input data for the entire production control system and ensuring the accuracy and stability of the industrial production of snow lotus fruit and dogwood mixed pulp and compound enzymes.

[0069] The path generation module receives material quality profiles and order process requirements, reads equipment operating status from the processing resource pool, maps process fitness to weighted coefficients of reward functions with energy consumption and efficiency as objectives, calculates the cumulative reward expectation of candidate process paths based on the policy network, and generates a path instruction sequence after screening.

[0070] Furthermore, such as Figure 2 As shown, the calculation of the cumulative expected reward for candidate process paths includes:

[0071] Based on the process adaptability of the order process requirements, the equipment operating status of the processing resource pool, and the material quality profile, candidate process paths are generated, including process sequence, equipment matching, and process parameters.

[0072] With energy consumption and efficiency as the objectives, the process adaptability is mapped to the weighted coefficients of the reward function, and the constraint penalty terms of the order process requirements are added;

[0073] Based on the policy network, the correlation characteristics of energy consumption and efficiency between processes in the candidate process path are captured by the attention mechanism. Combined with the time-series characteristics of equipment operation status, the instantaneous reward value of the candidate process path is calculated online, and the cumulative reward expectation is obtained by accumulating based on the time-series discount factor.

[0074] The production of compound enzymes involves multiple processes, and the process requirements vary from order to order. For example, functional enzymes require enhanced extraction of active ingredients, while flavor-oriented enzymes require optimized clarification and filtration. Furthermore, the equipment in the processing resource pools may have different operating statuses; some fermenters may be in standby mode, while others require maintenance. Ignoring equipment operating status can easily lead to interruptions in the process flow. The process adaptability of material quality profiles directly reflects the material's potential to adapt to different processes. Deviating from the process adaptability-based process flow will result in mismatches between process parameters and material characteristics. For example, the material may be suited for low-temperature fermentation, but the process flow may be set to a high-temperature process. Therefore, it is crucial to analyze the order's process requirements and clarify the process objectives, including active ingredient retention, production efficiency, and flavor optimization. Combining the process adaptability of the mixture of snow lotus fruit and dogwood, a basic process framework is determined. For instance, the process sequence for a functional enzyme order is "raw material cleaning → grading and crushing → low-temperature mixing → constant-temperature fermentation → gradient extraction → precision filtration → aseptic filling," while a flavor-oriented enzyme order adds a flavor blending step.

[0075] The system reads the equipment operating status from the processing resource pool, including equipment availability, operating load, and maintenance cycle. It then matches processes with suitable equipment; for example, a low-temperature mixing process is matched with a homogenizer with temperature control, and a constant-temperature fermentation process is matched with a stable and appropriately sized fermenter. This avoids assigning core processes to equipment that is about to undergo maintenance. Based on process adaptability, the system optimizes process parameters. For instance, if the process adaptability indicates that the material is suitable for low-speed mixing, the corresponding mixing speed is set; if it is suitable for a specific fermentation pH value, the acid-base adjustment parameters of the fermentation process are adjusted. This ultimately forms multiple candidate process paths, including process sequences, equipment matching, and process parameters. These candidate process paths simultaneously consider the personalized needs of orders, the actual operating capabilities of equipment, and the material adaptability potential, preventing paths from becoming unexecutable or resulting in poor process effects due to deviations from key factors. This provides a set of foundational paths with practical application value for the accurate calculation of subsequent accumulated reward expectations.

[0076] In the industrial production of compound enzymes, energy consumption and efficiency are economic indicators that need to be highlighted in the reward function. Process adaptability reflects the degree to which materials are adapted to the process path. Process paths with high adaptability are more likely to achieve energy consumption and efficiency targets, and should be mapped to weighted coefficients to strengthen the reward weight of adaptable paths. At the same time, order process requirements are an insurmountable core constraint. If the process path violates this constraint, even if the energy consumption and efficiency are optimal, it has no practical value. Constraint penalty terms need to be added to avoid paths that do not meet order requirements. First, the basic framework of the reward function is constructed with the goal of reducing production energy consumption and improving production efficiency. Process adaptability is mapped to weighted coefficients, where the higher the process adaptability, the greater the weight of the corresponding process path in terms of energy consumption optimization and efficiency improvement. For example, for paths with high process adaptability, the proportion of efficiency-related indicators in the reward function increases, encouraging the selection of paths that are adapted to the materials.

[0077] Then, constraint penalty items are set according to the process requirements of the compound enzyme order. If the process settings or process parameters of the candidate process path will cause the order requirements to be unmet, such as the absence of an active ingredient extraction process in the functional enzyme path and insufficient filtration precision in the taste enzyme path, constraint penalty items are added to the reward function to weaken the reward value of the process path. At the same time, the setting of constraint penalty items must be in line with the characteristics of the scenario. For example, if the temperature parameters of the fermentation process deviate from the process adaptability range, they are also included in the constraint penalty scope to ensure that the process path takes into account both energy consumption efficiency and order requirements. The reward function strengthens the correlation between material adaptability and economic indicators through weighted coefficients, and ensures the rigidity of order requirements through constraint penalty items, avoiding product non-compliance caused by only considering energy consumption and efficiency. This allows the reward function to comprehensively and objectively evaluate the comprehensive value of the candidate process path and provide evaluation logic for the calculation of the immediate reward value.

[0078] There are strong correlations between the processes in the production of compound enzymes. For example, the degree of crushing in the crushing process directly affects the energy consumption and mixing efficiency of the subsequent mixing process, while the uniformity of mixing affects the reaction rate of the fermentation process. Traditional evaluation methods tend to overlook this correlation, leading to a one-sided calculation of reward values. At the same time, the operating status of equipment is not constant. For example, after long-term operation, fermentation tanks may experience temporal changes such as increased energy consumption and decreased temperature control accuracy. If temporal characteristics are ignored, the calculation of reward values ​​will be out of sync with the actual operating conditions of the equipment. In addition, the impact of subsequent processes on the overall path value needs to be reasonably weighed. The importance of recent processes should be highlighted through a time-series discount factor. For example, the filtration and filling processes, which are close to the finished product, have a greater impact on product quality and should be given higher weights to achieve accurate calculation of the expected cumulative reward.

[0079] A strategy network adapted to the production scenario of compound enzymes is constructed. This strategy network embeds an attention mechanism, which captures the correlation characteristics between processes by analyzing the energy consumption and efficiency data of each process in the candidate process path. For example, it identifies the positive correlation between the energy consumption reduction of the crushing process and the efficiency improvement of the mixing process, or the negative correlation between the temperature fluctuation of the fermentation process and the energy consumption increase of the filtration process, and assigns higher evaluation weights to process combinations with high correlation. At the same time, the temporal characteristics of equipment operation status are acquired in real time, such as the energy consumption changes of the fermenter at different production stages and the operational stability trend of the homogenizer, and these are integrated into the strategy network so that the reward value calculation can dynamically adapt to the changes in equipment status. Based on the strategy network and the reward function, the instant reward value of each process is calculated online. If the process meets the energy consumption and efficiency targets, conforms to the order process constraints, and has good synergy with the preceding and following processes, a positive instant reward value is assigned; otherwise, a lower reward value is assigned or a penalty item is triggered.

[0080] The immediate reward values ​​of each process are accumulated based on a time-series discount factor. Processes closer to the time are given a higher discount weight, including late-stage fermentation control and precision filtration. Processes further back in time, such as raw material cleaning, are given a relatively lower weight. This results in the expected cumulative reward for each candidate process path. An attention mechanism accurately captures the correlation characteristics of processes, avoiding misjudgments of the overall path value caused by isolated evaluation of single processes. The integration of equipment time-series characteristics ensures that the reward value calculation aligns with the actual operating status of the equipment, improving the accuracy of the evaluation. The application of the time-series discount factor reasonably balances the importance of processes at different stages, ensuring that the expected cumulative reward comprehensively and objectively reflects the overall value of the path, including economic indicators, order suitability, and equipment suitability. This lays the foundation for generating the path instruction sequence and avoids problems such as excessive energy consumption, low efficiency, or unmet order requirements in subsequent production processes due to inaccurate path evaluation.

[0081] Specifically, the generated path instruction sequence includes:

[0082] Candidate paths are selected by screening based on the cumulative reward expectation of the candidate process paths and by combining the matching verification of process adaptability and equipment operating status.

[0083] Based on the fluctuating nature of the quality index, the adjustment range of process parameters for each process in the selected path is dynamically configured, and the process connection sequence of the selected path is optimized based on the correlation characteristics of energy consumption and efficiency between processes output by the strategy network.

[0084] The optimized candidate paths are validated, and the validated candidate paths are converted into equipment execution instructions that include process sequence, process parameters and adjustment range, generating a path instruction sequence.

[0085] While the expected cumulative reward can reflect the overall value of candidate paths, relying solely on this for selection can lead to issues where theoretically optimal paths may not be practically suitable. For example, some paths may have high expected cumulative rewards, but there may be implicit discrepancies between the process adaptability and the actual characteristics of the materials. This means that the extraction temperature preset in the path may not perfectly match the stable temperature range of the active ingredients in the materials, or the equipment operating status may change dynamically after the path is generated, such as a sudden failure of the originally matched fermenter or a decrease in available load. Directly using such process paths as the execution target can easily lead to production interruptions or substandard product quality. First, candidate process paths are sorted from high to low according to their expected cumulative reward, and several top-ranked paths are selected, prioritizing those with outstanding overall performance. Then, the process adaptability of the mixture of snow lotus fruit and dogwood is combined to verify the matching between the process settings of the core processes of the process path and the material's adaptability potential. For example, if the process adaptability indicates that the material is suitable for low-temperature long-term fermentation, the temperature and duration parameters of the fermentation process in the path are checked to see if they match. If the deviation is too large, the path is eliminated.

[0086] Finally, the equipment operating status of the processing resource pool is updated in real time, including the current availability of equipment, real-time load, and parameter execution accuracy. It verifies whether the equipment matched with the process path can still meet the process requirements. For example, it verifies whether the current filtration accuracy of the filtration equipment meets the standard and whether the real-time temperature control capability of the fermenter is suitable for the path parameters. This avoids assigning processes to equipment that has reached its load limit or has insufficient parameter accuracy. Ultimately, multiple candidate paths with high comprehensive value, high material adaptability, and equipment feasibility are determined. Through cumulative reward expectation screening and dual adaptability verification, the process path with the best overall performance is retained while hidden adaptability defects and equipment risks are eliminated. This ensures that the candidate paths have actual production feasibility, avoids production errors caused by blindly pursuing theoretical optimality, and ensures that the optimized paths can directly meet the equipment execution requirements.

[0087] The quality index of the mixture of snow lotus fruit and dogwood exhibits natural fluctuations, such as differences in the content of active ingredients and particle uniformity among different batches of raw materials. If a fixed range of process parameters is used for adjustment, when the quality index fluctuations exceed the parameter adaptation range, it can easily lead to unstable product quality, such as fluctuations in the extraction rate of active ingredients and uneven taste. At the same time, there is a strong correlation between energy consumption and efficiency between the processes in the production of compound enzymes. For example, the particle size of the crushing process directly affects the energy consumption of the mixing process, and the timing of the fermentation process affects the efficiency of the extraction process. If the process sequence is unreasonable, such as allowing the crushed material to stand for a long time before mixing, it will cause particle agglomeration, resulting in energy waste or decreased efficiency. First, analyze the nature of the fluctuations in the quality index in the material quality profile, including the amplitude and dimension of the fluctuations, and whether it is a fluctuation in the content of active ingredients or a fluctuation in particle size. For each process in the selected path, dynamically configure the adjustment range of process parameters. For example, if the quality index shows large fluctuations in the content of active ingredients, expand the temperature and time adjustment range of the extraction process to ensure that different batches of materials can be fully extracted. If the particle size fluctuates greatly, optimize the speed adjustment range of the crushing process. The adjustment range must be close to the execution range of the equipment parameters to avoid exceeding the equipment's capacity.

[0088] The strategy network outputs correlation characteristics of energy consumption and efficiency between processes, including the energy consumption synergy characteristics of crushing and mixing processes and the efficiency correlation patterns of fermentation and extraction processes. This optimizes the timing of process connections, such as starting the mixing process immediately after the crushing process to avoid material agglomeration and increased mixing energy consumption, and preheating the extraction equipment in advance when the fermentation process is nearing its end to shorten process switching intervals, improve overall production efficiency, and avoid timing conflicts to prevent multiple processes from occupying the same type of equipment simultaneously. The dynamically configured adjustment range of process parameters can adapt to fluctuations in material quality index, ensuring the flexibility and adaptability of process parameters. The optimization of process connection timing strengthens the synergy between processes, reduces energy waste and time loss, improves the actual execution efficiency of the process path and the stability of product quality, and provides a technical basis for the conversion of subsequent equipment execution instructions, ensuring that it can directly guide the equipment to flexibly respond to material fluctuations and efficiently complete process collaboration.

[0089] While the optimized candidate paths possess adaptability and synergy, potential execution conflicts still exist. For example, there may be overlapping conflicts in the parameter adjustment ranges of different processes, or conflicts between the timing arrangements and equipment maintenance time. Directly translating these into instructions could lead to chaotic equipment execution. Furthermore, equipment execution requires standardized and concrete instructions, such as temperature control instructions for fermenters and pressure adjustment instructions for filtration equipment, rather than abstract path descriptions. The optimized candidate paths undergo multi-dimensional verification, including the compatibility of process parameter adjustment ranges, the feasibility of process connection timing, and the matching of equipment parameters. For instance, whether the temperature adjustment ranges of the mixing and fermentation processes are reasonably connected without logical conflicts, whether the timing arrangements avoid equipment maintenance windows, whether equipment switchover preparation time is reserved, and whether the parameters in the path are within the rated execution range of the equipment.

[0090] For candidate paths that pass verification, they are converted into equipment execution instructions according to the equipment execution specifications. The execution sequence, process parameters, and parameter adjustment ranges for each process are clearly defined. Process parameters include mixing speed, fermentation temperature, and extraction time, while parameter adjustment ranges include the allowable temperature fluctuation range and pressure adjustment range. These are then associated with corresponding equipment identifiers, such as fermenter #3 and filtration equipment #2, ensuring that the equipment execution instructions correspond to the equipment. Finally, all equipment execution instructions for each process are integrated to form a complete path instruction sequence, including clear process execution logic, precise parameter settings, and flexible adjustment ranges. Final verification thoroughly eliminates potential execution conflicts among candidate paths, ensuring the feasibility of the instructions. The equipment execution instructions enable the equipment to directly identify and accurately execute them, avoiding operational deviations caused by ambiguous instructions. The adjustment ranges allow the equipment to flexibly respond to minor fluctuations in the production process, ensuring production stability and ensuring that actual production strictly adheres to the process path planning, achieving the product quality and production efficiency targets required by the order.

[0091] Before physical execution, the process simulation module simulates the biotransformation process in the path instruction sequence. When the simulation confidence is less than a preset threshold, it sends abnormal simulation data and triggers path replanning; otherwise, it generates feedforward control instructions including parameter corrections and metabolic inflection point markers and starts the biotransformation process.

[0092] Specifically, such as Figure 3 As shown, the triggers for path replanning include:

[0093] Before physical execution, simulation is performed by combining the process parameters of the bioconversion process in the path instruction sequence, the fluctuation characteristics of the quality index in the material quality profile, and the equipment operating status.

[0094] Multiple simulation scenarios are generated through Monte Carlo simulation. For each simulation scenario, the deviation distribution between the reaction conditions in the biotransformation process and the measured values ​​of similar historical processes is calculated to form the simulation confidence level.

[0095] The preset threshold is dynamically adjusted according to the stringency of the order process requirements. When the simulation confidence level is less than the preset threshold, the abnormal source is located by causal inference and abnormal simulation data is sent to trigger path replanning.

[0096] The biotransformation process of compound enzymes is crucial in determining the content of active ingredients and the taste of the finished product. Even slight deviations in reaction conditions can lead to substandard product quality. These conditions include fermentation temperature, pH value, and stirring rate. The quality index of the mixed snow lotus fruit and dogwood pulp exhibits natural fluctuations, such as differences in sugar content and fiber residue between different batches. Furthermore, equipment operating status changes over time. Directly following the path instructions could result in production failure due to unforeseen fluctuations or equipment malfunctions. Therefore, the process simulation module is invoked, inputting the process parameters of the fermentation process from the path instruction sequence, including preset fermentation temperature, pH adjustment range, stirring rate, and fermentation cycle. It also imports the fluctuation characteristics of the quality index from the material quality profile, including the fluctuation range of sugar content and the viscosity fluctuation characteristics caused by fiber residue, as well as the real-time operating status of the fermenter, such as current temperature control error, stirrer motor load, and pH sensor calibration status.

[0097] Based on the kinetic model of compound enzyme fermentation, this method reflects the correlation between substrate consumption, product generation, and reaction conditions. It simulates indicators such as the generation curve of active ingredients, changes in system pH, and material uniformity during fermentation, and outputs simulation process data, including product concentration and equipment energy consumption at different time points. Through pre-execution simulation, the impact of material fluctuations and equipment operating status on the biotransformation process is simulated before physical production, exposing potential problems in advance. For example, uneven stirring due to material viscosity fluctuations and decreased product activity due to equipment temperature control deviations can be avoided. This avoids the waste of raw materials and time caused by direct execution, provides basic input for Monte Carlo simulation, and enables multi-scenario deviation analysis to have specific process parameters and reaction characteristics as a basis, ensuring that the subsequent simulation confidence calculation is consistent with the actual production logic.

[0098] A single simulation scenario cannot cover all uncertainties in the biotransformation process. For example, extreme fluctuations in material quality and sudden minor equipment malfunctions can lead to the risk that the simulation is successful but the actual production deviates. The reliability of the simulation results needs to be verified by comparison with historical similar processes. If the simulated reaction conditions deviate too much from the historical measured values, it indicates that there are problems with the simulation environment or input parameters, and further verification is required. Based on the aforementioned basic simulation environment, Monte Carlo simulation is used to generate multiple sets of simulation scenarios. Each set of simulation scenarios introduces different degrees of uncertainty variables, such as extreme values ​​of material quality index fluctuations and random minor deviations in equipment operating parameters, to simulate the combination of fluctuations that may occur in actual production.

[0099] The reaction conditions of the fermentation process in each simulation scenario are extracted and compared with the measured values ​​of similar historical compound enzyme processes. Deviation values ​​such as temperature and pH deviations are calculated, with the measured values ​​including reaction condition records under the same material batches and equipment conditions. The deviation distribution characteristics of all scenarios are statistically analyzed, including the deviation concentration range and the proportion of extreme deviations. The proportion of scenarios where the overall deviation falls within the historical acceptable range is used as the simulation confidence level. The higher the proportion of scenarios, the stronger the consistency between the simulation results and actual production, and the higher the simulation confidence level. Monte Carlo simulation covers multi-dimensional uncertainties, avoiding the limitations of a single scenario. By comparing the deviation distribution with historical measured values, the reliability of the simulation results is objectively quantified, providing a clear and credible basis for subsequent decision-making, rather than subjective judgment.

[0100] Different orders have varying degrees of stringency in their process requirements for compound enzymes. For example, functional enzymes have a smaller tolerance for errors in the content of active ingredients, while ordinary enzymes have a larger tolerance for errors in taste. If a fixed threshold is used to determine the simulation confidence level, it will lead to insufficient risk control for stringent orders or excessive replanning for lenient orders, resulting in wasted efficiency. At the same time, when the simulation confidence level is insufficient, it is necessary to identify the root cause of the anomaly, i.e., whether it is excessive material fluctuation, abnormal equipment status, or unreasonable path parameters, in order to replan accordingly. A preset threshold is set according to the stringency of the order's process requirements. For functional enzyme orders, the preset threshold is set to a higher level to require more simulation scenarios to deviate from historical measured values ​​within the acceptable range, while for ordinary enzyme orders, the preset threshold is set to a relatively lower level. The calculated simulation confidence level is compared with the preset threshold. If the simulation confidence level is less than the preset threshold, it indicates that the reliability of the simulation results is insufficient and the risk of failing to meet the actual production standards is high, in which case causal inference is initiated.

[0101] By analyzing deviation data from various simulation scenarios, the causal relationship between quality index fluctuations, equipment operating status parameters, path process parameters, and deviations is established. For example, extreme deviation scenarios are often accompanied by out-of-tolerance temperature control accuracy in fermenters, indicating that the source of the anomaly is the equipment status. If deviations are concentrated in scenarios with large fluctuations in material viscosity, the source of the anomaly is the material quality. After locating the source of the anomaly, the abnormal simulation data, including deviation distribution and anomaly source information, is compiled and sent to the path generation module to trigger path replanning. This may involve adjusting process parameters to adapt to material fluctuations or changing the equipment matching scheme. Preset thresholds are used to adapt to the personalized stringency of order requirements, balancing quality risks and production efficiency. Causal inference accurately locates the source of the anomaly, making path replanning more targeted and avoiding blind adjustments. Ultimately, replanning ensures the reliability of the bioconversion process, reduces the risk of unqualified finished products, provides a more reliable execution basis for the bioconversion process, and ensures that subsequent physical execution meets the order process requirements.

[0102] Specifically, generating feedforward control commands includes:

[0103] When the simulation confidence level is not less than the preset threshold, the parameter correction amount of the process parameters is calculated based on the fluctuation nature of the quality index and the stability characteristics of the equipment operating status, combined with the simulation data.

[0104] The abrupt change characteristics of reaction conditions are extracted from simulation data. Combined with the shift patterns of metabolic inflection points in similar historical processes and the synergistic effects of chemical components, the time nodes and control parameter thresholds of metabolic inflection points are determined to form metabolic inflection point identifiers.

[0105] Integrate parameter correction values ​​and metabolic inflection point markers to generate feedforward control commands and initiate the biotransformation process.

[0106] Achieving a satisfactory simulation confidence level only indicates that the simulation environment is consistent with actual production. However, the quality index of the mixed sap of yacon and cornus officinalis still exhibits natural fluctuations, including differences in the content of active ingredients and the viscosity of the system. Although the equipment operation is stable, it still has inherent deviations, such as slight baseline offsets in fermenter temperature control and fixed errors in agitator speed. If the original process parameters in the path instruction sequence are directly adopted, the actual reaction state will deviate from expectations due to material fluctuations and equipment deviations. First, the fluctuation characteristics of the quality index in the material quality profile are extracted to clarify the direction of the fluctuations' impact on the fermentation process. For example, high viscosity can easily lead to insufficient mass transfer, requiring adjustment of agitation-related parameters. Then, the stable characteristics of the equipment operation are obtained, including the actual temperature control baseline value of the fermenter, the actual speed output accuracy of the agitator, and the systematic error of the pH sensor.

[0107] By combining pre-execution simulation data, such as the parameter adaptation effects at different stages of the simulation process and the correlation between product generation and parameters, parameter correction amounts are calculated for process parameters. For example, if the mass index shows that the content of active ingredients in the material is low, and considering the simulation data showing that increasing the temperature can promote the generation of active ingredients, and the equipment temperature control baseline is low, the parameter correction amount is to moderately increase the fermentation temperature. If the system viscosity is high, and considering the simulation data showing that optimizing the stirring rate can improve mass transfer, the parameter correction amount is to adjust the stirring rate to a range suitable for the viscosity. The parameter correction amount fully integrates the material fluctuation characteristics, the inherent state of the equipment, and the simulation verification results, so that the process parameters are transformed from theoretically optimal to practically optimal, avoiding reaction deviations caused by objective differences between materials and equipment, and laying a parameter foundation for the precise execution of the biotransformation process.

[0108] The biotransformation process of compound enzymes exhibits metabolic inflection points, such as peak substrate consumption points, abrupt changes in the rate of active ingredient generation, and abrupt changes in the pH of the fermentation system. The reaction mechanisms and parameter requirements before and after these metabolic inflection points differ significantly. For example, before the inflection point, substrate decomposition needs to be promoted, while after, side reactions need to be suppressed. Failure to accurately identify metabolic inflection points can easily lead to delayed or misjudged parameter adjustments, affecting product quality and production efficiency. Simulation data can predict the changing trends of reaction conditions in advance. Combined with historical experience and the synergistic patterns of material components, accurate prediction of metabolic inflection points can be achieved. Abrupt changes in reaction conditions can be extracted from simulation data, including sudden changes in the pH of the fermentation system, sudden increases or decreases in the rate of active ingredient generation, and rapid decreases in substrate concentration. These characteristics are direct signals of metabolic inflection points. Furthermore, by retrieving historical records of metabolic inflection points from similar compound enzyme processes, the shift in the timing of these inflection points can be analyzed, such as the earlier or later occurrence of inflection points under different material batches and equipment conditions.

[0109] Then, by combining the synergistic effects of chemical components in the mixture of yacon and cornus officinalis, including the synergistic relationship between the formation of polysaccharides and flavonoids and the regulatory effect of organic acids on the metabolic process, the correlation between mutation characteristics and metabolic stage transitions was verified. Finally, the specific time node of the metabolic inflection point and the control parameter thresholds before and after the metabolic inflection point were determined and integrated to form a metabolic inflection point identifier. The time node includes a certain period after the start of fermentation, and the control parameter thresholds include the upper limit of temperature before the metabolic inflection point and the pH control range after the metabolic inflection point. Through triple verification of mutation characteristics, historical patterns and component synergistic characteristics of simulation data, the accurate prediction of the metabolic inflection point is achieved, the time window and threshold range of parameter adjustment are clarified, and the improper parameter adjustment caused by the ambiguity of the metabolic inflection point identification is avoided, ensuring the smooth transition of the metabolic process and the stability of product quality.

[0110] The execution of biotransformation processes requires unified and clear pre-control instructions. Integrating scattered parameter corrections and metabolic inflection point markers into standardized instructions ensures that the closed-loop execution module can quickly identify requirements, avoiding execution confusion caused by information dispersion. Simultaneously, as pre-guidance, feedforward control instructions allow the execution module to pre-set reaction conditions and adjustment logic, providing a basis for precise control after process initiation. The calculated parameter corrections for each process parameter are integrated with the determined metabolic inflection point markers, generating feedforward control instructions in a format recognizable by the closed-loop execution module. These instructions clearly indicate the corrected parameters for each process step, the parameter effective time, the specific criteria for judging the metabolic inflection point, and the parameter adjustment rules before and after the inflection point. Logical verification of the feedforward control instructions is performed, such as checking for conflicts between parameter corrections and control parameter thresholds for metabolic inflection points, and ensuring that the time nodes conform to the fermentation sequence, ensuring that the feedforward control instructions are logically consistent.

[0111] After successful verification, the feedforward control command is sent to the closed-loop execution module, triggering the start of the bioconversion process, such as initiating fermenter preheating, material feeding, and equipment operation. The standardized feedforward control command integrates process parameters and metabolic inflection point information, enabling the closed-loop execution module to respond quickly and accurately, avoiding execution deviations. The process start-up and command issuance are synchronized, ensuring that the bioconversion follows the optimized control logic from the initial stage, guaranteeing that the reaction process proceeds as expected, reducing subsequent adjustment costs, and laying the foundation for closed-loop control that generates adjustment amounts by monitoring the substrate concentration change rate, thus ensuring the accuracy and stability of the entire bioconversion process.

[0112] The closed-loop execution module sets the reaction conditions of the bioconversion process according to the feedforward control instructions, and monitors the substrate concentration change rate to generate the adjustment amount of the reaction conditions. After bioconversion, it obtains the quality of the finished product and compares it with the quality index. When it deviates from the order process requirements, it feeds back to adjust the simulation parameters and the reward function of the strategy network.

[0113] Furthermore, the adjustments to the reaction conditions include:

[0114] The reaction conditions of the biotransformation process are set according to the feedforward control instructions, the substrate concentration change rate is monitored to generate adjustment signals, and the metabolic inflection point markers of the feedforward control instructions are combined to determine the current reaction stage.

[0115] Based on the fluctuating nature of the quality index and the stable characteristics of equipment operation, a basic adjustment amount is generated for the adjustment range of the appropriate process parameters.

[0116] If the adjustment direction of the feedforward control command conflicts with that of the adjustment signal, the weights are dynamically allocated according to the simulation confidence level to form a coordinated adjustment amount. If there is no conflict, the adjustment value of the adjustment signal is superimposed on the basic adjustment amount to form a coordinated adjustment amount.

[0117] The synergistic adjustment amount is graded and corrected according to different reaction stages to form the adjustment amount of reaction conditions.

[0118] The fermentation process of compound enzymes exhibits significant stage characteristics. In the early stage, substrate decomposition is the main focus; in the middle stage, the emphasis is on the generation of active ingredients; and in the later stage, the production of byproducts needs to be suppressed. The requirements for reaction conditions differ significantly at each stage. For example, the early stage requires promoting substrate decomposition, while the later stage requires stabilizing the system pH. If the differences between stages are not uniformly adjusted, metabolic imbalances can easily occur. For instance, excessively high temperatures in the early stage can lead to excessively rapid substrate decomposition, resulting in insufficient generation of active ingredients in the later stage. First, the initial reaction conditions for fermentation are set according to feedforward control instructions to initiate the biotransformation process. The substrate concentration change rate is monitored in real time by online detection equipment, including the consumption rate of yacon polysaccharide and the conversion rate of cornus flavonoid precursors. When the substrate concentration change rate fluctuates significantly, a real-time adjustment signal is generated to indicate that the reaction conditions need to be adjusted to adapt to the substrate metabolic rhythm.

[0119] The system retrieves metabolic inflection point markers from the feedforward control commands, such as the time points of substrate consumption peaks and abrupt changes in the rate of bioactive ingredient generation, along with control parameter thresholds. It then compares the real-time substrate concentration change rate with these metabolic inflection point markers to determine if the substrate consumption peak has passed. Combined with the fermentation duration, the system ultimately identifies the current reaction stage, including the substrate decomposition phase, bioactive ingredient generation phase, and fermentation stabilization phase. This triple-judgment logic—feedforward control command presets, real-time monitoring, and metabolic inflection point marker comparison—accurately identifies the fermentation reaction stage, avoiding adjustment deviations caused by stage misjudgment. This lays the foundation for targeted generation of subsequent adjustment amounts, ensuring that the adjustment strategy matches the metabolic needs of each stage.

[0120] The quality index of the mixed yacon and cornus officinalis pulp fluctuates, leading to different metabolic efficiencies under the same reaction conditions. For example, batches with high polysaccharide content require longer decomposition times. Furthermore, while the equipment operates stably, it has inherent characteristics that can cause deviations between actual and theoretical reaction conditions. Without considering these factors, adjustments may exceed the suitable range of process parameters; for example, excessive adjustments may render the equipment unusable. The study aims to extract the fluctuation characteristics of the quality index in the raw material quality profile, analyze the impact of these fluctuations on fermentation process parameters (e.g., increasing the stirring rate to ensure mass transfer when viscosity is high), and obtain the stable characteristics of the equipment's operating status.

[0121] Based on the adjustment range of process parameters at the current reaction stage, such as the allowable temperature adjustment range during substrate decomposition, a basic adjustment amount is generated. If the mass index shows that the polysaccharide content is high and the mass transfer efficiency of the equipment agitator is stable, the basic adjustment amount is to moderately increase the stirring rate and control it within the safe speed range of the equipment. If the viscosity of the system fluctuates greatly due to fiber residue and there is a slight deviation in the temperature control of the fermenter, the basic adjustment amount is to slightly increase the set temperature to compensate for equipment deviation and adapt to the influence of viscosity on heat transfer. The basic adjustment amount fully integrates the material fluctuation characteristics and the inherent state of the equipment to ensure that the adjustment range is within the effective adaptation range of the process parameters. This avoids metabolic lag caused by insufficient adjustment and prevents equipment risks or material metabolic imbalance caused by excessive adjustment, providing a safe and suitable benchmark value for subsequent coordinated adjustments.

[0122] Feedforward control commands are formulated based on pre-execution simulations, emphasizing predictive adjustments, while real-time adjustment signals are generated based on substrate concentration change rates, emphasizing immediate corrections. The two can conflict in adjustment direction due to changes in the scenario. For example, feedforward control commands might require cooling to suppress side reactions, while real-time adjustment signals might require heating due to slow substrate decomposition. Simply choosing one can lead to metabolic biases; ignoring the feedforward could trigger side reactions, while ignoring the real-time signal could result in insufficient substrate decomposition. When there is no conflict, the advantages of both should be combined to enhance the adjustment effect. The adjustment directions of the feedforward control commands and the real-time adjustment signals should be compared to determine if a conflict exists. In case of conflict, the simulation confidence level reflecting the reliability of the feedforward control command is retrieved, and the weights of the two are dynamically allocated. A higher simulation confidence level results in a higher weight for the feedforward control command, and vice versa. The adjustment magnitude is then fused according to the weights to form a coordinated adjustment amount. If there is no conflict, the adjustment value of the real-time adjustment signal is superimposed on the basic adjustment amount. In case of conflict, the weights are dynamically allocated through simulation confidence level to balance the reliability of predictive adjustment and immediate correction, avoiding adjustment errors caused by a single signal dominating. In case of no conflict, the adjustment effect is enhanced by superposition to ensure that the metabolic process accelerates or stabilizes as expected, thereby improving the accuracy and flexibility of the adjustment.

[0123] Different reaction stages exhibit varying sensitivities to process parameter adjustments. For instance, the active ingredient formation stage is more sensitive to temperature fluctuations, with even minor deviations leading to active ingredient degradation. The substrate decomposition stage, on the other hand, is less sensitive to stirring rate adjustments; slightly larger adjustments promote rapid decomposition. Applying a uniform adjustment range directly to the synergistic adjustment could lead to metabolic disturbances in sensitive stages due to excessive adjustments, or inefficiency in non-sensitive stages due to insufficient adjustments. Based on the determined reaction stages, sensitivity levels are assigned to each stage, such as high sensitivity for the active ingredient formation stage, medium sensitivity for the substrate decomposition stage, and low sensitivity for the fermentation stabilization stage. The synergistic adjustment amounts are then graded and corrected, with the high sensitivity level... The adjustment range is narrowed in stages to avoid drastic fluctuations affecting the generation of active ingredients. In the medium-sensitivity stage, the synergistic adjustment amount is maintained or finely adjusted. In the low-sensitivity stage, the adjustment range can be appropriately expanded to accelerate the stabilization of the system. The adjusted amounts form the final reaction conditions after correction, clarifying the specific adjustment values ​​and execution timing of each process parameter. The graded correction adapts the adjustment amount to the metabolic sensitivity of different stages. While ensuring metabolic stability in the sensitive stage, it improves the adjustment efficiency in the non-sensitive stage, avoiding the quality risks or efficiency losses caused by a one-size-fits-all adjustment. Finally, a precise, safe and efficient reaction condition adjustment plan is formed, laying a stable production foundation for obtaining product quality and feedback adjustments after subsequent biotransformation.

[0124] Specifically, the feedback adjustment of simulation parameters includes:

[0125] After biotransformation, the quality of the finished product is obtained and compared with the quality index. When it deviates from the order process requirements, the deviation magnitude and type are analyzed, and the source of deviation is located through feature matching.

[0126] The simulation parameters to be adjusted are determined based on the deviation type of the deviation source, and the initial adjustment amount is generated by combining the parameter adjustment records and deviation magnitudes under the same type of deviation in history.

[0127] The initial adjustment amount is optimized based on the adjustment range constraint of the process parameters. The optimized simulation parameters are then verified through simulation until the quality of the finished product meets the order process requirements.

[0128] The quality of the finished compound enzyme product directly determines whether the order will be met. Its quality indicators may deviate from the order requirements due to material fluctuations, equipment parameter deviations, or unreasonable simulation parameter presets. If deviations are only found without clarifying the cause, adjustments will be blind and inefficient. After bioconversion, the content of yacon polysaccharide and cornus flavonoids in the finished product is detected by high performance liquid chromatography, the pH value is measured by a pH meter, and the clarity is measured by a turbidimeter. Sensory evaluation is combined to obtain taste data. These measured values ​​are compared with the quality index in the material quality profile. If one or more indicators exceed the allowable range, such as the flavonoid content being lower than the target value or the acidity being higher, it is determined to be a deviation from the order requirements. The deviation range and type are analyzed. The deviation range includes how much the flavonoid content is lower than the target and how much the acidity is higher than the standard. The deviation types include insufficient active ingredients, abnormal acidity, and unacceptable clarity.

[0129] By retrieving a feature library of similar deviations from historical production data—for example, insufficient flavonoids are often associated with excessively high fermentation temperatures, and abnormal acidity is often associated with excessively long fermentation times—the source of the deviation is located through feature matching. For instance, if the current deviation type matches the historical characteristics of fermentation temperature deviation leading to degradation of active ingredients, it indicates that the temperature parameters of the fermentation process are preset too high, the raw material pretreatment is not sufficiently broken down, resulting in incomplete extraction, or the equipment stirring rate is too low, leading to uneven material mixing. By accurately analyzing the deviation features and locating the root cause, blind adjustments are avoided, allowing subsequent simulation parameter adjustments to directly target the core problem, thus improving adjustment efficiency and accuracy.

[0130] Different deviation sources correspond to different simulation parameters. For example, insufficient raw material crushing is associated with the simulation values ​​of crushing parameters in raw material pretreatment, while excessively high fermentation temperature is associated with simulation parameters of fermentation temperature. The parameters to be adjusted need to be locked first. Historical adjustment records of similar deviations contain effective adjustment rules verified in practice, including adjusting a certain parameter by a fixed amount under a certain type of deviation to restore quality. Combined with the current deviation range, a more reliable initial adjustment amount can be generated, avoiding trial and error from scratch. Based on the identified deviation source and deviation type, the simulation parameters to be adjusted are determined. If the deviation source is excessively high fermentation temperature leading to flavonoid degradation, then the parameters to be adjusted are the temperature simulation parameters of the fermentation process. If the deviation source is insufficient raw material crushing leading to insufficient polysaccharide extraction, then the parameters to be adjusted are the simulation parameters of crushing speed and crushing time in raw material pretreatment.

[0131] Retrieve parameter adjustment records for similar deviations in historical production and analyze adjustment patterns; combine the current deviation magnitude to generate initial adjustment amounts proportionally or based on trends; lock simulation parameters based on the deviation source to ensure the correct adjustment direction; generate initial adjustment amounts by combining historical records and the current deviation magnitude, drawing on practical experience while adapting to the current scenario, avoiding under- or over-adjustment, and providing a reasonable benchmark for subsequent optimization. The initial adjustment amounts provide a starting point for optimization under process parameter constraints, allowing it to be further refined within a safe and feasible range, ensuring that the adjusted parameters are both effective and in line with actual production.

[0132] The initial adjustment may exceed the allowable range of the actual process parameters. For example, the equipment's maximum cooling capacity may be limited, making the required cooling range impossible to achieve, or the raw material crushing speed adjustment may exceed the equipment's safe speed range. Direct application of these adjustments could lead to production interruptions or equipment damage. Furthermore, it's necessary to verify whether the adjusted simulation parameters truly address the deviation issue. The adjustment range of the process parameters corresponding to the simulation parameters to be adjusted should be clearly defined. The generated initial adjustment should be compared with this range; if it exceeds the range, optimization should be performed. The optimized simulation parameters should be input into the process simulation module to simulate the entire biotransformation process and predict the finished product quality indicators. If the simulation prediction results still do not meet the order requirements... The process involves repeatedly adjusting simulation parameters, optimizing, and simulating. For example, the simulation parameters for fermentation time are further fine-tuned until the simulated finished product quality fully meets the order's process requirements. At this point, the final adjusted simulation parameters are determined. The constraint of the adjustment range of process parameters ensures the actual feasibility of the adjustment amount and avoids the risk of equipment exceeding limits. Simulation verification confirms the adjustment effect in advance through simulation, avoiding repeated trial and error in actual production, reducing raw material waste and time loss, and ultimately forming an effective and feasible simulation parameter adjustment plan. This provides a more accurate simulation parameter basis for subsequent processing, ensures the closed-loop optimization capability of the entire production control system, and fundamentally reduces the risk of finished product quality deviation.

[0133] Specifically, the reward function of the feedback adjustment policy network includes:

[0134] When the quality of the finished product deviates from the order process requirements, analyze the magnitude of the quality deviation, calculate the deviation between actual energy consumption and energy consumption target and between actual efficiency and efficiency target, and correlate the difference between process adaptability and actual production adaptability to help determine the root cause of the deviation.

[0135] If the deviation is in quality, the weight of the constraint penalty item corresponding to the order process requirements is optimized; if the deviation is in energy consumption and efficiency, the correlation between the weighted coefficients of the process fitness mapping and the reward function is corrected.

[0136] By applying weight constraints to the adjustment range of process parameters and dynamically calibrating the adjustment range based on the deviation improvement rate, the adjusted reward function is input into the strategy network to verify the adaptability of the cumulative reward expectation of the newly generated candidate process path, so as to provide feedback to adjust the reward function of the strategy network.

[0137] The reward function of the strategy network is the basis for evaluating candidate process paths. If the reward function design is out of sync with actual production, such as excessively lenient penalties for quality constraints or unreasonable weighting coefficients for process adaptability, it will lead to quality, energy consumption, or efficiency deviations in the actual execution of high-reward paths output by the strategy network. When the quality of the finished product deviates from the order requirements, the magnitude of the quality deviation is quantified, such as the proportion of polysaccharide content lower than the target value and the degree to which acidity exceeds the standard range. The deviation between actual production energy consumption and the preset energy consumption target, as well as the deviation between actual production efficiency and the efficiency target, are calculated simultaneously. The process in the material quality profile is retrieved. Adaptability is assessed by comparing the predicted material and process compatibility potential with the actual material and process compatibility performance in production. This includes analyzing differences such as high process adaptability but low actual active ingredient extraction rate. If the difference is significant, it indicates that the weighting logic of process adaptability in the reward function is distorted. If quality deviation is dominant, it indicates insufficient weight of order constraint penalty terms. If energy consumption or efficiency deviation is dominant, it indicates an unreasonable mapping relationship between process adaptability and reward coefficient. Through multi-dimensional deviation analysis and root cause correlation, the defect type of the reward function can be accurately located, avoiding blind adjustments and providing a clear basis for subsequent targeted optimization of the reward function.

[0138] The constraint penalty terms of order process requirements in the reward function directly affect the control over quality. If the weight is too low, it will be difficult to constrain quality deviations. The correlation between process adaptability and weighting coefficient directly affects the accuracy of energy consumption and efficiency assessment. If the correlation is distorted, high-energy-consuming paths will be misjudged as superior. If the root cause of the deviation is quality deviation, including insufficient content of active ingredients or insufficient clarification, the weight of the constraint penalty terms of the corresponding order process requirements should be optimized. For example, functional enzyme orders have strict requirements for flavonoid content. If there is a deviation of insufficient flavonoid content, the weight of the constraint penalty term of flavonoid content deviation from the target in the reward function should be increased so that the schemes that ignore flavonoid retention in subsequent candidate paths will receive lower rewards.

[0139] If the root cause of the deviation is energy consumption and efficiency discrepancies, such as actual energy consumption far exceeding the target or excessively long production cycles, then the correlation between process fitness mapping and weighted coefficients should be corrected. If the actual energy consumption of paths with high process fitness is too high, it indicates that the energy consumption optimization weight in the original mapping relationship is insufficient, and the mapping logic needs to be adjusted so that the weighted coefficients of process fitness focus more on energy consumption and efficiency indicators. For example, increasing energy consumption can reduce its positive contribution to the weighted coefficients. For the constraint penalty term for quality deviation optimization, the control of the reward function over quality indicators should be strengthened. The weighted correlation for energy consumption / efficiency deviation should be corrected to improve the accuracy of the reward function's evaluation of economic indicators, making the evaluation logic of the reward function more consistent with the actual production target, and ensuring that subsequent optimization is both in line with the actual process and can effectively improve the deviation.

[0140] The parameter adjustment of the reward function should avoid extremes. For example, excessively high weights of constraint penalty terms may lead to excessively low rewards for all paths, or biased weighting coefficients may cause an imbalance in the evaluation. Constraints should be applied in conjunction with the adjustment range of process parameters, and the adjustment magnitude should be dynamically calibrated based on the improvement effect of the deviation. For example, if the deviation does not improve after adjustment, the magnitude should be increased to ensure the effectiveness of the adjustment. The adjustment effect should be verified through the actual output of the policy network to form a closed-loop optimization. Constraints should be applied to the weight parameters of the reward function in conjunction with the adjustment range of process parameters for processing snow lotus fruit and dogwood, including the reasonable range of fermentation temperature and the equipment limit of stirring rate. For example, the weight of constraint penalty terms should not exceed a certain threshold to avoid the absence of feasible paths due to excessive penalties. The value range of the weighting coefficients should be matched with the energy consumption capacity of the equipment. Based on historical adjustment experience and current deviation characteristics, the expected deviation improvement rate should be set, including the proportion that the quality deviation should be reduced after adjustment. If the improvement rate does not meet expectations after the initial adjustment, the adjustment magnitude should be dynamically calibrated, for example, by increasing the weight of constraint penalty terms or correcting the correlation strength of the weighting coefficients.

[0141] The final adjusted reward function is input into the strategy network to generate new candidate process paths. The expected cumulative reward is calculated and compared with the actual production performance of the path in terms of quality, energy consumption, and efficiency. If the expected cumulative reward significantly improves the fit with actual performance (e.g., a higher actual achievement rate for high-reward paths), the reward function adjustment is confirmed as effective, completing the feedback adjustment of the strategy network. Constraints on the adjustment range of process parameters prevent extreme adjustments to the reward function parameters, ensuring the rationality of the evaluation logic. Dynamic calibration matches the adjustment magnitude with the deviation improvement needs, improving adjustment efficiency. Verification is performed through the strategy network output, forming a closed-loop optimization that ensures the reward function continuously adapts to the actual production scenario. Ultimately, the candidate paths generated by the strategy network more accurately match order requirements and economic indicators, improving the adaptability of the entire chain from path selection and instruction generation to production execution. This fundamentally reduces the deviation risks in finished product quality, energy consumption, and efficiency, ensuring the stability and economy of industrial production.

Claims

1. An industrial production process control system, characterized in that, include: The module includes a material sensing module, a path generation module, a process simulation module, and a closed-loop execution module. The material sensing module acquires the physical form, chemical composition and surface defects of the material, and generates a material quality profile including quality index and process adaptability after extracting features through a deep convolutional neural network. The path generation module receives material quality profiles and order process requirements, reads equipment operating status from the processing resource pool, maps process fitness to weighted coefficients of reward functions with energy consumption and efficiency as objectives, calculates the cumulative reward expectation of candidate process paths based on the policy network, and generates a path instruction sequence after screening. Calculating the cumulative expected reward for the candidate process path includes: Based on the process adaptability of the order process requirements, the equipment operating status of the processing resource pool, and the material quality profile, candidate process paths are generated, including process sequence, equipment matching, and process parameters. With energy consumption and efficiency as the objectives, the process adaptability is mapped to the weighted coefficients of the reward function, and the constraint penalty terms of the order process requirements are added; Based on the policy network, the correlation characteristics of energy consumption and efficiency between processes in the candidate process path are captured through the attention mechanism. Combined with the time-series characteristics of equipment operation status, the instantaneous reward value of the candidate process path is calculated online, and the cumulative reward expectation is obtained by accumulating based on the time-series discount factor. Before physical execution, the process simulation module simulates the biotransformation process in the path instruction sequence. When the simulation confidence is less than a preset threshold, it sends abnormal simulation data and triggers path replanning. Otherwise, it generates feedforward control instructions including parameter correction and metabolic inflection point markers and starts the biotransformation process. The closed-loop execution module sets the reaction conditions of the bioconversion process according to the feedforward control instructions, and monitors the substrate concentration change rate to generate the adjustment amount of the reaction conditions. After bioconversion, it obtains the quality of the finished product and compares it with the quality index. When it deviates from the order process requirements, it feeds back to adjust the simulation parameters and the reward function of the strategy network.

2. The industrial production process control system as described in claim 1, characterized in that, The generated material quality profile includes: The physical morphology, chemical composition, and surface defects of the material are obtained, and spatiotemporal alignment and noise reduction are performed. Based on deep convolutional neural networks, spatial distribution features of physical morphology, synergistic features of chemical composition and topological correlation features of surface defects are extracted hierarchically. The hierarchical features are then fused through a cross-modal attention mechanism to generate fused features that include basic features and process-sensitive features. The process scenario weights are adjusted according to the sensitivity of different order processes to materials. The basic quality index is obtained by weighting the basic features and the benchmark weights, and the process sensitivity index is obtained by weighting the process sensitive features and the process scenario weights. The basic quality index and the process sensitivity index are added together to form the quality index. Meta-learning is used to calculate the feature distance between the fused features and the order process, and the process fitness is generated by weighting. The quality index and process fitness are calibrated, and a material quality profile is generated when the calibration deviation is less than the deviation threshold.

3. The industrial production process control system as described in claim 2, characterized in that, The generated path instruction sequence includes: Candidate paths are selected by screening based on the cumulative reward expectation of the candidate process paths and by combining the matching verification of process adaptability and equipment operating status. Based on the fluctuating nature of the quality index, the adjustment range of process parameters for each process in the selected path is dynamically configured, and the process connection sequence of the selected path is optimized based on the correlation characteristics of energy consumption and efficiency between processes output by the strategy network. The optimized candidate paths are validated, and the validated candidate paths are converted into equipment execution instructions that include process sequence, process parameters and adjustment range, generating a path instruction sequence.

4. The industrial production process control system as described in claim 3, characterized in that, The trigger path replanning includes: Before physical execution, simulation is performed by combining the process parameters of the bioconversion process in the path instruction sequence, the fluctuation characteristics of the quality index in the material quality profile, and the equipment operating status. Multiple simulation scenarios are generated through Monte Carlo simulation. For each simulation scenario, the deviation distribution between the reaction conditions in the biotransformation process and the measured values ​​of similar historical processes is calculated to form the simulation confidence level. The preset threshold is dynamically adjusted according to the stringency of the order process requirements. When the simulation confidence level is less than the preset threshold, the abnormal source is located by causal inference and abnormal simulation data is sent to trigger path replanning.

5. The industrial production process control system as described in claim 4, characterized in that, The generated feedforward control commands include: When the simulation confidence level is not less than the preset threshold, the parameter correction amount of the process parameters is calculated based on the fluctuation nature of the quality index and the stability characteristics of the equipment operating status, combined with the simulation data. The abrupt change characteristics of reaction conditions are extracted from simulation data. Combined with the shift patterns of metabolic inflection points in similar historical processes and the synergistic effects of chemical components, the time nodes and control parameter thresholds of metabolic inflection points are determined to form metabolic inflection point identifiers. Integrate parameter correction values ​​and metabolic inflection point markers to generate feedforward control commands and initiate the biotransformation process.

6. The industrial production process control system as described in claim 5, characterized in that, The adjustment amounts for the reaction conditions include: The reaction conditions of the biotransformation process are set according to the feedforward control instructions, the substrate concentration change rate is monitored to generate adjustment signals, and the metabolic inflection point markers of the feedforward control instructions are combined to determine the current reaction stage. Based on the fluctuating nature of the quality index and the stable characteristics of equipment operation, a basic adjustment amount is generated for the adjustment range of the appropriate process parameters. If the adjustment direction of the feedforward control command conflicts with that of the adjustment signal, the weights are dynamically allocated according to the simulation confidence level to form a coordinated adjustment amount. If there is no conflict, the adjustment value of the adjustment signal is superimposed on the basic adjustment amount to form a coordinated adjustment amount. The synergistic adjustment amount is graded and corrected according to different reaction stages to form the adjustment amount of reaction conditions.

7. The industrial production process control system as described in claim 6, characterized in that, The feedback adjustment of simulation parameters includes: After biotransformation, the quality of the finished product is obtained and compared with the quality index. When it deviates from the order process requirements, the deviation magnitude and type are analyzed, and the source of deviation is located through feature matching. The simulation parameters to be adjusted are determined based on the deviation type of the deviation source, and the initial adjustment amount is generated by combining the parameter adjustment records and deviation magnitudes under the same type of deviation in history. The initial adjustment amount is optimized based on the adjustment range constraint of the process parameters. The optimized simulation parameters are then verified through simulation until the quality of the finished product meets the order process requirements.

8. The industrial production process control system as described in claim 7, characterized in that, The feedback adjustment of the reward function of the policy network includes: When the quality of the finished product deviates from the order process requirements, analyze the magnitude of the quality deviation, calculate the deviation between actual energy consumption and energy consumption target and between actual efficiency and efficiency target, and correlate the difference between process adaptability and actual production adaptability to help determine the root cause of the deviation. If the deviation is in quality, the weight of the constraint penalty item corresponding to the order process requirements is optimized; if the deviation is in energy consumption and efficiency, the correlation between the weighted coefficients of the process fitness mapping and the reward function is corrected. By applying weight constraints to the adjustment range of process parameters and dynamically calibrating the adjustment range based on the deviation improvement rate, the adjusted reward function is input into the strategy network to verify the adaptability of the cumulative reward expectation of the newly generated candidate process path, so as to provide feedback to adjust the reward function of the strategy network.

Images