Agent-based industrial malt manufacturing system and automatically optimizing operation of industrial malt manufacturing equipment and method thereof

Abstract

An automated industrial malt manufacturing device (11) and an agent- and digital twin-based malt manufacturing system (1) are proposed for the automated operation of the industrial malt manufacturing device (11) of the agent- and digital twin-based malt manufacturing system (1). The malt manufacturing device (11) converts supplied grains (2, 21, 21i) into output malt (3). The malt manufacturing device (11) includes: a soaking device (111) for wetting the supplied grains (2, 21, 21i) until the moisture content of the supplied grains (2, 21, 21i) reaches a certain level (111). 1) Reaching a defined moisture content level (1112); a germination device (112) configured to receive moistened grains (22) after being moistened by a soaking device (111), for germinating the moistened grains (22) by activating enzymes in the moistened grains (22) and converting the moistened grains (22) into germinated green malt (23); and a drying kiln (113) configured to receive germinated grains (23) after being germinated by the germination device (112), for drying the germinated green malt (23) to a defined moisture content level (1131). The system (1) based on intelligent agents and digital twins includes a digital twin structure (12) providing a virtual digital representation of the physical malt manufacturing equipment (11). In addition, it includes sensors and / or measuring devices (13) associated with the physical malt manufacturing equipment (11), which transmit measured sensory parameter values ​​(131), wherein the system (1) dynamically monitors the transmitted sensory parameter values ​​(131) and updates the digital twin (12) based on the parameter values.

Description

Technical Field

[0001] This invention relates to the field of malt manufacturing equipment and processes. In particular, it relates to the industrial automation of malt manufacturing equipment for germinating and kiln-drying grains. The malt manufacturing equipment may also include a device for soaking. Specifically, this disclosure relates to an automatically operated and manipulated malt manufacturing apparatus that allows product batches (production quantities per batch) to be automatically adjusted, specifically increased, and / or optimized in a stepwise manner. The malt manufacturing equipment is used to produce malt (a higher-value raw material) from a grain feedstock. The malt is then further used in the manufacture of products such as beer, distillates, or for use, for example, in the food industry. Generally, this invention relates to industrial instruments for malting, and particularly to “integrated” instruments and malt manufacturing equipment capable of soaking, germinating, kiln-drying, and rotating grains during the malt manufacturing process. Background Technology

[0002] Malting is the term used to prepare brewing ingredients, employing the controlled germination of grains in moist air. Barley is generally the preferred grain for malting; however, other grains such as wheat, rye, sorghum, millet, triticale, or oats can be malted and subsequently used in brewing, distillation, or food production. Barley is the most common grain used to produce malt for brewing because of its high starch-to-protein ratio and adherent outer shell, which contributes to economical yields and ease of processing in brewing and for this purpose produces the characteristic flavors associated with malt. Malting aims to transform or modify the physical structure of barley grains and allow for the synthesis or activation of a range of enzymes, making the final product, malt, more readily usable in subsequent stages of brewing, distillation, or food production. During the malting process, the production and / or release of hydrolytic enzymes are maximized, resulting in cell wall degradation and protein dissolution with minimal starch breakdown. To achieve this, malting aims to accelerate germination and slow down germ growth—both inherently conflicting activities. Any sprouting or root growth that occurs during the malting process is physically removed from the final product prior to storage and delivery, and thus minimizing germ growth reduces losses that occur during the process. The final product of malting, malt, is physically similar to the original barley grain, but is brittle when ground, reflecting the complex biochemical changes that occur during the malting process.

[0003] For example, in the malting process of grains used to produce beer or whiskey, the grains are typically first soaked when water is added, and then germinated under controlled temperature and moisture conditions, being turned at regular intervals during the process. To stop the germination process and stabilize the malt, it is then dried using a supply of hot air. This process is also known as kiln drying. For this purpose, different sizes of equipment are provided depending on the production volume. To begin the malting process, water is added to the grains (barley, wheat, rye, etc.) to overcome dormancy, causing the grains to begin germination due to water absorption. Malt makers refer to this first of the three process steps in total during malting as "soaking."

[0004] Soaking begins with a wet stage, typically in a cylindrical-conical stainless steel container called a soaking tank. In this container, the grains are submerged in 15... Up to 20 At a water temperature of [temperature missing], and using forced ventilation to maintain movement. If the product is not circulated during the wet soaking period, the grains may die due to lack of oxygen. After approximately three to five hours, the water is drained and the first dry soak begins. This occurs through ventilation, in the form of removing accumulated carbon dioxide via suction from a radial fan. The drying phase lasts approximately 10 hours, followed by another shorter wet phase, and then another drying phase. Once the grains have reached approximately 44 [units missing], [the process continues]. The moisture content (after about 24 hours, depending on the grain) is determined, and the second germination process begins, germination.

[0005] Germination takes place on a kiln bed, where the grains are transferred after the soaking process. The grains remain there for approximately four to six days, depending on the grain, variety, year, and growing region. After approximately 15 days of germination, the grains are then processed using a run-of-flow fan. Up to 20 Under continuous cooling and ventilation with humidified air, the grains can continue to grow. During germination, the skeletal material that retains the starch cells is broken down, and the grains are thus opened. Enzymes that can convert starch into sugars are also formed during germination. Once the grains have been fully opened, the growth process must be stopped by drying in the kiln during the third and final process step.

[0006] By increasing the air volume flow rate and raising the temperature to approximately 50 degrees Celsius. Up to 65 It takes approximately 14 hours to start the kiln drying process. Then use approximately 80... Up to 85 The grains are dried at even higher temperatures, which should produce color and aroma. After about 6 hours, the drying process in the kiln is stopped by cooling with fresh air, thus completing the malting process. For example, these basic malt-making processes are described in the following publications: by Narziss, L., "Malz [Malt]", in Heiss, R. (ed.), "Food Technology: Biotechnological, Chemical, Mechanical, and Thermal Methods of Food Processing", Springer Berlin Heidelberg, 2013; and by Narziss, L., "From Raw Materials to Cold Malt Wheat – Developments Over the Past 25 Years, Newsletter from the German Association of Master Brewers and Master Malters" Raw Material to the ColdWort-Developments of the Last 25 Years, Newsletter of the German MasterBrewers and Master Maltsters Association], Issue 2, May 2018.

[0007] WO2013 / 044984A1 describes an apparatus and method for soaking, germinating, fermenting, and / or combinations thereof in grains, wherein the apparatus comprises a container having at least one plate that can be mounted within the container and has at least one opening for supplying and / or removing fluid. EP2336458A1 discloses a circular container, particularly a germination box or kiln in a malting house, and a method for its manufacture. Malting equipment is also known from documents DE1206835B, US Patent No. 2,500,775A, CN208562299U, and DE2656365A1. Small-scale malting equipment differs from industrial malting equipment in that it is designed for smaller annual production capacity. The production capacity for small-scale malting equipment is limited to the range of approximately 1 tonne / batch to 50 tonnes / batch. Small-scale malting equipment for testing and educational purposes has a production capacity of less than 1 tonne / batch.

[0008] In the prior art, there are three types of malt manufacturing systems: one-chamber systems, two-chamber systems, and three-chamber systems. A one-chamber system should be understood as meaning that the three process steps of soaking, germination, and kiln drying are carried out in one unit. In the case of Central European barley, this totals 7 days (1 day of soaking, 5 days of germination, and 1 day of kiln drying). Therefore, a maximum of 52 batches can be produced in 365 process days per year (365 days / 7 days / batch). During the malt manufacturing process, the grain remains in the same unit. Since all three process steps (soaking, germination, and kiln drying) are carried out in one unit, only one batch can be produced. The shape of a one-chamber system can include both rectangular boxes and cylindrical tanks. Both have a kiln bed through which processing air is supplied to the grain during malt manufacturing. In a two-chamber system, soaking and germination / kiln drying are carried out in different chambers. Therefore, the peripheral equipment for the soaking process step is independent of the germination and kiln drying process steps, which are carried out in separate shared units. However, the soaked grain must be transferred from the soaking unit to the germination / kiln drying unit. Therefore, approximately 61 batches can be produced in 365 process days per year (365 days / 6 days / batch). This is possible because soaking can be performed again simultaneously in the soaking unit during the transition from germination to kiln drying, and thus no additional day for soaking is lost. Kiln drying and soaking can occur in parallel. In a two-chamber system, soaking takes place in a cylindrical-conical tank and is then transferred to a combined germination / kiln drying unit. This germination / kiln drying unit can include rollers that rotate during germination to turn the product, or it can include a rectangular box equipped with a rotating machine. A circular box with a rotating machine is also possible. In a three-chamber system, soaking, germination, and kiln drying are separate and independent of each other in terms of peripheral equipment. Therefore, approximately 73 batches can be produced in 365 process days per year (365 days / 5 days / batch). Because after the germination device is unloaded into the kiln drying unit, it can be refilled with soaked material from the soaking tank. Therefore, soaking, germination, and kiln drying can occur in parallel. The three-chamber system includes a cylindrical-conical soaking tank and includes a rectangular germination box and a rectangular kiln, or a rectangular germination box and a circular kiln, or a circular germination box and a circular kiln.

[0009] Malt production of barley should provide high yields in the field, high extract yields in the brewery, and numerous enzymes such as... amylase, amylase, protease and For glucanase to have suitable activity, it should be as free of harmful microorganisms as possible, such as Fusarium fungi. Therefore, there is always a technological need to improve and optimize the malting process, especially automated malting equipment. In the existing technology, barley varieties that meet many of the required qualifications have been developed quite successfully. Traditional breeding will remain important in the future, but in addition, transgenic barley offers new possibilities where the malting process requires further adaptation and optimization. For example, genes encoding certain desired enzymes can be integrated into the barley genome. Lactic acid fermentation cultures are commonly used in several areas of the food industry. In malting, they appear to offer new possibilities for controlling the microbiome by preventing the growth of harmful microorganisms and by supporting beneficial microorganisms. It should be noted that the conversion of barley to transgenic varieties has been more difficult than that of some other grains. Successful conversion requires both cell and tissue culture. Therefore, in the case of barley, the development of gene transfer technology and tissue culture systems has been parallel. Existing technologies cover both cell and tissue culture, protoplast isolation and regeneration, and transient and stable conversion of tissue cultures. Another technical challenge is that the characteristics of new genetically modified barley may not always resemble those of conventional barley varieties. Therefore, there is a need for technologies that enable autonomous, optimized manipulation to accommodate the new characteristics of the barley being supplied.

[0010] Barley and malt can pose additional problems for brewers. Severe Fusarium contamination of barley produced by malt can lead to the formation of deoxynivalenol (DON) and other mycotoxins. As a water-soluble compound, DON is washed away during barley soaking, but it is produced by Fusarium fungi during germination. DON is not removed or destroyed during the brewing process. Fusarium contamination can also cause so-called beer guzzling, which means an immediate, rapid, and uncontrolled spontaneous over-foaming upon opening the bottle or can. Fusarium graminearum, Fusarium graminearum, and Fusarium cladosporidis are active guzzling inducers. The production of mycotoxins can occur concurrently with the production of the components that cause guzzling. Strict control and monitoring of imported barley can be crucial. However, sometimes there simply isn't enough high-quality barley available in certain regions. Microbiome management in a way that prevents harmful organisms and favors neutral or beneficial ones can minimize the risks posed by microbial contamination of barley. The method involves using lactic acid bacteria or *E. repens* as starter cultures in malting to reduce fungal contamination and improve malt quality. Regardless of the natural variations in the barley microbiota, adding starter cultures ensures high malt quality. The impact of lactic acid bacteria starter cultures on the malting process is based on the microbial compounds they produce, as well as their other properties such as enzyme activity. However, the use of lactic acid bacteria starter cultures in malting requires additional control and optimization. When new technologies are applied to the malting process, ensuring high beer quality is paramount. Summary of the Invention

[0011] The object of this invention is to provide an industrialized automated malt manufacturing system and method for the automated control and / or optimized control of industrial malt manufacturing processes. There is an unmet need in the art for a single instrument and a self-optimizing malt manufacturing system capable of performing, operating, manipulating, and autonomously regulating all steps of a controlled malt manufacturing process. One object of this invention is to overcome these needs using a novel, inventive system and method.

[0012] According to the invention, these objectives are achieved in particular by the features of the independent claims. Furthermore, other advantageous embodiments can be derived from the dependent claims and the related description.

[0013] Specifically, these objectives are achieved by the present invention, wherein during the automated operation of an industrial malting equipment in a malting manufacturing system based on intelligent agents and digital twins, the malting equipment converts supplied grains into output malt, wherein the malting equipment includes: a soaking device for wetting the supplied grains until the moisture content of the supplied grains reaches a predetermined moisture content level; and a germination device configured to receive the wetted grains after wetting by the soaking device, for germinating the wetted grains by activating enzymes in the wetted grains and converting the wetted grains into germinated green wheat. The system includes malt; and a drying kiln configured to receive germinated grains after germination via germination equipment, for drying the germinated green malt to a defined moisture content level. The agent- and digital twin-based system includes a digital twin structure providing a virtual digital representation of the physical malt manufacturing equipment and sensors and / or measuring devices associated with the physical malt manufacturing equipment, which transmit measured sensory parameter values. The agent- and digital twin-based malt manufacturing system includes a data monitoring and aggregation engine that, through a base... A digital twin is dynamically monitored to update sensory parameter values, wherein the digital twin includes at least one digital agent and a digital environment that captures characteristic parameters of a malt manufacturing device. The digital agent autonomously performs actions on at least a predefined area or unit of the digital environment. The digital agent includes a monitoring unit for capturing the state of at least one unit of the digital environment of the malt manufacturing device, which includes the characteristic parameters of the malt manufacturing device and / or the transmitted sensory parameter values ​​of at least one unit of the digital environment. The digital agent generates one or more actions to be applied to the digital environment and the malt manufacturing device, respectively. As the digital agent acts on the digital environment and the malt manufacturing device, the states of the digital environment and the malt manufacturing device evolve to subsequent states, respectively. Upon completion of an action, at least one characteristic parameter value and / or sensory parameter value changes between the subsequent state and the previous state. Based on the changed characteristic parameter value and / or sensory parameter value, the digital environment provides a reward to the digital agent. The digital agent autonomously maximizes the reward by favoring actions with higher rewards, thereby being sequentially enhanced.

[0014] This invention has the particular advantage of allowing for additional energy consumption per ton of output material in malt manufacturing equipment and The system of this invention allows for simultaneous interaction with multiple environments, which enables improved stability and, additionally, reduces interaction time. The system allows for stability through the use of data collected from a replay buffer containing previously gathered experience. The off-policy algorithmic architecture used allows for the reuse of experience, i.e., based on a set of measurement parameters and automatically learning from a diverse set of interactions, resulting in faster convergence and improved exploration capabilities. The digital twin of this invention has improved efficiency due to the ability to parallelize in the form of a tensor comprising a measurement matrix, where each row corresponds to a state or action of a different environment. As shown in the table below, the system of this invention provides an AI-driven and optimized operating mode for drying malt based on corresponding optimized automated fan speed control, significantly saving energy consumption during the malt manufacturing process.

[0015]

[0016] exist Figure 3 The invention demonstrates how to automatically manipulate energy consumption per ton using AI-driven optimization based on digital twins and agents. The three basic steps to reduce [something]. Attached Figure Description

[0017] The invention will be explained in more detail by way of example with reference to the accompanying drawings, in which:

[0018] Figure 1a A block diagram of an agent- and digital twin-based malt manufacturing system 1 is shown as an example illustrating the automated operation of an industrial malt manufacturing device 11 for use in an agent- and digital twin-based malt manufacturing system 1 according to the present invention. The agent- and digital twin-based system 1 includes a digital twin structure 12 that provides a virtual digital representation of the physical malt manufacturing device 11 and sensors and / or measuring devices 13 associated with the physical malt manufacturing device 11, the sensors and / or measuring devices 13 transmitting measured sensory parameter values ​​131.

[0019] Figure 1b A block diagram of an agent- and digital twin-based fermentation system 1 is shown as an example illustrating the automated operation of an industrial fermentation device 11 for use in an agent- and digital twin-based fermentation system 1 according to the present invention. The agent- and digital twin-based system 1 includes a digital twin structure 12 that provides a virtual digital representation of the physical fermentation device 11 and sensors and / or measuring devices 13 associated with the physical fermentation device 11, the sensors and / or measuring devices 13 transmitting measured sensory parameter values ​​131.

[0020] Figure 2A block diagram illustrating the development of a digital engine based on the agent-based virtual physical system 1 of the present invention is shown as an example.

[0021] Figure 3 Exemplary examples illustrate the use of AI-driven optimizations based on digital twins and agents of the present invention to automatically manipulate energy consumption per ton and The diagram shows the three steps to reduce [something].

[0022] Figure 4 A diagram illustrating, exemplarily, demonstrates the application of reinforcement learning by digital engine 123. (Policy) It is a (potentially random) function of state s, which returns the action to be performed in state s. After performing an action in state s, the reward r is received. The environment is now in state s. It should be noted that the functions for controlling the state, actions, and rewards can be stochastic.

[0023] Figure 5 A diagram illustrating an exemplary step of the simulation is shown, which may include, for example,: (1) the agent reads the state of the environment (temperature = 25°C). (2) The agent generates an action to be applied to the environment (increase temperature), (3) the action is applied to the environment, which returns a reward (+1), and (4) the agent is now in a new state (temperature = 26). In this system 1, one step can, for example, correspond to one minute in the simulation.

[0024] Figure 6 A diagram is shown as an example illustrating an external air temperature and humidity simulation provided by the Vasicek model structure, as implemented by the present invention.

[0025] Figure 7 An exemplary diagram illustrates how a digital agent 121 learns to minimize power consumption, even if this results in continuous operation exceeding 30 hours. This could be caused by misspecification of rewards. To overcome this problem, rewards could, for example, include a more explicit reference to moisture.

[0026] Figure 8 A graph is shown exemplarily illustrating the preferred choice of the TD3 algorithm with respect to the 22 / 212 moisture reward. In this case, the learning curve of TD3 (Dual Delay Deep Deterministic Policy Gradient) shows that the algorithm, if chosen, is learning how to minimize moisture.

[0027] Figure 9A graph is shown exemplarily illustrating how digital agent 121 learns to apply the same actions (maximum temperature, maximum fan speed, no reverted air) invariantly. However, further penalties for energy consumption result in other constant actions. It should be noted that other algorithms besides TD3 that utilize entropy to ensure non-constant behavior (A2C, SAC) also converge to constant actions. Furthermore, changes in the relative weights of rewards may lead to constant actions; however, this is not the desired effect.

[0028] Figure 10 An exemplary diagram is shown illustrating the net working duration extrapolated by fresh air control.

[0029] Figures 11a to 11c An exemplary diagram is shown illustrating that net working duration is generally independent of the following: (i) initial moisture 212 ( Figure 11a ), (ii) dry weight 214 ( Figure 11b (ii) average ambient temperature 413 ( Figure 11c ).

[0030] Figure 12 A graph is shown as an example illustrating the determination of the appropriate net working duration. For example, it can be measured as by... Figure 12 The distribution shown is shown. In Figure 12 In the example, the net job duration distribution indicates that 36 hours is a reasonable hard threshold for drying time and should be adjusted accordingly.

[0031] Figure 13 A diagram illustrating an exemplary implementation of temperature scheduling is shown, where, for example, a low-pass filter (HP filter + natural spline) can then be used to smooth KNN predictions.

[0032] Figure 14 A diagram is shown illustrating an exemplary implementation of a smoothing model, which can be constructed, for example, to fit the estimated moving standard deviation of the temperature schedule.

[0033] Figure 15 A graph illustrating the reward function is shown as an example; this reward function can then be modified specifically to account for temperature scheduling. For instance, the heater power term can be discarded because it has little impact on the total reward and is implicit in the temperature scheduling term.

[0034] Figure 16 The diagram illustrates, for example, one of the technical problems that may arise in this context, where modeling may lead to unrealistic scenarios (increased embellishment) in the simulation.

[0035] Figure 17A diagram is shown as an example illustrating unstable actions caused by policy collapse.

[0036] Figure 18 A diagram is shown illustrating an exemplary application of Monte Carlo MPC, which allows the digital twin 12 of the present invention to parallelize a large number of environments 122. Therefore, the framework of the present invention is developed to allow the implementation of Monte Carlo MPC (MC-MPC): (1) using the techniques of the present invention, sampling a batch of trajectories of N actions at each step; (ii) applying each trajectory in parallel to different environments; (3) evaluating the trajectories according to a total cost function; (4) the first action of the optimal trajectory is finally applied to environment 122; (5) due to parallelization, the technical advantage of the system of the present invention is that MC-MPC is not as computationally expensive as analytical MPC; and (6) the batch size of all experiments can be set, for example, equally, for example, to 10K (therefore, at each step, 10K possible future trajectories are sampled).

[0037] Figure 19 A diagram is shown illustrating an application of an evolutionary strategy, which can also be implemented to find the optimal trajectory. For example, (1) a first set (10K) of random trajectories can be sampled, (2) the trajectories can be evaluated, (3) the optimal trajectory can be stored and copied multiple times (10K), (4) the copy of the optimal trajectory can be perturbed with random noise, and (5) points 2 to 4 can be iterated until convergence. Detailed Implementation

[0038] Figure 1a The architecture of a possible implementation of an agent- and digital twin-based malt manufacturing system 1 is schematically shown for the automated operation of an industrial malt manufacturing equipment 11. Figure 2An exemplary digital engine 123 developed using the proposed agent-based virtual physical system 1 is illustrated. Malt manufacturing equipment 11 converts supplied grains 2, 21, 21i into output malt 3. Malt manufacturing equipment 11 includes: a soaking device 111 for wetting the supplied grains 2, 21, 21i until the moisture content 1111 of the supplied grains 2, 21, 21i reaches a predetermined moisture content level 1112; a germination device 112 configured to receive the wetted grains 22 after wetting by the soaking device 111, for germinating the wetted grains 22 by activating enzymes in the wetted grains 22 and converting the wetted grains 22 into germinated green malt 23; and a drying kiln 113 configured to receive the germinated grains 23 after germination by the germination device 112, for drying the germinated green malt 23 to a predetermined moisture content level 1131.

[0039] The system 1 based on intelligent agents and digital twins also includes a digital twin 12 that provides a virtual digital representation of the physical malt manufacturing equipment 11, and sensors and / or measuring devices 13 associated with the physical malt manufacturing equipment 11 that transmit measured sensory parameter values ​​131.

[0040] As a variation of the implementation, the agent-based digital twin system 1 may, for example, include more than one digital agent 121, i.e., at least two digital agents 121, which interact through communication and act on the digital environment 122 and the malt manufacturing device 11 respectively. Each digital agent 121 has different units 1212 defined by different interaction characteristics on the digital environment 122 and the malt manufacturing device 11. The at least two digital agents 121 may be linked or connected to each other, for example, through operation sequence linking, and the digital agents 121 operate autonomously and act independently of each other.

[0041] The malt manufacturing system 1 based on intelligent agents and digital twins includes a data monitoring and aggregation engine 14, which dynamically monitors the transmitted sensory parameter values ​​131 by updating the digital twin 12 based on sensory parameter values ​​131.

[0042] The digital twin 12 includes at least a digital agent 121 and a digital environment 122. The digital environment 122 captures characteristic parameters 1221 of the malt manufacturing equipment 11, and the digital agent 121 autonomously performs actions 1211 on at least a predefined area or unit 1212 of the digital environment 122. The actions 1211 may be triggered, for example, by at least one of fan speed, input temperature, recycled air, and / or fresh air.

[0043] The digital agent 121 includes a monitoring unit 1213 for capturing the state 1222 of at least one unit 1212 of the digital environment 122 of the malt manufacturing equipment 11. State 1222 includes characteristic parameters 1221 of the malt manufacturing equipment 11 and / or sensory parameter values ​​131 transmitted by at least one unit 1212 of the digital environment 122. For example, state 1222 can be monitored by measuring at least one of the following parameter values ​​12223: mass flow rate, recovery amount, dry weight, absolute humidity of input grains, absolute humidity of output malt, mixing temperature, output temperature, wet temperature, absolute humidity, evaporation rate, evaporation weight, moisture, fan power, heater power, freshness, and temperature exchanger. State 1222 may, for example, include at least external temperature and / or humidity. External temperature and humidity parameter values ​​may be provided, for example, through a digital twin structure.

[0044] The digital agent generation will be applied to one or more actions of the digital environment 122 of the digital twin 112 and the malt manufacturing equipment 11, respectively. When the digital environment 122 is acted upon by the digital agent 121, the state 12221 of the digital environment 122 and the malt manufacturing equipment 11 evolves to the subsequent state 12222, respectively.

[0045] Upon completion of action 1211, at least one characteristic parameter value 1221 and / or sensory parameter value 131 changes between the subsequent state 12222 and the previous state 12221. Based on the aforementioned changed characteristic parameter value 1221 and / or sensory parameter value 131, the digital environment 122 provides a reward 1223 to the digital agent 121. The digital agent 121 is sequentially enhanced by autonomously maximizing the reward 1223 by favoring actions 1211 that have higher rewards 1223. The initial value of the reward can be, for example, set to... The fan power and heater power are mutually normalized values, and This is a relative weight. Furthermore, the relative weight can be set to, for example, 0.01 to set the initial value.

[0046] The agent-based digital twin system 1 and / or digital engine 123 may include, for example, a Vasicek modeling structure 1233 for simulating external temperature and humidity parameter values.

[0047] For example, the automated operation of operation 114 of the industrial malt manufacturing equipment can be completed if the measured moisture content is below a predetermined threshold and / or if the automated operation of operation 114 exceeds a predetermined time limit. The moisture content threshold can be set, for example, to 20. Or less. The scheduled time limit can be set, for example, to 30 hours or more.

[0048] If task 114 is completed when the measured moisture content is below a predetermined threshold, then, for example, digital environment 122 may return a high reward value, or digital agent 121 or system 1 may otherwise capture a high reward value. If task 114 is completed beyond a predetermined time limit, then, for example, digital environment 122 may return a high penalty value, or digital agent 121 or system 1 may otherwise capture a high penalty value.

[0049] exist Figure 2 The development of a digital engine based on the agent-based virtual physical system 1 of the present invention is shown in more detail below. As already mentioned, the digital agent 121 autonomously performs actions 1211 on at least a predefined area or unit 1212 of the digital environment 122. The digital agent 121 provides all the necessary scheduling for the actions to be performed, while the digital engine 123 provides artificial intelligence for optimizing the strategy for the parameters used in the next optimization step. For example, the digital engine 123 applies reinforcement learning to the digital environment 122 and the digital malt manufacturing equipment twin. Figure 4 This demonstrates the application of reinforcement learning by digital engine 123. (Strategy) It is a (potentially random) function of state s, which returns the action to be performed in state s. After performing an action in state s, the reward r is received. The environment is now in state s. It should be noted that the functions for controlling the state, actions, and rewards can be stochastic.

[0050] The technical problems encountered by applying reinforcement learning can be explained by the following relationship:

[0051]

[0052]

[0053]

[0054] One of the problems stems from choosing between an on-policy (123131) or off-policy (123132) algorithm for reinforcement learning (RL). The technical difference lies in how these methods approach learning. The on-policy method learns from the digital engine's own current policy (even if it's not yet optimal), while the off-policy method learns from a different (potentially better or smarter) policy than the one currently used by the digital engine. The main difference lies in how they handle exploration and development, which technically affects their convergence properties and practical performance in different types of environments.

[0055] Clearly, the on-policy method 123131 is straightforward because the digital engine 123 learns from its own trial and error. However, the on-policy method 123131 may be less efficient because the digital engine 123 may be learning from suboptimal actions. The off-policy method 123132, such as Q-learning, can be more efficient because the digital engine 123 learns from potentially optimal actions. However, the implementation of the off-policy method 123132 may be more complex because the digital engine 123 needs to attempt to reconcile its observations with its actions. While the on-policy method 123131 of SARSA directly evaluates or improves the policy followed by the agent 121, the off-policy method 123132 of Q-learning uses potentially off-policy data (i.e., data generated from different policies) to evaluate or improve the target policy.

[0056] In this system 1, the on-policy method 123131 is used: (i) stability is achieved by interacting with multiple environments simultaneously; (ii) the on-policy algorithm 123131 typically requires more interaction time with the environment, but can effectively handle changing policies and exploration challenges; and (iii) A2C, A3C, PPO, TRPO... On the other hand, the off-policy method 123132 is used: (i) stability is achieved by using data collected from a replay buffer containing previously collected experience; (ii) the off-policy algorithm 123132 is often able to reuse experience and can learn from a diverse set of interactions, resulting in faster convergence and improved exploration capabilities; and (iii) DQN, DDPG, TD3, SAC...

[0057] Typically, the most computationally intensive phase is the interaction with the digital environment 122, and especially the transitions from state s to state s. The evolution of digital twins. For this reason, on-policy algorithms can generate as many copies of the digital environment 122 as there are available logic processors, and run all of these copies in parallel. The structure of the digital twin 12 provided by the system 1 of the present invention is efficient enough to allow parallelization in the form of tensors (where each row corresponds to a matrix of states or actions of different environments 122). Currently, on-policy algorithms are probably the logical choice. In particular, A2C is probably the most common, and many implementations are publicly available.

[0058] Now, a wrapper must be built around the agent-based digital twin 12, making it behave as an environment 122 including a digital malt-making equipment twin 120 for reinforcement learning 12313. Both environment 122 and A2C need to be modified for cross-tensor parallelization.

[0059] Both the agent and the environment 122 are derived from the digital twin 12. The environment 122 represents the physical malt-making equipment 11 and its environment 4 as a whole. A state is a set of information that allows the agent to understand what is happening in the environment 122. Actions are a set of controls through which the agent 121 can influence the environment 122. This is determined by… Figure 5 As shown. A step in the simulation may include, for example: (1) the agent reads the state of the environment (temperature = 25). (2) The agent generates an action to be applied to the environment (increase temperature), (3) the action is applied to the environment, which returns a reward (+1), and (4) the agent is now in a new state (temperature = 26). In this system 1, one step can, for example, correspond to one minute in the simulation.

[0060] The state can consist of variables such as: mass flow rate, revert amount, dry weight, absolute humidity of input grains (abs_hum_in), absolute humidity of output malt (abs_hum_out), mixing temperature (temp_mix), output temperature (temp_out), humid temperature (temp_wet), absolute humidity (abs_hum_wet), evaporation rate (evap_rate), evaporation weight (evap_weight), moisture, fan power (fan_power), heater power (heater_power), fresh amount (fresh_amount), and temperature exchanger (temp_exchanger). Actions can consist of variables such as: fan speed (fan_speed), input temperature (temp_in), reverted air (revert_air), and / or fresh air (fresh_air).

[0061] External temperature and humidity should be part of the state, but due to the digital malt manufacturing twin structure of the present invention, they can be more easily provided (i.e., implemented) as (uncontrollable) elements of action. Therefore, the action will then have two additional elements that the agent cannot control.

[0062] At this point, one of the technical challenges will be how to simulate external temperature and humidity. External air temperature and humidity can be simulated, for example, using a Vasicek model structure. Vasicek modeling is commonly used in finance to model interest rates when they can be positive or negative. Technically, this is an implementation of a mean reversion process that ensures: (i) the simulated data does not deviate too much from the initial values ​​(a job “only” lasts 30 hours), and (ii) volatility and mean...

[0063]

[0064] The regression rate is controllable, (iii) trends and oscillations are still possible, and (iv) negative temperatures are allowed.

[0065] The simulation of external air temperature and humidity achieved by this invention is as follows: Figure 6 As shown.

[0066] Another technical issue involves the implementation of rewards and terminal condition structures. Rewards are perhaps the most important parameter to be adjusted. They determine which actions should be encouraged and which should be discouraged. The digital agent learns to maximize "long-term" rewards in such a way. As an initial estimate, the reward can be set, for example, to...

[0067]

[0068] It should be noted that both fan power and heater power are normalized to have the same range. The relative weight is (temporarily) 0.01. The event (job) ends when one of the following two conditions is met: (1) the moisture content drops below a certain threshold (temporarily set to 20%), and (ii) the job lasts for more than 30 hours. Digital Environment 122 returns a large reward (+100) in the first case and a large penalty (-100) in the second case.

[0069] As by Figure 7 As shown, the digital agent 121 learns to minimize power consumption, even though this results in continuous operation for over 30 hours. This could be caused by misspecification of rewards. To overcome this problem, rewards could, for example, include a more explicit reference to moisture.

[0070] Moisture can be added to rewards in two different ways, for example:

[0071]

[0072]

[0073] Several values ​​of relative weights can be tested, for example. Time series of ambient temperature and absolute humidity can be obtained from the data, for example, and stretched or cropped so that their length matches the length of each job (30 hours).

[0074] For example, the following three algorithms can be tested with different hyperparameters: A2C or A3C (Asynchronous Advantage Actor-Critic Algorithm), TD3 (Dual Delay Deep Deterministic Policy Gradient), and / or SAC (Soft Actor-Critic). The latter algorithm, SAC, is an off-policy algorithm that optimizes stochastic policies, bridging stochastic policy optimization and DDPG-like methods. For this invention, SAC, for example, can be a preferred implementation variation due to its entropy regularization feature. Therefore, for this invention utilizing SAC, the policy can be trained, for example, to maximize the tradeoff between expected reward and entropy, a measure of stochasticity in the policy. This has a close connection to the explore-explore tradeoff: increasing entropy leads to more exploration, which can accelerate later learning. It also prevents the policy from prematurely converging to bad local optima. Typically, the following on-policy methods can be considered: (i) reinforcement, (ii) A2C (actor-commentator), (iii) PPO (proximal policy optimization), (iv) TRPO (trust region policy optimization), and the following off-policy methods can be considered: (i) soft actor-commentator: SAC (twin SAC), (ii) deep deterministic policy gradient: DDPG (deep deterministic policy gradient), (iii) TD3, and (iv) DQN (deep Q-network) such as basic dual DQN, bootstrap DQN, or QRDQN. Since QRDQN is Q-learning, this is shown to be preferred, for example, in multi-agent settings, i.e., if more than one digital agent 121 is implemented (see above). For example, for the system of the present invention, the environment can be envisioned as passive. Then, to overcome the technical problems, the use of minimax Q-learning algorithms is proposed here. However, for the present invention, A2C, TD3, and SAC may be preferred implementations.

[0075] Regarding the 22 / 212 moisture reward, the learning curve of TD3 (Dual Delay Deep Deterministic Policy Gradient) shows that, if chosen, the algorithm is learning how to minimize moisture (see...). Figure 8 However, another algorithm produces a similar learning curve. Figure 9The diagram illustrates, exemplarily, a digital agent 121 learning to apply the same actions (maximum temperature, maximum fan speed, no reverted air) invariantly. However, further penalizing energy consumption results in other constant actions. It should be noted that other algorithms besides TD3 that utilize entropy to ensure non-constant behavior (A2C, SAC) also converge to constant actions. Furthermore, changes in the relative weights of rewards can lead to constant actions; however, this is not the desired effect, as it may hinder (but not prevent) training. In this case, high variance in rewards could be a possible cause of constant actions. Therefore, TD3 is chosen because its learning curve is smoother, allowing for overcoming the aforementioned technical problems.

[0076] As another variation of the implementation, exponential smoothing can be added to the action to avoid unstable behavior. Additionally, as another variation, system 1 can be implemented by modeling the relationship between temperature and moisture. This variation has the technical advantage of allowing temperature regulation.

[0077] As by Figure 10 As shown, the net working duration can be technically inferred through fresh air control. However, as indicated by... Figures 11a to 11c As shown, the net working duration is generally independent of the following: (i) initial moisture 212 ( Figure 11a ), (ii) dry weight 214 ( Figure 11b (ii) average ambient temperature 413 ( Figure 11c To determine the net work duration, for example, it can be measured as by... Figure 12 The distribution shown is shown. In Figure 12 In the example, the net job duration distribution indicates that 36 hours is a reasonable hard threshold for drying time and should be adjusted accordingly.

[0078] For temperature-controlled scheduling, such as low-pass filters (HP filters + natural splines), it can then be used to smooth KNN predictions (e.g. Figure 13 (As shown in the diagram). Finally, a smoothing model can be constructed, for example, to fit the estimated moving standard deviation of the temperature schedule (e.g., Figure 14 As shown). Figure 15 As shown, the reward function can then be modified to account for temperature scheduling. For example, the heater power term can be discarded because it has little impact on the total reward and is implicit in the temperature scheduling term.

[0079] As another variation of the implementation, motion smoothing can be achieved. For example, exponential motion smoothing can be implemented.

[0080]

[0081] Therefore, results with strong smoothing (TAU=0.05) can be obtained. One technical problem arising in this context is that modeling may lead to unrealistic scenarios (increased water content) in the simulation, such as... Figure 16 As shown in the diagram. This must be controlled by System 1. This technical problem can be solved by: (1) investigating the causes of unstable actions, (2) investigating the causes of the “increased moisture scenario”, (3) introducing different combinations of TAU and reward coefficients, and (4) conducting tests under unseen atmospheric conditions. One technical possibility is unstable actions caused, for example, by policy collapse. Although this has never been observed using System 1 of the present invention, it can be quickly examined in the present situation. First, Proximal Policy Optimization (PPO) is a reinforcement learning algorithm designed to avoid policy collapse. However, PPO can converge to unstable actions and incur the “increased moisture” problem in some cases. However, even when PPO does not incur the “increased moisture” problem, it may still converge to unstable actions. For System 1, it is necessary to consider the case where the moisture above the kiln bed is lower than the external moisture. In fact, this may occur in the initial stage of operation when the temperature above the bed is low. This may lead to increased moisture. In this situation, the simplest solution could be to define some lower bounds in the action, such as the following, to always ensure a minimum temperature and minimum fan speed: (i) the input temperature (temperature_in) must be at least 20 degrees, and (ii) the fan speed (fan_speed) must be at least 30. .

[0082] Finally, the unstable behavior of System 1 may also be caused by problems in the calibration of the reinforcement learning algorithm and is not necessarily the optimal way to resolve the environment. To address this, Model Predictive Control (MPC) can be implemented to first check whether unstable behavior also exists in this context. MPC involves using a mathematical model of the system to predict its future behavior and then optimizing the control signal within a finite time frame to minimize a certain objective (e.g., cost, error). Here, MPC can be performed iteratively, continuously updating the control input based on new measurements and predictions, allowing it to adapt to changing conditions and optimize performance in real time. For each step in the simulation, the total cost function for the next N steps is computed as a function of the next N actions (trajectories). A Powell minimization algorithm can then be applied to find the set of N actions that minimize the cost function. The first action of the optimal trajectory can be applied to the environment. This process can then be iterated until the event ends. Since each minimization step requires performing N actions on the environment, this technique is computationally very expensive.

[0083] By applying Monte Carlo MPC, the digital twin 12 of the present invention allows for the parallelization of a large number of environments 122. Therefore, the framework of the present invention is developed to allow the implementation of Monte Carlo Model Predictive Control (MC-MPC): (1) using the technique of the present invention, a batch of trajectories of N actions are sampled at each step, (ii) each trajectory is applied in parallel to different environments, (3) the trajectory is evaluated according to the total cost function, (4) the first action of the optimal trajectory is finally applied to environment 122, (5) due to parallelization, the technical advantage of the system of the present invention is that MC-MPC is not as computationally expensive as analytical MPC, and (6) the batch size of all experiments can be set, for example, equally, for example, to 10K (therefore, at each step, 10K possible future trajectories are sampled).

[0084] Additionally, evolutionary strategies can be implemented, for example, to find the optimal trajectory (e.g.) Figure 19 (as shown in the figure). For example, (1) the first set (10K) of random trajectories can be sampled, (2) the trajectories can be evaluated, (3) the best trajectory can be stored and copied multiple times (10K), (4) the copy of the best trajectory can be perturbed with random noise, and (5) points 2 to 4 can be iterated until convergence.

[0085] It should be noted that in Exemplary System 1, MC-MPC with 100 look-ahead steps was found to be the best performing method. In this Exemplary System 1, the performance of MC-MPC showed comparable to that implemented by a reinforcement learning agent. However, MPC solutions are difficult to generalize to new environments because they rely heavily on simulation.

[0086] The optimal strategy to be trained can aim to reduce process time while adhering to a predetermined temperature at the expense of energy efficiency. In this context, new strategies can be trained to grant privileges to other features. Strategies could be: (i) a balanced mode, (ii) a quality mode (encouraging adherence to temperature scheduling), (iii) an economy mode (enhancing energy efficiency while adhering to temperature scheduling), and (iv) a lightning mode (ending the job as quickly as possible).

[0087] Figure 1bThe schematic illustration shows the architecture of a possible implementation of another variant of the system of the present invention, which can be applied to an agent- and digital twin-based fermentation system 1 for the automated operation of an industrial fermentation apparatus 11 of the agent- and digital twin-based fermentation system 1. The fermentation apparatus 11 ferments a culture medium 2 to produce product material 3, wherein the fermentation apparatus 11 includes a fermenter 111 for processing and culturing the culture medium 2, the culture medium 2 comprising microorganisms and / or enzymes and / or substrates and / or at least one nutrient, to initiate and culture the fermentation process within the fermenter 111, to convert at least one nutrient into product material 3 by microorganisms and / or enzymes under defined fermentation parameters 1112 within the fermenter 111. Fermentation parameters 1112 include: the temperature of the culture medium 2—used to manipulate the microbial activity of the microorganisms; and / or the process pH of the culture medium 2; and / or the dissolved oxygen in the fermenter 111; and / or the revolutions per minute (rpm) of the stirring device in the fermenter 111 for mixing the culture medium 2; and / or the substrate concentration and / or nutrient supply, used to manipulate the growth of the microorganisms cultured in the fermenter 111.

[0088] Fermentation equipment is designed for industrial use, performing large-scale fermentation processes. Fermentation equipment 11 can be used for: food and beverages, such as beer, wine, cheese, yogurt, and other fermented foods; biofuels, such as ethanol produced from fermented sugars or starches; chemicals, such as organic acids, solvents, and other commercial chemicals; pharmaceuticals, such as antibiotics, vitamins, and other bioactive compounds; and alternative proteins, such as plant-based and microbial protein products. Typical fermentation equipment includes:

[0089] Fermentation tank 111 can be a large container in which the fermentation process takes place. Fermentation equipment 11 includes a sterilization system for maintaining aseptic conditions and / or a temperature control system including an operable heater for maintaining optimal fermentation conditions and / or a stirring device for mixing culture medium 2.

[0090] The culture medium 2 in fermenter 111 typically comprises one or more components to support microbial growth and the formation of product material 3. Components of the culture medium 2 used in the fermentation process include, for example, carbohydrates such as glucose, sucrose, or starch; oils and fats that provide energy and carbon for microbial cell growth and product material 3 formation; ammonia such as ammonium salts, urea, corn extract, soybean meal; amino acids essential for protein synthesis and microbial cell growth; nitrogen that can affect enzyme production and metabolite formation; minerals such as calcium, chlorine, magnesium, phosphorus, potassium, sulfur, and trace elements (copper, cobalt, iron, manganese, molybdenum, zinc) critical for enzyme function, cell structure, and metabolic processes; and substrate components such as water, which provide the (aqueous) environment for all biochemical reactions and nutrient transport. Air or oxygen supplied, for example, through an aeration system, is critical for aerobic fermentation, influencing growth rate, metabolic pathways, and product material 3 formation. The specific composition of the culture medium 2 depends on the microorganisms cultured in fermenter 111, the desired output material 3, and the scale of fermentation equipment 11.

[0091] The system 1 based on intelligent agents and digital twins also includes a digital twin 12 that provides a virtual digital representation of the physical fermentation device 11 and sensors and / or measuring devices 13 associated with the physical fermentation device 11, the sensors and / or measuring devices 13 transmitting measured sensory parameter values ​​131.

[0092] The digital twin 12 includes at least one digital agent 121 and a digital environment 122. The digital environment 122 captures characteristic parameters 1221 of the fermentation equipment 11, and the digital agent 121 autonomously performs actions 1211 on at least one predefined region or unit 1212 of the digital environment 122. The action 1211 may be triggered, for example, by at least one of the following: the temperature of the culture medium 2, the pH of the culture medium 2, the dissolved oxygen in the fermenter 111, the rpm (revolutions per minute) of the agitator device, the nutrient concentration in the culture medium 2, the inoculum density in the culture medium, and the growth rate of the microorganisms in the culture medium 2.

[0093] State 1222 can be monitored, for example, by measuring at least one of the following parameter values ​​12223: temperature of culture medium 2, pH of culture medium 2, dissolved oxygen in fermenter 111, rpm of the stirring device in fermenter for mixing culture medium 2, concentration of substrate in culture medium 2, inoculum density in culture medium 2, and nutrient concentration in culture medium 2. State 1222 may, for example, include at least one external temperature and / or humidity. The external temperature and humidity parameter values ​​may, for example, be provided via a digital twin structure.

[0094] The digital agent generation will be applied to one or more actions of the digital environment 122 and the fermentation device 11 of the digital twin 112, respectively. When the digital environment 122 is acted upon by the digital agent 121, the digital environment 122 evolves from the state 12221 of the digital environment 122 and the fermentation device 11 to the subsequent state 12222, respectively.

[0095] Based on the changed characteristic parameter values ​​1221 and / or sensory parameter values ​​131, the digital environment 122 feeds back rewards 1223 to the digital agent 121. The digital agent 121 autonomously maximizes the rewards 1223 by favoring actions 1211 that yield higher rewards 1223, thus being sequentially enhanced. The initial value of the reward can be, for example, set to -power stirrer + A power heater, wherein the fan power and heater power are mutually normalized values, and This is a relative weight. Furthermore, the relative weight can be set to, for example, 0.01 to set the initial value.

[0096] For example, the automated operation of the industrial fermentation equipment 11 can be completed if the growth rate and / or cell viability of the culture medium 2 are measured to be below a defined threshold and / or if the automated operation of the operation 114 of the industrial fermentation equipment exceeds a predetermined time limit.

[0097] The industrial beer production process utilizes malt 3 and 31 to produce wort, and then produces beer. This production process includes the following steps:

[0098] a) Saccharification, used to extract fermentable sugars from malt 3, 31 using enzymes that occur during malt production.

[0099] b) Boiling is used to sterilize the wort and add hops.

[0100] c) Fermentation by yeast is used to convert sugar into alcohol and CO2.

[0101] The malting and fermentation processes used in beer production are closely intertwined. The malt 3, 31 produced by the malting equipment 11 significantly influences the fermentation process in the fermentation tank 111 and the product material 3, such as the final beer quality. The malting equipment 11 converts the supplied grains 2, 21, 21i into malt 3, 31, wherein the output malt 3, 31 includes those activated during malting. -Amylase and -Amylase, the -Amylase and -Amylase is crucial for the saccharification step and is ultimately necessary for, for example, industrial beer production. The germination equipment 112 is activating enzymes such as 2, 21, and 21i in the grain kernels. - Glucanase and amylase. These enzymes break down starch into fermentable sugars during the fermentation steps in fermenter 111. In addition, malt produces proteins that supply the grains 2, 21, and 21i, which affects the foam stability of beer 3 during fermentation, the yeast nutrition of malt during fermentation, and the transparency of the output material 3 during fermentation.

[0102] The techniques taught herein can be applied to other systems, not necessarily those described above. Elements and actions from the various examples described above can be combined to provide alternative implementations of the technique. Some alternative implementations of the technique may include not only elements specific to those described above.

[0103] List of reference numerals

[0104] 1. Agent-based digital twin system

[0105] 11 Industrial Malt Manufacturing Equipment / Industrial Fermentation Equipment

[0106] 111 Soaking Equipment / Fermentation Tank

[0107] 1111 Moisture content

[0108] 1112 Defined moisture content level after wetting / Defined fermentation parameters 112 Germination equipment

[0109] 113 Drying Kiln

[0110] 1131 specifies the moisture content level after drying.

[0111] 114 Operations automated by industrial malt manufacturing equipment

[0112] 115 physical units / region

[0113] 1151 Physics Unit 1

[0114] 1152 Physics Unit 2

[0115] ...

[0116] 115i physical unit i

[0117] 12 Agent-Based Digital Twins

[0118] 120 Digital Malt Manufacturing Equipment Twin / Digital Fermentation Equipment Twin

[0119] 1201 Simulation

[0120] 1202 Synchronization

[0121] 1203 Senses

[0122] 121 Digital Intelligent Agent

[0123] 1211 action

[0124] 1212 Environment's operable predefined regions / cells

[0125] 1213 Monitoring Unit

[0126] 1214 Dispatch

[0127] 122 Digital Environment

[0128] 1221 Environmental / Malt Manufacturing Equipment / Fermentation Equipment Characteristic Parameters

[0129] Characteristic parameters of 12211 malt manufacturing equipment

[0130] 12212 Environmental characteristics

[0131] 1222 status

[0132] 12221 Previous state

[0133] The subsequent status of 12222

[0134] 12223 State parameter value

[0135] 1223 Reward

[0136] 1224 Digital Device Twin Layer

[0137] 12241 Simulation

[0138] 12242 Synchronization

[0139] 12243 Senses

[0140] 1225 Digital Device Replica Layer

[0141] Digital components for 12251, ..., 1225i malt manufacturing equipment

[0142] 123 Digital Engine

[0143] 1231 Artificial Intelligence

[0144] 12311 Machine Learning

[0145] 12312 Neural Network

[0146] 12313 Reinforcement Learning

[0147] 123131 On-policy approach

[0148] 123132 Off-policy approach

[0149] 1232 Dynamic Scheduling

[0150] 1233 Vasicek Modeling Structure

[0151] 13 Sensors and / or measuring devices

[0152] 131 sensory parameter values

[0153] 14 Data Monitoring and Aggregation Engine

[0154] 2 Germination-promoting materials / culture media

[0155] 21 Supply of grain

[0156] 211 Grain Type

[0157] 2111 Grains

[0158] 21111 One or more cereal genotypes

[0159] 2112 barley grains

[0160] 21121 One or more barley genotypes

[0161] 2113 Corn

[0162] 21131 One or more maize genotypes

[0163] 212 Moisture content

[0164] 213 Impurities and Foreign Matter

[0165] 214 test weight

[0166] 215 1000 kernel weight (TKW)

[0167] 216 Grain volume and size distribution

[0168] 217 Hardness (Particle Size Index (PSI), Near Infrared (NIR) Hardness, Single Kernel Characteristic System (SKCS), Peel Index)

[0169] 218 Other characteristics

[0170] 2161 protein

[0171] 2162 calcium

[0172] 2163 Phosphorus

[0173] 2164 magnesium

[0174] 2165 potassium

[0175] 2166 sulfur

[0176] 2167 acid detergent fiber ( )

[0177] 2168 Neutral Detergent Fiber ( )

[0178] 2169 fat

[0179] 218 grain temperature

[0180] 22 Moistened grains

[0181] 23 sprouted green wheat sprouts

[0182] 3 Output Materials / Product Materials

[0183] 31 Malt

[0184] 311 Malt Extract

[0185] 312 glycation power

[0186] 313 Malt wort viscosity

[0187] 4. Physical environment (excluding malt manufacturing equipment)

[0188] 41 Environmental Measurement Parameters

[0189] 411 air humidity

[0190] 412 atmospheres

[0191] 413 Ambient temperature of malt manufacturing equipment

[0192] ...

[0193] 41i The i-th environmental condition

Claims

1. A malt manufacturing system (1) based on intelligent agents and digital twins, for the automated operation of an industrial malt manufacturing device (11) of the malt manufacturing system (1), the malt manufacturing device (11) converting supplied grains (2, 21, 21i) into output malt (3), wherein, The malt manufacturing equipment (11) includes: a soaking device (111) for wetting the supplied grains (2, 21, 21i) until the moisture content (1111) of the supplied grains (2, 21, 21i) reaches a predetermined moisture content level (1112); a germination device (112) configured to receive the moistened grains (22) after being moistened by the soaking device (111), for germinating the moistened grains (22) by activating enzymes in the moistened grains (22) and converting the moistened grains (22) into germinated green malt (23); and a drying kiln (113) configured to receive the germinated grains (23) after being germinated by the germination device (112), for drying the germinated green malt. Malt (23) is dried to a defined moisture content level (1131), wherein the agent- and digital twin-based system (1) includes a digital twin structure (12) providing a virtual digital representation of the physical malt manufacturing equipment (11) and sensors and / or measuring devices (13) associated with the physical malt manufacturing equipment (11), the sensors and / or measuring devices (13) transmitting measured sensory parameter values ​​(131), and wherein the agent- and digital twin-based malt manufacturing system (1) includes a data monitoring and aggregation engine (14) that dynamically monitors the transmitted sensory parameter values ​​(131) by updating the digital twin (12) based on the sensory parameter values ​​(131), characterized in that, The digital twin (12) includes at least one digital agent (121) and a digital environment (122), the digital environment (122) capturing characteristic parameters (1221) of the malt manufacturing equipment (11), and the digital agent (121) autonomously performing actions (1211) on at least a predefined area or unit (1212) of the digital environment (122). The digital agent (121) includes a monitoring unit (1213) for capturing the state (1222) of at least one unit (1212) of the digital environment (122) of the malt manufacturing equipment (11). The state (1222) includes characteristic parameters (1221) of the malt manufacturing equipment (11) and / or sensory parameter values ​​(131) transmitted by at least one unit (1212) of the digital environment (122). The agent generation will be applied to one or more actions of the digital environment (122) and the malt manufacturing equipment (11) of the digital twin (112), respectively, wherein the digital environment (122) evolves from a state (12221) to a subsequent state (12222) when the digital agent (121) acts on the digital environment (122) and the malt manufacturing equipment (11), respectively. When the action (1211) is completed, at least one characteristic parameter value (1221) and / or sensory parameter value (131) changes between the subsequent state (12222) and the previous state (12221), wherein, based on the changed characteristic parameter value (1221) and / or sensory parameter value (131), the digital environment (122) feeds back a reward (1223) to the digital agent (121), and the digital agent (121) autonomously maximizes the reward (1223) by favoring actions (1211) with higher rewards (1223), thereby being sequentially enhanced.

2. The malt manufacturing system (1) based on intelligent agents and digital twins according to claim 1, characterized in that, The state (1222) is monitored by measuring at least one of the following parameter values ​​(12223): mass flow rate, recovery amount, dry weight, absolute humidity of input grains, absolute humidity of output malt, mixing temperature, output temperature, wet temperature, absolute humidity, evaporation rate, evaporation weight, moisture, fan power, heater power, freshness, and temperature exchanger.

3. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 1 or 2, characterized in that, The action (1211) is triggered by at least one of fan speed, input temperature, recycled air and / or fresh air.

4. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 1 to 3, characterized in that, The agent-based digital twin system (1) includes at least two digital agents (121), which interact through communication and act on the digital environment (122) and the malt manufacturing equipment (11) respectively. Each of the digital agents (121) has different units (1212) defined by different interaction characteristics on the digital environment (122) and the malt manufacturing equipment (11), and / or the at least two digital agents (121) are linked or connected to each other through operation sequence links. The digital agents (121) operate autonomously and act independently of each other.

5. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 1 to 4, characterized in that, The state (1222) includes at least external temperature and / or humidity.

6. The malt manufacturing system (1) based on intelligent agents and digital twins according to claim 5, characterized in that, The external temperature and humidity parameters are provided by the digital twin structure.

7. The malt manufacturing system (1) based on intelligent agents and digital twins according to claim 6, characterized in that, The agent-based digital twin system (1) and / or digital engine (123) includes a Vasicek modeling structure (1233) for simulating the external temperature and humidity parameter values.

8. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 1 to 7, characterized in that, The initial value of the reward was set to Wherein, the fan power and heater power are mutually normalized values, and It is a relative weight.

9. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 8, characterized in that, The relative weight is set to 0.01 to set the initial value.

10. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 1 to 10, characterized in that, If the moisture content is measured to be below a predetermined threshold and / or if the automated operation of the operation (114) of the industrial malt manufacturing equipment exceeds a predetermined time limit, then the automated operation of the operation (114) of the industrial malt manufacturing equipment (11) is completed.

11. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 10, characterized in that, The defined threshold for the moisture content is set to 20. Or smaller.

12. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 10 or 11, characterized in that, The predetermined time limit is set to 30 hours or more.

13. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 10 to 12, characterized in that, If the task (114) is completed when the moisture content is measured to be below a defined threshold, the digital environment (122) will return a high reward value, or the system (1) will otherwise capture the high reward value.

14. The malt manufacturing system (1) based on intelligent agents and digital twins according to any one of claims 10 to 13, characterized in that, If the task (114) is completed before the predetermined time limit is exceeded, the digital environment (122) returns a high penalty value, or the system (1) otherwise captures the high penalty value.

15. A malt manufacturing system (1) based on intelligent agents and digital twins for the automated operation of an industrial fermentation apparatus (11) of a fermentation system (1) based on intelligent agents and digital twins, said fermentation apparatus (11) fermenting a culture medium (2) for producing product material (3), wherein, The fermentation equipment (11) includes a fermenter (111) for processing and culturing the culture medium (2), the culture medium (2) comprising microorganisms and / or enzymes and / or substrates and / or at least one nutrient to initiate and cultivate a fermentation process within the fermenter (111) to convert the at least one nutrient into the product material (3) by means of the microorganisms and / or enzymes under defined fermentation parameters within the fermenter (111). The fermentation parameters (1112) include: the temperature of the culture medium (2) – used to manipulate the microbial activity of the microorganisms and / or the temperature of the enzymes; and / or the process pH of the culture medium (2); and / or the dissolved oxygen in the fermenter; and / or the rpm of the stirring device in the fermenter for mixing the culture medium (2); and / or the substrate concentration and / or nutrient supply, used to manipulate the growth of the microorganisms cultured in the fermenter (111). The system (1) based on intelligent agents and digital twins includes a digital twin structure (12) providing a virtual digital representation of the physical fermentation device (11) and sensors and / or measuring devices (13) associated with the physical fermentation device (11), the sensors and / or measuring devices (13) transmitting measured sensory parameter values ​​(131), and wherein the fermentation system (1) based on intelligent agents and digital twins includes a data monitoring and aggregation engine (14), the data monitoring and aggregation engine (14) dynamically monitoring the transmitted sensory parameter values ​​(131) by updating the digital twin (12) based on the sensory parameter values ​​(131), characterized in that, The digital twin (12) includes at least one digital agent (121) and a digital environment (122), the digital environment (122) capturing characteristic parameters (1221) of the fermentation device (11), and the digital agent (121) autonomously performing actions (1211) on at least a predefined region or unit (1212) of the digital environment (122). The digital agent (121) includes a monitoring unit (1213) for capturing the state (1222) of at least one unit (1212) of the digital environment (122) of the fermentation device (11), the state (1222) including characteristic parameters (1221) of the fermentation device (11) and / or sensory parameter values ​​(131) transmitted by at least one unit (1212) of the digital environment (122), the digital agent generating one or more actions to be applied to the digital environment (122) and the fermentation device (11) of the digital twin (112), wherein the digital environment (122) evolves from the state (12221) of the digital environment (122) and the fermentation device (11) to a subsequent state (12222) when the digital agent (121) acts on the digital environment (122) and the fermentation device (11) respectively, and When the action (1211) is completed, at least one characteristic parameter value (1221) and / or sensory parameter value (131) changes between the subsequent state (12222) and the previous state (12221). Based on the changed characteristic parameter value (1221) and / or sensory parameter value (131), a reward (1223) is fed back to the digital agent (121) through the digital environment (122). The digital agent (121) is sequentially enhanced by autonomously maximizing the reward (1223) by favoring actions (1211) with higher rewards (1223).

Images