Yogurt viscosity control method and device based on reinforcement learning

Abstract

The present invention provides a yogurt viscosity control method and device based on reinforcement learning. The method is applied to a control device including a data acquisition module and an edge processing unit. Specifically, the method comprises: establishing a digital twin model of the demulsification process; collecting environmental parameters during the demulsification process in real time, and constructing a time series data set in combination with the simulation parameters output by the digital twin model; correcting the digital twin model according to the time series data set based on a reinforcement learning algorithm; generating a stirring control strategy based on the corrected digital twin model in combination with a preset viscosity value; and adjusting the stirring parameters of the demulsification process according to the stirring control strategy. Based on the efficient control of yogurt viscosity achieved through the digital twin model, the present invention further introduces a reinforcement learning algorithm to make the digital twin model more realistic and ensure the accuracy of its output control strategy, thereby taking into account both the control efficiency and accuracy of yogurt viscosity control.

Description

Technical Field

[0001] The present invention relates to the technical field of yogurt processing technology, and in particular to a yogurt viscosity control method and device based on reinforcement learning. Background Art

[0002] Stirred yogurt is a semi-fluid dairy product with a certain viscosity, made by stirring the fermented curd before canning and then selectively adding ingredients such as fruit and jam. The taste and viscosity of yogurt are important indicators for consumers to evaluate yogurt. However, since yogurt is made from milk, its composition is affected by factors such as climate, feed, and variety, and there are certain differences between individuals. This means that even if the same process is used in yogurt production, it is still difficult to achieve consistent viscosity. Currently, the detection and control of yogurt viscosity mostly relies on the results of sampling and laboratory testing. Although the test results are relatively accurate, it takes a long time and the detection efficiency is low, resulting in low control efficiency of yogurt viscosity.

[0003] To overcome the inefficient viscosity control caused by low laboratory sampling and testing efficiency, related technologies are attempting to use digital twin technology to simulate the production process of stirred yogurt, achieving efficient viscosity control. Digital twin technology can simulate different production scenarios and process parameter combinations in a virtual environment, analyze their impact on yogurt viscosity, and identify the optimal production process and parameter settings for efficient viscosity control.

[0004] Although digital twin technology can simulate the yogurt production process by establishing a corresponding virtual model, there are many unpredictable factors in the actual production environment, such as slight differences in raw materials, equipment wear, changes in environmental conditions, etc. These factors may cause the constructed virtual model to deviate from the actual situation, thereby leading to low accuracy in controlling yogurt viscosity. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art in which it is difficult to take into account both control accuracy when using digital twin technology to achieve efficient control of yogurt viscosity. A yogurt viscosity control method and device based on reinforcement learning are provided, and a digital twin model is used to simulate the yogurt demulsification process to achieve efficient control of yogurt viscosity. On this basis, a reinforcement learning algorithm is further introduced to optimize and adjust the virtual environment of the digital twin model in real time according to the actual collected environmental parameters to correct the digital twin model, improve the accuracy of the digital twin model under virtual environment training, and ensure the accuracy of the stirring control strategy output by the digital twin model, so as to achieve the effect of taking into account both control efficiency and control accuracy of yogurt viscosity control.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] Yogurt viscosity control method based on reinforcement learning, including:

[0008] Establish a digital twin model of the demulsification process;

[0009] Real-time collection of environmental parameters during the demulsification process and construction of a time series data set based on simulation parameters output by the digital twin model;

[0010] Based on reinforcement learning algorithms, the digital twin model is modified according to the time series data set;

[0011] Based on the modified digital twin model, the stirring control strategy is generated in combination with the preset viscosity value;

[0012] Adjust the stirring parameters of the demulsification process according to the stirring control strategy.

[0013] Furthermore, the method of correcting the digital twin model based on the time series data set based on the reinforcement learning algorithm includes:

[0014] Time-align the acquired environmental parameters and simulation parameters, and obtain the corresponding difference indicators;

[0015] Construct a training dataset based on the time-aligned environmental parameters, simulation parameters, and corresponding difference indicators;

[0016] Establish an environment model neural network and train the environment model neural network based on the training data set;

[0017] The environmental model neural network obtains a correction strategy based on the time series data set and adjusts the model parameters of the digital twin model according to the correction strategy.

[0018] Furthermore, the environmental model neural network obtains a correction strategy based on the time series data set, including:

[0019] The time series data set is input into the environment model neural network, and the environment model neural network outputs the correction parameters;

[0020] A correction strategy for generating digital twin models based on correction parameters.

[0021] Furthermore, the correction strategy for generating the digital twin model based on the correction parameters includes:

[0022] Identify the correction parameter type and associate it with the model parameters of the digital twin model;

[0023] The correction target parameters are determined based on the association matching results, and a correction strategy for the correction target parameters is generated according to the correction parameter values output by the environmental model neural network.

[0024] Furthermore, the stirring control strategy is generated based on the modified digital twin model and the preset viscosity value, including:

[0025] Output viscosity change curve based on the revised digital twin model;

[0026] Obtain viscosity adjustment requirements based on preset viscosity values and viscosity change curves;

[0027] Generates agitation control strategies based on viscosity adjustment requirements.

[0028] Furthermore, the establishment of a digital twin model of the demulsification process includes:

[0029] Obtain equipment parameters and historical production data of the mixing equipment in the demulsification process;

[0030] Create and initialize a multi-dimensional sub-model of the demulsification process based on equipment parameters and historical production data;

[0031] The sub-models are coupled, and the model parameters of each sub-model are calibrated and optimized through historical production data to obtain a digital twin model of the demulsification process.

[0032] Furthermore, the environmental parameters include at least viscosity data, shear rate data and temperature data of the yogurt.

[0033] A yogurt viscosity control device based on reinforcement learning, used to execute the yogurt viscosity control method based on reinforcement learning, comprising:

[0034] A data acquisition module is provided at the stirring device to collect real-time environmental parameters of the stirred yogurt during the demulsification process;

[0035] The edge processing unit is connected to the data acquisition device for generating a stirring control strategy based on the collected environmental parameters through a digital twin model and reinforcement learning algorithm, and adjusting the stirring parameters of the stirring equipment.

[0036] Furthermore, the data acquisition module at least includes:

[0037] The torque sensor is installed between the stirring paddle and the driving motor of the stirring device to collect the viscous resistance torque of the stirring paddle;

[0038] An angle sensor is provided between the stirring paddle and the driving motor of the stirring device to detect the rotation angle of the stirring paddle;

[0039] The temperature sensor is installed in the fermentation tank of the stirring equipment and is used to collect temperature data during the demulsification process.

[0040] Furthermore, the data acquisition module further includes:

[0041] The data processing unit is connected to the torque sensor and the angle sensor respectively, and is used to calculate the viscosity data and shear rate data of the yogurt according to the collected viscous resistance torque and rotation angle of the stirring paddle.

[0042] The beneficial effects of the present invention are:

[0043] (1) By establishing a digital twin model of the demulsification process to simulate the yogurt demulsification process, and by digitally modeling key factors such as temperature changes and viscosity changes involved in the demulsification process, the complex physical and chemical reactions are converted into quantifiable and simulatable digital signals to achieve efficient control of yogurt viscosity. On this basis, a reinforcement learning algorithm is introduced to optimize and adjust the virtual environment of the digital twin model in real time according to the actual collected environmental parameters to correct the digital twin model and improve the accuracy of the digital twin model trained in the virtual environment. This ensures that the constructed digital twin model can be closer to the actual yogurt demulsification process, improves the accuracy of the output stirring control strategy, and takes into account both the control efficiency and control accuracy of yogurt viscosity control.

[0044] (2) Use the environmental model neural network to adjust the virtual environment and then complete the corresponding correction work of the digital twin model. When faced with complex situations where multiple factors such as temperature and stirring speed are coupled to affect viscosity, the neural network can quickly establish a precise mathematical mapping relationship, providing more accurate data support for the digital twin model, making the digital twin model closer to the actual production process and ensuring the correction efficiency and accuracy of the digital twin model. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of a process of the present invention;

[0046] Figure 2 This is a schematic diagram of the configuration of a data acquisition module on a fermentation tank according to an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the overall system topology of a yogurt viscosity control device according to an embodiment of the present invention;

[0048] Figure 4 1 is a schematic diagram of a virtual training environment for controlling yogurt viscosity according to an embodiment of the present invention;

[0049] Figure 5 It is a schematic diagram of an optimization and adjustment framework of an environmental model neural network and a digital twin model in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further described below with reference to the accompanying drawings and examples.

[0051] Example: In the stirred yogurt process, the demulsification process is a critical step in adjusting the yogurt's viscosity after curdling, which is the origin of the name stirred yogurt. Although yogurt is a shear-thinning fluid, and its viscosity decreases with increasing shear rate, it is a shear-thinning fluid at the initial low shear rate stage of demulsification. It is not until the structural breakdown (SB) point that the yogurt's viscosity rapidly changes, quickly transforming into a shear-thinning fluid. The viscosity can then be precisely adjusted by adjusting process parameters such as shear rate.

[0052] The characteristics of the structural failure point of stirred yogurt are determined by the number and bond strength of the colloidal network generated during the solidification process. The colloidal network is determined by the previous processing technology and the milk source itself. If the previous processing technology remains unchanged, the characteristics of the colloidal network are determined by the raw materials. However, because yogurt is made from milk, its composition is affected by factors such as climate, feed, and variety, and there are certain differences between individual products. This means that even if yogurt production uses the same process flow in the previous processing, it is still difficult to achieve consistent viscosity when entering the demulsification process.

[0053] To ensure the yogurt's taste, certain numerical requirements are often imposed on the yogurt's viscosity after the demulsification process. This requires that the viscosity control during the demulsification process be adaptively adjusted based on the yogurt's actual viscosity. The existing method of adjusting the demulsification process through manual sampling and laboratory testing is inefficient. Therefore, to achieve efficient control of yogurt viscosity and reduce the inefficiency caused by manual viscosity control, this embodiment further proposes simulating the yogurt demulsification process through digital twin technology to achieve efficient control of yogurt viscosity.

[0054] Considering that although digital twin technology can simulate the yogurt production process, there are many unpredictable factors in the actual production process. The virtual model constructed by digital twin technology may deviate from the actual situation to a certain extent, and cannot take into account the control efficiency and accuracy of yogurt control. This embodiment further proposes a yogurt viscosity control method based on reinforcement learning, such as Figure 1 Shown, including:

[0055] Establish a digital twin model of the demulsification process;

[0056] Real-time collection of environmental parameters during the demulsification process and construction of a time series data set based on simulation parameters output by the digital twin model;

[0057] Based on reinforcement learning algorithms, the digital twin model is modified according to the time series data set;

[0058] Based on the modified digital twin model, the stirring control strategy is generated in combination with the preset viscosity value;

[0059] Adjust the stirring parameters of the demulsification process according to the stirring control strategy.

[0060] In the yogurt production process, demulsification is a key stage that affects yogurt viscosity. By adjusting the process parameters during demulsification, the viscosity of the yogurt can be effectively controlled. By establishing a digital twin model of the demulsification process, the actual demulsification scene can be highly restored in a virtual environment, intuitively presenting physical phenomena such as fluid movement and droplet breakup during the demulsification process. The established digital twin model can be used to predict in advance the impact of different equipment parameters and process conditions on the demulsification effect and yogurt viscosity. When viscosity adjustment is required, the impact of different parameter combinations on demulsification and yogurt viscosity can be tested in virtual space without actual testing, thereby optimizing production parameter settings and achieving precise control of yogurt viscosity.

[0061] Specifically, the establishment of a digital twin model of the demulsification process includes:

[0062] Obtain equipment parameters and historical production data of the mixing equipment in the demulsification process;

[0063] Create and initialize a multi-dimensional sub-model of the demulsification process based on equipment parameters and historical production data;

[0064] The sub-models are coupled, and the model parameters of each sub-model are calibrated and optimized through historical production data to obtain a digital twin model of the demulsification process.

[0065] When establishing a digital twin model, creating multidimensional sub-models is the basis for achieving accurate simulation. Based on the parameters of the mixing equipment and historical production data, multiple targeted sub-models are established based on different physical processes and influencing factors, and initial parameters are set for them. Then, the overall digital twin model is constructed using a multi-physical field coupling method to comprehensively simulate the physical and chemical changes in the yogurt demulsification process.

[0066] The above-mentioned multidimensional sub-model specifically includes a fluid mechanics sub-model, a heat transfer sub-model and a chemical reaction sub-model.

[0067] Among them, the fluid mechanics sub-model is based on the theory of computational fluid dynamics and is established according to the structural parameters of the stirring equipment and the stirring speed data during the production process. The structural parameters include at least the diameter of the stirring paddle, the shape of the blades, the volume of the stirring tank and other parameters. It can accurately simulate the flow state, vortex distribution, mixing uniformity and other fluid movements of yogurt in the stirring tank, thereby accurately describing the influence of these fluid movements on the dispersion of emulsion droplets and the demulsification effect during the demulsification process, ensuring that the digital twin model can accurately reflect the dynamic changes of yogurt in the stirring tank, and provide a flow basis for the subsequent analysis of the influence of other physical processes on demulsification.

[0068] The heat transfer sub-model is based on the principles of heat transfer and is further established in combination with data such as the material, wall thickness, and heating or cooling device parameters of the mixing tank. The model is adjusted and optimized in combination with temperature change data in historical production data to accurately describe the heat transfer process between the yogurt and the tank body and the environment, ensuring that the digital twin model can accurately calculate the impact of temperature changes on the yogurt demulsification reaction rate, viscosity changes, fluid properties, etc.

[0069] The chemical reaction sub-model simulates the chemical changes occurring during the yogurt demulsification process, such as the aggregation of fat globules and the denaturation of proteins, by establishing chemical reaction kinetic equations. It then accurately describes the relationship between these chemical reactions and factors such as temperature and stirring intensity, providing a data basis for subsequent viscosity change analysis.

[0070] The sub-models mentioned in this embodiment can be established using simulation software tailored to specific needs. For example, COMSOL Multiphysics (multi-physics field) simulation software from COMSOL (Sweden) can be used to construct sub-models for each physical field. Model parameters, such as initial temperature, initial velocity field, and initial chemical reaction state, are then set using the equipment parameters of the mixing equipment and historical production data to initialize the relevant model parameters, enabling the model to simulate the corresponding specific physical process. However, to fully reflect the true dynamics of the demulsification process, the sub-models must be integrated and optimized.

[0071] Specifically, sub-model integration and optimization are achieved through sub-model coupling. Sub-model coupling simulates the multi-physics coupling of the demulsification process by establishing a parameter transfer relationship between the sub-models. For example, the fluid flow rate and pressure distribution calculated in the fluid mechanics sub-model will affect the convective heat transfer coefficient in the heat transfer sub-model. The temperature change obtained by the heat transfer sub-model will change the surface tension and viscosity of the emulsion droplets reflected in the chemical reaction sub-model, thereby affecting the breakup behavior of the emulsion droplets. Therefore, by setting corresponding interfaces and data transfer rules in the simulation software, information exchange and collaborative calculations between the sub-models can be achieved, thus achieving coupling of the sub-models.

[0072] The historical production data is then input into the coupled model, and the model output results, such as the simulated yogurt viscosity and temperature change curve, are compared with the detection data in the actual production process. Then, optimization algorithms such as genetic algorithms and particle swarm optimization algorithms are used to iteratively optimize the parameters of each sub-model, so that the final digital twin model can fit the actual production situation of yogurt.

[0073] However, the yogurt production process is affected by many unpredictable factors. Even if a large amount of historical production data is used to build a digital twin model, there will still be certain differences between the constructed digital twin model and the actual situation. Therefore, this embodiment utilizes the MBPO (Model-Based Policy Optimization) framework. By collecting corresponding environmental parameters in real time during the yogurt demulsification process, the virtual environment of the digital twin model is adjusted in real time according to the reinforcement learning algorithm, realizing the optimization adjustment of the yogurt viscosity control strategy output by the digital twin model.

[0074] Wherein, the environmental parameters include at least viscosity data, shear rate data and temperature data of the yogurt.

[0075] On this basis, the digital twin model is modified according to the time series data set based on the reinforcement learning algorithm, including:

[0076] Time-align the acquired environmental parameters and simulation parameters, and obtain the corresponding difference indicators;

[0077] Construct a training dataset based on the time-aligned environmental parameters, simulation parameters, and corresponding difference indicators;

[0078] Establish an environment model neural network and train the environment model neural network based on the training data set;

[0079] The environmental model neural network obtains a correction strategy based on the time series data set and adjusts the model parameters of the digital twin model according to the correction strategy.

[0080] Taking into account the similarity of the yogurt production process, the collected production data is used as the training basis for the environmental model neural network, such as the production data collected by the same mixing equipment in multiple time periods and the corresponding simulation data of the digital twin model.

[0081] During the yogurt demulsification process, environmental parameters are collected in real time by sensors, while the simulation parameters output by the digital twin model are generated based on model calculations. Considering factors such as the frequency of data collection and model calculations, transmission delays, and other factors, there may be a temporal misalignment between the two parameters. Therefore, to ensure the accuracy of subsequent digital twin model corrections, the environmental and simulation parameters are first matched in the time dimension. This achieves temporal alignment between the environmental and simulation parameters, ensuring that the actual data at the same moment corresponds to the simulated data, ensuring data consistency and comparability.

[0082] After completing the time alignment, we further obtain corresponding difference indicators based on the actual data and simulation data at each time point, such as relative error, absolute error, etc., to quantify the error degree of the digital twin model, and thus provide a quantitative basis for subsequent model optimization.

[0083] Based on the specific time point, the environmental parameters, simulation parameters, and corresponding difference indicators at that time are further integrated into a single data record. Under the MBPO architecture, the yogurt production control in the digital twin model acts as an intelligent agent, adjusting the mixing parameters of the mixing equipment according to preset rules or initial strategies. Rewards are then given based on the deviation between the final yogurt viscosity and the preset viscosity value after the parameter adjustments. If the simulation parameters at a certain time point meet the corresponding preset rules or initial strategies, a parameter adjustment control action will be initiated, and the corresponding control action and reward calculation results will also need to be included in the training dataset.

[0084] Taking absolute error as the difference indicator, the environmental parameter b at time point a = {viscosity , shear rate ,temperature }, simulation parameter c={viscosity , shear rate ,temperature }, difference index d = {viscosity error , shear rate error , temperature error }, where the viscosity error , shear rate error , temperature error Then, if there is a simulation parameter at time point a that satisfies the preset rule or initial strategy, and a stirring control strategy is proposed for control action B that increases the stirring speed, the digital twin model predicts the reward points associated with control action B under the corresponding simulation parameters and the reward points under the corresponding actual environment parameters, and then calculates the corresponding reward point error d. The data record A = {a, b, c, d, B, d} constituted by the data at time point a.

[0085] The integrated data records are then sorted chronologically to construct a training dataset. To improve generalization capabilities, the data can also be preprocessed by normalization and standardization to make the data distribution more consistent with the requirements of neural network training.

[0086] In the MBPO framework, the environmental model neural network used must have the dual functions of simulating environmental dynamics and guiding strategy optimization. Therefore, this embodiment specifically uses a neural network architecture that combines LSTM and a fully connected layer to establish an environmental model neural network, and combines the collected environmental parameters and simulation parameter combinations for training. Rewards are used to guide the direction of neural network learning to accurately predict the optimal parameter combination of the digital twin model.

[0087] Generate a training data set for the environmental model neural network. After completing the training of the environmental model neural network, the digital twin model can be corrected based on the currently collected time series data set through the trained environmental model neural network to ensure that the digital twin model can be close to the actual production situation of the yogurt demulsification process.

[0088] Specifically, the time series data set is first input into the environment model neural network, and the environment model neural network outputs the correction parameters;

[0089] A correction strategy for generating digital twin models based on correction parameters.

[0090] After receiving a time-series dataset, the environment model neural network calculates using its internally trained parameters and complex network structure to determine simulation parameters that yield higher rewards under the corresponding environmental parameters. It then outputs correction parameters based on these simulation parameters. These correction parameters primarily target the parameters of the virtual environment used to train the digital twin model. By adjusting these virtual environment parameters, the model parameters of the trained digital twin model can be corrected.

[0091] The correction parameters output by the environmental model neural network are usually in the form of multidimensional vectors. Therefore, the correction strategy for generating a digital twin model based on the correction parameters includes:

[0092] Identify the correction parameter type and associate it with the model parameters of the digital twin model;

[0093] The correction target parameters are determined based on the association matching results, and a correction strategy for the correction target parameters is generated according to the correction parameter values output by the environmental model neural network.

[0094] Specifically, the type of correction parameter can be analyzed using predefined parameter tags, and then the model parameters related to the correction parameter type can be determined through association matching. The association matching can be achieved by calculating the correlation between the correction parameter type and the model parameter. By screening the correlation exceeding the corresponding threshold, the model parameters with strong correlation with the corresponding parameter type are screened out. The screened model parameters are used as the correction target parameters, and then the optimal combination of each correction target parameter is determined through the corresponding correction parameter values using an optimization algorithm such as an optimal path search algorithm.

[0095] The digital twin model is adjusted according to the optimal combination of the determined corrected target parameters, and the yogurt demulsification process is re-simulated based on the corrected digital twin model, so that the yogurt stirring control strategy is adjusted in time according to the preset viscosity value.

[0096] Specifically, the stirring control strategy is generated based on the modified digital twin model and the preset viscosity value, including:

[0097] Output viscosity change curve based on the revised digital twin model;

[0098] Obtain viscosity adjustment requirements based on preset viscosity values and viscosity change curves;

[0099] Generates agitation control strategies based on viscosity adjustment requirements.

[0100] Under the adjusted virtual environment training, the revised digital twin model has higher simulation accuracy and re-outputs the viscosity change curve based on the execution of the current stirring control strategy.

[0101] The preset target yogurt viscosity value is then compared with the viscosity curve output by the digital twin model. Using difference calculation and trend prediction algorithms, the simulated viscosity change is evaluated to determine whether it meets production requirements. For example, if the viscosity curve shows the final viscosity is lower than the preset value, the difference between the two values can be further calculated and the viscosity trend analyzed to determine at which stage of the demulsification process the deviation caused the final viscosity to fall short of the target. This allows for the identification of viscosity adjustments required at the corresponding stage.

[0102] Based on the viscosity adjustment requirements and the influence of stirring parameters on yogurt viscosity, a stirring control strategy is developed. Stirring parameters include stirring speed, stirring time, and stirring blade angle, and different parameters affect viscosity to varying degrees and in varying ways. Specifically, a model correlating stirring parameters with viscosity changes, trained using historical production data and simulation experimental data, is used to calculate the stirring control strategy required to achieve the preset viscosity value.

[0103] Another aspect of this embodiment provides a yogurt viscosity control device based on reinforcement learning, including:

[0104] A data acquisition module is provided at the stirring device to collect real-time environmental parameters of the stirred yogurt during the demulsification process;

[0105] The edge processing unit is connected to the data acquisition device for generating a stirring control strategy based on the collected environmental parameters through a digital twin model and reinforcement learning algorithm, and adjusting the stirring parameters of the stirring equipment.

[0106] The stirring equipment used in the stirred yogurt demulsification process mainly includes a stirring tank and a stirring paddle and a baffle arranged in the stirring tank. Among them, the stirring tank is generally equipped with a jacket for heating or cooling, and the stirring paddle needs to cooperate with the drive motor to work. The motor drives the stirring paddle to rotate, so that the sodium acid material in the stirring tank flows and mixes. The stirring intensity can be adjusted to different degrees by controlling parameters such as the speed of the drive motor, the position of the baffle, and the temperature of the fermentation tank to achieve the purpose of adjusting the viscosity of the yogurt.

[0107] Based on the existing stirring equipment, a data acquisition module is set up to collect different environmental parameters related to yogurt viscosity during the yogurt demulsification process. Then, through the edge processing unit, the viscosity change analysis of the stirred yogurt in the fermentation tank is realized according to the corresponding environmental parameters, so as to output the stirring control strategy of the stirring equipment and achieve precise control of the yogurt viscosity.

[0108] Wherein, the data acquisition module at least includes:

[0109] The torque sensor is installed between the stirring paddle and the driving motor of the stirring device to collect the viscous resistance torque of the stirring paddle;

[0110] An angle sensor is provided between the stirring paddle and the driving motor of the stirring device to detect the rotation angle of the stirring paddle;

[0111] The temperature sensor is installed in the fermentation tank of the stirring equipment and is used to collect temperature data during the demulsification process.

[0112] The configuration of the data acquisition module on the fermentation tank is as follows: Figure 2 shown.

[0113] In order to process the collected data, the data collection module further includes:

[0114] The data processing unit is connected to the torque sensor and the angle sensor respectively, and is used to calculate the viscosity data and shear rate data of the yogurt according to the collected viscous resistance torque and rotation angle of the stirring paddle.

[0115] In order to simplify the device, for the torque sensor and angle sensor, a shaft end torque sensor that can simultaneously measure the angle and torque can be used to achieve simultaneous acquisition of the two parameters.

[0116] The data processing unit may be a single chip microcomputer, which may process and calculate the data collected by the torque sensor and the angle sensor according to a preset program to obtain the desired shear rate and viscosity data.

[0117] In order to more accurately obtain the shear rate and viscosity data at the initial stage of demulsification, the torque sensor and angle sensor must be set to a sampling frequency of no less than 100 Hz. Therefore, when establishing data interaction, it will not be transferred through the PLC, but will be connected to the microcontroller through the 485 bus and then converted to an Ethernet connection edge processing unit.

[0118] Temperature data generally does not fluctuate significantly and can be collected at a frequency of 0.5 Hz to 1 Hz. Therefore, for data interaction between the temperature sensor and the edge processing unit, a PLC Ethernet interface can be used to connect the edge processing unit.

[0119] The overall system topology of the yogurt viscosity control device is as follows Figure 3 shown.

[0120] The edge processing unit, equipped with digital twin technology and a reinforcement learning algorithm based on the MBPO framework, can build a digital model of the demulsification process based on digital twin technology. This digital twin model is primarily based on the yogurt fluid mechanics sub-model, and is constructed in conjunction with other physical field sub-models. Before building the physical field sub-model, a three-dimensional model of the fermentation tank and agitator, as well as a digital model of the drive motor, must be established as the virtual training environment foundation for the digital twin model. This foundation is then used to complete the coupling of all physical field sub-models. The digital twin model also includes an initial stirring control strategy for yogurt viscosity control. This strategy can be set based on historical data or operational experience, and its parameters are adjustable.

[0121] The digital model of the drive motor can be constructed using a CFD simulation algorithm to calculate the digital change in motor torque under the action of the simulated motor. After determining the stirring control strategy, the edge processing unit can adjust the specific stirring parameters by modulating the frequency of the drive motor. The edge processing unit sends the corresponding frequency modulation instructions to the drive motor via the PLC interface. Therefore, when building the corresponding virtual environment, it is necessary to further construct a digital model of the corresponding frequency converter.

[0122] Specifically, the virtual training environment for yogurt viscosity control constructed by the edge processing unit is as follows: Figure 4 shown.

[0123] Considering that traditional simulation environment adjustments require calling CFD system calculations, which has a large amount of calculations, the edge processing unit is equipped with the MBPO framework and simultaneously builds an environmental model neural network to adjust the virtual training environment according to the actual environment and the simulated environment, optimize the digital twin model, and thus optimize the stirring control strategy.

[0124] The optimization and adjustment framework of the edge processing unit for the environmental model neural network and digital twin model under the MBPO framework is as follows: Figure 5 shown.

[0125] The embodiment described above is only a preferred solution of the present invention and does not limit the present invention in any form. Other variations and modifications are possible without exceeding the technical solution described in the claims.

Claims

1. A yogurt viscosity control method based on reinforcement learning, characterized in that: include: Establish a digital twin model of the demulsification process; Among them, the equipment parameters and historical production data of the stirring equipment in the demulsification process are obtained; Create and initialize a multi-dimensional sub-model of the demulsification process based on equipment parameters and historical production data; The multidimensional sub-models are coupled and their parameters are calibrated and optimized using historical production data to obtain a digital twin model of the demulsification process. Real-time collection of environmental parameters during the demulsification process and construction of a time series data set based on simulation parameters output by the digital twin model; Based on reinforcement learning algorithms, the digital twin model is modified according to the time series data set; Among them, the acquired environmental parameters and simulation parameters are time-aligned, and the corresponding difference indicators are obtained; Construct a training dataset based on the time-aligned environmental parameters, simulation parameters, and corresponding difference indicators; Establish an environment model neural network and train the environment model neural network based on the training data set; Based on the modified digital twin model, the stirring control strategy is generated in combination with the preset viscosity value; Among them, the viscosity change curve is output based on the modified digital twin model; Obtain viscosity adjustment requirements based on preset viscosity values and viscosity change curves; Generate stirring control strategy based on viscosity adjustment requirements; Adjust the stirring parameters of the demulsification process according to the stirring control strategy; The environmental parameters include viscosity data, shear rate data and temperature data of the yogurt.

2. The yogurt viscosity control method based on reinforcement learning according to claim 1, characterized in that The environmental model neural network obtains a correction strategy based on the time series data set, including: The time series data set is input into the environment model neural network, and the environment model neural network outputs the correction parameters; A correction strategy for generating digital twin models based on correction parameters.

3. The yogurt viscosity control method based on reinforcement learning according to claim 2, characterized in that: The correction strategy for generating a digital twin model based on correction parameters includes: Identify the correction parameter type and associate it with the model parameters of the digital twin model; The correction target parameters are determined based on the association matching results, and a correction strategy for the correction target parameters is generated according to the correction parameter values output by the environmental model neural network.

4. A yogurt viscosity control device based on reinforcement learning, configured to execute the yogurt viscosity control method based on reinforcement learning according to any one of claims 1 to 3, characterized in that: include: The data acquisition module is set at the stirring device and is used to collect the environmental parameters of the yogurt during the demulsification process in real time; The edge processing unit is connected to the data acquisition device for generating a stirring control strategy through a digital twin model and reinforcement learning algorithm in combination with the collected environmental parameters, and adjusting the stirring parameters of the stirring equipment during the demulsification process.

5. The yogurt viscosity control device based on reinforcement learning according to claim 4, characterized in that: The data acquisition module at least includes: The torque sensor is installed between the stirring paddle and the driving motor of the stirring device to collect the viscous resistance torque of the stirring paddle; An angle sensor is provided between the stirring paddle and the driving motor of the stirring device to detect the rotation angle of the stirring paddle; The temperature sensor is installed in the fermentation tank of the stirring equipment and is used to collect temperature data during the demulsification process.

6. The yogurt viscosity control device based on reinforcement learning according to claim 4, characterized in that: The data acquisition module also includes: The data processing unit is connected to the torque sensor and the angle sensor respectively, and is used to calculate the viscosity data and shear rate data of the yogurt according to the collected viscous resistance torque and rotation angle of the stirring paddle.

Images